2023-03-29T10:39:06.787 INFO:root:teuthology version: 0.0.1.dev78+ge372916 2023-03-29T10:39:06.787 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev} --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2023-03-29_10:31:35-fs-wip-vshankar-testing-20230317.095222-testing-default-smithi/7225143 --name vshankar-2023-03-29_10:31:35-fs-wip-vshankar-testing-20230317.095222-testing-default-smithi --verbose -- /home/teuthworker/archive/vshankar-2023-03-29_10:31:35-fs-wip-vshankar-testing-20230317.095222-testing-default-smithi/7225143/orig.config.yaml 2023-03-29T10:39:06.841 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-03-29T10:39:06.921 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-03-29_10:31:35-fs-wip-vshankar-testing-20230317.095222-testing-default-smithi/7225143 branch: wip-vshankar-testing-20230317.095222 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev} email: vshankar@redhat.com first_in_suite: false job_id: '7225143' kernel: flavor: default kdb: true sha1: 1776e8f494e94fd3414d7679e5782ec22e83c95f last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-03-29_10:31:35-fs-wip-vshankar-testing-20230317.095222-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-vshankar-testing-20230317.095222 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: f03d69a8741c605ae6711afe2c9991ac46095f7d ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: f03d69a8741c605ae6711afe2c9991ac46095f7d selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20230317.095222 sha1: f03d69a8741c605ae6711afe2c9991ac46095f7d owner: scheduled_vshankar@teuthology priority: 30 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: f03d69a8741c605ae6711afe2c9991ac46095f7d sleep_before_teardown: 0 suite: fs suite_branch: wip-vshankar-testing-20230317.095222 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_f03d69a8741c605ae6711afe2c9991ac46095f7d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: f03d69a8741c605ae6711afe2c9991ac46095f7d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi111.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDlY8v0fYkp8mXTemAm3+xgrF3RHMFPpqbu5eVeI11m/QA3a2Cn4kO8mS7G4jEaRIghy7Gqk3mSZ0Eb2C/J9Yk5MeS70I5kzk7QdKDfydRx+fB81rS9SldGQrHCkut00Vb66ZrdV4X5Hcnmd7CNSn8VXcY/l3GikDCuejWB4I2nzhoaXIxBS6pha2wsovbi5gr8TxKEAfv6tp/bZ9Ypty5Nb565FDyhGY2dmotiyFSiWV7xJIBbfBSbW53ueYcx7KzEwuhjL76oy942Uqs9VWfHQAO0/Ge0gyDj1B+Gp8MCndRlFojL2uezsu/Qlm1bKj3laNv7Roij5FUfPe/ROxxJ2qT3sOPNaExi0WrDNfYIl9Zlu9KCTXtM9OgMtoFy4HtJMTsu4JtxIEgND/rRrCUEwzpZR3sW2FsJNrq/ETslEc4dDDDrra3B3yg3ryvJG0la4+0YVdpx1MQG6Y/N1ef1/vxSALcmobQAV4XFIs64gNcyX7c2Q4xBCvRP90EOQEE= smithi119.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6r7aIQgpyhdYAjjhoFwotCukZ0xwUNAjXcC7AbsOUpeSzdaPbKFC8vx7JqTb3OKG6g9Gs7JinOu4o5otVVEMz7zMUeoTmAj2r8ZD9fuLP85rvPgyHWaQLx/uyPXlZSw9kU4/MWNspML5fmhvx3WJ2J6poT9LWZ5nzuzGLm6QWhU9kNffj2xvxumMOxzXij55J+2UYvckDruUrH1f4NQkmOEJ+CqmgpcdBRW0hD2SfebaUiZ8KmUQH6QYPG4Sxj5a1nSH+jNl8EZUon0P2v8jOYtclslXZFBz1FhZ8Q8omf3UVWgoseNoLigazJMTCxNL96RuakWCYzIPUw2lzvwVwNVI/tLKFAuu3utmaywOh3t+qC1QBslB7bT9oIt09D+0rXLplxZZIqba/5JrjIGb3HlxuHdiKEQ82ABzztub6rZHVyCG1JeFmKlhVlg5ZPAr9FeClwisrmr9TXvLGAyRy2nF6gT+yvDAo5gOG8J45qylWCO2x5/7ZxiJ1Qi4ntJk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.tests_from_xfstests_dev teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: e3729161eecf8e6d09a4069acbb883cdcd5e9f50 timestamp: 2023-03-29_10:31:35 tube: smithi use_shaman: true user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.803 2023-03-29T10:39:06.921 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_f03d69a8741c605ae6711afe2c9991ac46095f7d/qa; will attempt to use it 2023-03-29T10:39:06.922 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_f03d69a8741c605ae6711afe2c9991ac46095f7d/qa/tasks 2023-03-29T10:39:06.922 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-03-29T10:39:06.924 INFO:teuthology.task.internal:Checking packages... 2023-03-29T10:39:06.952 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f03d69a8741c605ae6711afe2c9991ac46095f7d' 2023-03-29T10:39:06.952 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-03-29T10:39:06.952 INFO:teuthology.packaging:ref: None 2023-03-29T10:39:06.952 INFO:teuthology.packaging:tag: None 2023-03-29T10:39:06.952 INFO:teuthology.packaging:branch: wip-vshankar-testing-20230317.095222 2023-03-29T10:39:06.952 INFO:teuthology.packaging:sha1: f03d69a8741c605ae6711afe2c9991ac46095f7d 2023-03-29T10:39:06.952 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-vshankar-testing-20230317.095222 2023-03-29T10:39:07.098 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-3066.gf03d69a8 2023-03-29T10:39:07.098 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-03-29T10:39:07.105 INFO:teuthology.task.internal:no buildpackages task found 2023-03-29T10:39:07.105 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-03-29T10:39:07.136 INFO:teuthology.task.internal:Saving configuration 2023-03-29T10:39:07.154 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-03-29T10:39:07.160 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-03-29T10:39:07.187 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi111.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-03-29_10:31:35-fs-wip-vshankar-testing-20230317.095222-testing-default-smithi/7225143', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-03-29 10:33:32.117641', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM8ipQ0KUk1v7Ly6MJgadjXr1xwpmVjnZhjsw1QZBZCOF6d7zvwpb+eekJWckuwz7sVZPANxe8jK16Eye0n9tJg='} 2023-03-29T10:39:07.216 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-03-29_10:31:35-fs-wip-vshankar-testing-20230317.095222-testing-default-smithi/7225143', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-03-29 10:33:32.116063', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOBzBT00de0gMib4PlgY9ynA8OXmjAVmCWg1BUDPa5nktfjMHfz6ojh+pDv6JVT2ZQ6Ov7XOvHNA+yUWbX0r8+s='} 2023-03-29T10:39:07.216 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-03-29T10:39:07.223 INFO:teuthology.task.internal:roles: ubuntu@smithi111.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-03-29T10:39:07.223 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-03-29T10:39:07.223 INFO:teuthology.run_tasks:Running task console_log... 2023-03-29T10:39:07.301 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8a65de6e50>, signals=[15]) 2023-03-29T10:39:07.301 INFO:teuthology.run_tasks:Running task internal.connect... 2023-03-29T10:39:07.308 INFO:teuthology.task.internal:Opening connections... 2023-03-29T10:39:07.308 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi111.front.sepia.ceph.com 2023-03-29T10:39:07.309 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-29T10:39:07.376 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2023-03-29T10:39:07.377 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-29T10:39:07.445 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-03-29T10:39:07.451 DEBUG:teuthology.orchestra.run.smithi111:> uname -m 2023-03-29T10:39:07.471 INFO:teuthology.orchestra.run.smithi111.stdout:x86_64 2023-03-29T10:39:07.472 DEBUG:teuthology.orchestra.run.smithi111:> cat /etc/os-release 2023-03-29T10:39:07.530 INFO:teuthology.orchestra.run.smithi111.stdout:NAME="CentOS Stream" 2023-03-29T10:39:07.530 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION="8" 2023-03-29T10:39:07.530 INFO:teuthology.orchestra.run.smithi111.stdout:ID="centos" 2023-03-29T10:39:07.531 INFO:teuthology.orchestra.run.smithi111.stdout:ID_LIKE="rhel fedora" 2023-03-29T10:39:07.531 INFO:teuthology.orchestra.run.smithi111.stdout:VERSION_ID="8" 2023-03-29T10:39:07.531 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM_ID="platform:el8" 2023-03-29T10:39:07.531 INFO:teuthology.orchestra.run.smithi111.stdout:PRETTY_NAME="CentOS Stream 8" 2023-03-29T10:39:07.531 INFO:teuthology.orchestra.run.smithi111.stdout:ANSI_COLOR="0;31" 2023-03-29T10:39:07.531 INFO:teuthology.orchestra.run.smithi111.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-03-29T10:39:07.531 INFO:teuthology.orchestra.run.smithi111.stdout:HOME_URL="https://centos.org/" 2023-03-29T10:39:07.531 INFO:teuthology.orchestra.run.smithi111.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-03-29T10:39:07.531 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-03-29T10:39:07.531 INFO:teuthology.orchestra.run.smithi111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-03-29T10:39:07.532 INFO:teuthology.lock.ops:Updating smithi111.front.sepia.ceph.com on lock server 2023-03-29T10:39:07.559 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2023-03-29T10:39:07.579 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2023-03-29T10:39:07.579 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2023-03-29T10:39:07.638 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="CentOS Stream" 2023-03-29T10:39:07.638 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="8" 2023-03-29T10:39:07.638 INFO:teuthology.orchestra.run.smithi119.stdout:ID="centos" 2023-03-29T10:39:07.638 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="rhel fedora" 2023-03-29T10:39:07.638 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="8" 2023-03-29T10:39:07.638 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el8" 2023-03-29T10:39:07.639 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="CentOS Stream 8" 2023-03-29T10:39:07.639 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2023-03-29T10:39:07.639 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-03-29T10:39:07.639 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://centos.org/" 2023-03-29T10:39:07.639 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-03-29T10:39:07.639 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-03-29T10:39:07.639 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-03-29T10:39:07.639 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2023-03-29T10:39:07.659 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-03-29T10:39:07.669 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-03-29T10:39:07.676 INFO:teuthology.task.internal:Checking for old test directory... 2023-03-29T10:39:07.676 DEBUG:teuthology.orchestra.run.smithi111:> test '!' -e /home/ubuntu/cephtest 2023-03-29T10:39:07.678 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2023-03-29T10:39:07.695 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-03-29T10:39:07.702 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-03-29T10:39:07.702 DEBUG:teuthology.orchestra.run.smithi111:> test -z $(ls -A /var/lib/ceph) 2023-03-29T10:39:07.736 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2023-03-29T10:39:07.765 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-03-29T10:39:07.831 INFO:teuthology.run_tasks:Running task kernel... 2023-03-29T10:39:07.854 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '1776e8f494e94fd3414d7679e5782ec22e83c95f'} 2023-03-29T10:39:07.854 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '1776e8f494e94fd3414d7679e5782ec22e83c95f'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '1776e8f494e94fd3414d7679e5782ec22e83c95f'}}, timeout 300 2023-03-29T10:39:07.854 DEBUG:teuthology.orchestra.run.smithi111:> test -f /run/.containerenv -o -f /.dockerenv 2023-03-29T10:39:07.855 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2023-03-29T10:39:07.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T10:39:07.873 DEBUG:teuthology.task.kernel:sha1 for mon.b is 1776e8f494e94fd3414d7679e5782ec22e83c95f 2023-03-29T10:39:07.873 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "1776e8f494e94fd3414d7679e5782ec22e83c95f"... 2023-03-29T10:39:07.874 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2023-03-29T10:39:07.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T10:39:07.875 DEBUG:teuthology.task.kernel:sha1 for mon.a is 1776e8f494e94fd3414d7679e5782ec22e83c95f 2023-03-29T10:39:07.875 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "1776e8f494e94fd3414d7679e5782ec22e83c95f"... 2023-03-29T10:39:07.875 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2023-03-29T10:39:07.932 INFO:teuthology.orchestra.run.smithi119.stdout:4.18.0-408.el8.x86_64 2023-03-29T10:39:07.932 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-408.el8.x86_64 vs 1776e8f494e94fd3414d7679e5782ec22e83c95f 2023-03-29T10:39:07.932 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-03-29T10:39:07.932 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1776e8f494e94fd3414d7679e5782ec22e83c95f 2023-03-29T10:39:07.935 INFO:teuthology.orchestra.run.smithi111.stdout:4.18.0-408.el8.x86_64 2023-03-29T10:39:07.935 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-408.el8.x86_64 vs 1776e8f494e94fd3414d7679e5782ec22e83c95f 2023-03-29T10:39:07.936 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-03-29T10:39:07.936 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1776e8f494e94fd3414d7679e5782ec22e83c95f 2023-03-29T10:39:08.070 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum upgrade -y linux-firmware 2023-03-29T10:39:08.077 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum upgrade -y linux-firmware 2023-03-29T10:39:11.614 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-8 - AppStream 34 MB/s | 29 MB 00:00 2023-03-29T10:39:11.837 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS-8 - AppStream 30 MB/s | 29 MB 00:00 2023-03-29T10:39:17.314 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-8 - BaseOS 44 MB/s | 28 MB 00:00 2023-03-29T10:39:17.687 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS-8 - BaseOS 40 MB/s | 28 MB 00:00 2023-03-29T10:39:21.613 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-8 - Extras 158 kB/s | 18 kB 00:00 2023-03-29T10:39:22.024 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-8 - PowerTools 21 MB/s | 5.8 MB 00:00 2023-03-29T10:39:22.069 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS-8 - Extras 153 kB/s | 18 kB 00:00 2023-03-29T10:39:22.479 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS-8 - PowerTools 21 MB/s | 5.8 MB 00:00 2023-03-29T10:39:23.256 INFO:teuthology.orchestra.run.smithi119.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-03-29T10:39:23.728 INFO:teuthology.orchestra.run.smithi111.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2023-03-29T10:39:24.440 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 13 MB/s | 14 MB 00:01 2023-03-29T10:39:24.971 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 12 MB/s | 14 MB 00:01 2023-03-29T10:39:27.840 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 430 kB/s | 24 kB 00:00 2023-03-29T10:39:28.379 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 479 kB/s | 24 kB 00:00 2023-03-29T10:39:31.301 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-03-29T10:39:31.301 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-03-29T10:39:31.302 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-03-29T10:39:31.302 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-03-29T10:39:31.302 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2023-03-29T10:39:31.302 INFO:teuthology.orchestra.run.smithi119.stdout: linux-firmware noarch 20230217-113.git83f1d778.el8 CentOS-BaseOS 263 M 2023-03-29T10:39:31.302 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:39:31.302 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-03-29T10:39:31.302 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-03-29T10:39:31.302 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 1 Package 2023-03-29T10:39:31.302 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:39:31.303 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 263 M 2023-03-29T10:39:31.303 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-03-29T10:39:31.836 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2023-03-29T10:39:31.837 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2023-03-29T10:39:31.837 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2023-03-29T10:39:31.837 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2023-03-29T10:39:31.837 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrading: 2023-03-29T10:39:31.837 INFO:teuthology.orchestra.run.smithi111.stdout: linux-firmware noarch 20230217-113.git83f1d778.el8 CentOS-BaseOS 263 M 2023-03-29T10:39:31.837 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:39:31.837 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2023-03-29T10:39:31.837 INFO:teuthology.orchestra.run.smithi111.stdout:================================================================================ 2023-03-29T10:39:31.837 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrade 1 Package 2023-03-29T10:39:31.837 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:39:31.838 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 263 M 2023-03-29T10:39:31.838 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2023-03-29T10:39:36.616 INFO:teuthology.orchestra.run.smithi119.stdout:linux-firmware-20230217-113.git83f1d778.el8.noa 50 MB/s | 263 MB 00:05 2023-03-29T10:39:36.617 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-03-29T10:39:36.617 INFO:teuthology.orchestra.run.smithi119.stdout:Total 50 MB/s | 263 MB 00:05 2023-03-29T10:39:38.181 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-03-29T10:39:38.235 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-03-29T10:39:38.235 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-03-29T10:39:38.793 INFO:teuthology.orchestra.run.smithi111.stdout:linux-firmware-20230217-113.git83f1d778.el8.noa 38 MB/s | 263 MB 00:06 2023-03-29T10:39:38.795 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2023-03-29T10:39:38.795 INFO:teuthology.orchestra.run.smithi111.stdout:Total 38 MB/s | 263 MB 00:06 2023-03-29T10:39:40.354 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2023-03-29T10:39:40.371 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-03-29T10:39:40.399 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-03-29T10:39:40.406 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2023-03-29T10:39:40.406 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2023-03-29T10:39:42.533 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2023-03-29T10:39:42.555 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2023-03-29T10:39:58.447 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-03-29T10:39:58.811 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : linux-firmware-20230217-113.git83f1d778.el8.noarch 1/2 2023-03-29T10:39:59.041 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : linux-firmware-20220726-110.git150864a4.el8.noarch 2/2 2023-03-29T10:39:59.332 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: linux-firmware-20220726-110.git150864a4.el8.noarch 2/2 2023-03-29T10:39:59.332 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : linux-firmware-20230217-113.git83f1d778.el8.noarch 1/2 2023-03-29T10:40:00.143 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : linux-firmware-20220726-110.git150864a4.el8.noarch 2/2 2023-03-29T10:40:00.143 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:40:00.143 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2023-03-29T10:40:00.143 INFO:teuthology.orchestra.run.smithi119.stdout: linux-firmware-20230217-113.git83f1d778.el8.noarch 2023-03-29T10:40:00.143 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:40:00.143 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-03-29T10:40:00.303 INFO:teuthology.task.kernel:Downloading kernel 1776e8f494e94fd3414d7679e5782ec22e83c95f on mon.b... 2023-03-29T10:40:00.303 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1776e8f494e94fd3414d7679e5782ec22e83c95f 2023-03-29T10:40:00.444 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/1776e8f494e94fd3414d7679e5782ec22e83c95f/centos/8/flavors/default/x86_64/ 2023-03-29T10:40:00.445 DEBUG:teuthology.orchestra.run.smithi119:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.3.0_rc3_g1776e8f494e9-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/1776e8f494e94fd3414d7679e5782ec22e83c95f/centos/8/flavors/default/x86_64/ --input-file=- 2023-03-29T10:40:00.447 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi119.front.sepia.ceph.com to complete... 2023-03-29T10:40:00.664 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2023-03-29T10:40:01.072 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : linux-firmware-20230217-113.git83f1d778.el8.noarch 1/2 2023-03-29T10:40:01.323 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : linux-firmware-20220726-110.git150864a4.el8.noarch 2/2 2023-03-29T10:40:01.626 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: linux-firmware-20220726-110.git150864a4.el8.noarch 2/2 2023-03-29T10:40:01.627 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : linux-firmware-20230217-113.git83f1d778.el8.noarch 1/2 2023-03-29T10:40:02.257 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : linux-firmware-20220726-110.git150864a4.el8.noarch 2/2 2023-03-29T10:40:02.258 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:40:02.258 INFO:teuthology.orchestra.run.smithi111.stdout:Upgraded: 2023-03-29T10:40:02.258 INFO:teuthology.orchestra.run.smithi111.stdout: linux-firmware-20230217-113.git83f1d778.el8.noarch 2023-03-29T10:40:02.258 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:40:02.258 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2023-03-29T10:40:02.395 INFO:teuthology.task.kernel:Downloading kernel 1776e8f494e94fd3414d7679e5782ec22e83c95f on mon.a... 2023-03-29T10:40:02.395 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1776e8f494e94fd3414d7679e5782ec22e83c95f 2023-03-29T10:40:02.533 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/1776e8f494e94fd3414d7679e5782ec22e83c95f/centos/8/flavors/default/x86_64/ 2023-03-29T10:40:02.533 DEBUG:teuthology.orchestra.run.smithi111:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.3.0_rc3_g1776e8f494e9-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/1776e8f494e94fd3414d7679e5782ec22e83c95f/centos/8/flavors/default/x86_64/ --input-file=- 2023-03-29T10:40:02.535 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi111.front.sepia.ceph.com to complete... 2023-03-29T10:40:17.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29 10:40:17 URL:https://2.chacra.ceph.com/r/kernel/testing/1776e8f494e94fd3414d7679e5782ec22e83c95f/centos/8/flavors/default/x86_64/kernel-6.3.0_rc3_g1776e8f494e9-1.x86_64.rpm [377992976/377992976] -> "/tmp/kernel.x86_64.rpm" [1] 2023-03-29T10:40:17.422 INFO:teuthology.orchestra.run.smithi119.stderr:FINISHED --2023-03-29 10:40:17-- 2023-03-29T10:40:17.422 INFO:teuthology.orchestra.run.smithi119.stderr:Total wall clock time: 17s 2023-03-29T10:40:17.422 INFO:teuthology.orchestra.run.smithi119.stderr:Downloaded: 1 files, 360M in 17s (21.6 MB/s) 2023-03-29T10:40:17.423 INFO:teuthology.task.kernel:Installing kernel 1776e8f494e94fd3414d7679e5782ec22e83c95f on mon.b... 2023-03-29T10:40:17.423 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-03-29T10:40:19.054 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying... ######################################## 2023-03-29T10:40:19.470 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing... ######################################## 2023-03-29T10:40:19.569 INFO:teuthology.orchestra.run.smithi119.stdout:Updating / installing... 2023-03-29T10:40:25.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29 10:40:25 URL:https://2.chacra.ceph.com/r/kernel/testing/1776e8f494e94fd3414d7679e5782ec22e83c95f/centos/8/flavors/default/x86_64/kernel-6.3.0_rc3_g1776e8f494e9-1.x86_64.rpm [377992976/377992976] -> "/tmp/kernel.x86_64.rpm" [1] 2023-03-29T10:40:25.624 INFO:teuthology.orchestra.run.smithi111.stderr:FINISHED --2023-03-29 10:40:25-- 2023-03-29T10:40:25.624 INFO:teuthology.orchestra.run.smithi111.stderr:Total wall clock time: 23s 2023-03-29T10:40:25.624 INFO:teuthology.orchestra.run.smithi111.stderr:Downloaded: 1 files, 360M in 23s (15.9 MB/s) 2023-03-29T10:40:25.625 INFO:teuthology.task.kernel:Installing kernel 1776e8f494e94fd3414d7679e5782ec22e83c95f on mon.a... 2023-03-29T10:40:25.625 DEBUG:teuthology.orchestra.run.smithi111:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-03-29T10:40:27.252 INFO:teuthology.orchestra.run.smithi111.stdout:Verifying... ######################################## 2023-03-29T10:40:27.706 INFO:teuthology.orchestra.run.smithi111.stdout:Preparing... ######################################## 2023-03-29T10:40:27.806 INFO:teuthology.orchestra.run.smithi111.stdout:Updating / installing... 2023-03-29T10:40:57.237 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-6.3.0_rc3_g1776e8f494e9-1 ######################################## 2023-03-29T10:41:05.395 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-6.3.0_rc3_g1776e8f494e9-1 ######################################## 2023-03-29T10:41:23.861 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi119.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-03-29T10:41:23.861 DEBUG:teuthology.orchestra.run.smithi119:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-03-29T10:41:23.914 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/System.map-6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:23.914 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/config-6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:23.914 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/vmlinuz-6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:23.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:23.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel 2023-03-29T10:41:23.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch 2023-03-29T10:41:23.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86 2023-03-29T10:41:23.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto 2023-03-29T10:41:23.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-03-29T10:41:23.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-03-29T10:41:23.914 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-03-29T10:41:23.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-03-29T10:41:23.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-03-29T10:41:23.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-03-29T10:41:23.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-03-29T10:41:23.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/crc32c-intel.ko 2023-03-29T10:41:23.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-03-29T10:41:23.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-03-29T10:41:23.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-03-29T10:41:23.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-03-29T10:41:23.915 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/serpent-avx2.ko 2023-03-29T10:41:23.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-03-29T10:41:23.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-03-29T10:41:23.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-03-29T10:41:23.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-03-29T10:41:23.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-03-29T10:41:23.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-03-29T10:41:23.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-03-29T10:41:23.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kernel 2023-03-29T10:41:23.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kernel/cpu 2023-03-29T10:41:23.916 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kernel/cpu/mce 2023-03-29T10:41:23.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-03-29T10:41:23.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kvm 2023-03-29T10:41:23.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kvm/kvm-amd.ko 2023-03-29T10:41:23.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kvm/kvm-intel.ko 2023-03-29T10:41:23.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kvm/kvm.ko 2023-03-29T10:41:23.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto 2023-03-29T10:41:23.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/ansi_cprng.ko 2023-03-29T10:41:23.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/anubis.ko 2023-03-29T10:41:23.917 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/arc4.ko 2023-03-29T10:41:23.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx 2023-03-29T10:41:23.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx/async_memcpy.ko 2023-03-29T10:41:23.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx/async_pq.ko 2023-03-29T10:41:23.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx/async_raid6_recov.ko 2023-03-29T10:41:23.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx/async_tx.ko 2023-03-29T10:41:23.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx/async_xor.ko 2023-03-29T10:41:23.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx/raid6test.ko 2023-03-29T10:41:23.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/authenc.ko 2023-03-29T10:41:23.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/authencesn.ko 2023-03-29T10:41:23.918 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/blake2b_generic.ko 2023-03-29T10:41:23.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/blowfish_common.ko 2023-03-29T10:41:23.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/blowfish_generic.ko 2023-03-29T10:41:23.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/camellia_generic.ko 2023-03-29T10:41:23.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/cast5_generic.ko 2023-03-29T10:41:23.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/cast6_generic.ko 2023-03-29T10:41:23.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/cast_common.ko 2023-03-29T10:41:23.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/ccm.ko 2023-03-29T10:41:23.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/cmac.ko 2023-03-29T10:41:23.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/crc32_generic.ko 2023-03-29T10:41:23.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/crypto_engine.ko 2023-03-29T10:41:23.919 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/crypto_user.ko 2023-03-29T10:41:23.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/cts.ko 2023-03-29T10:41:23.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/des_generic.ko 2023-03-29T10:41:23.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/dh_generic.ko 2023-03-29T10:41:23.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/echainiv.ko 2023-03-29T10:41:23.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/essiv.ko 2023-03-29T10:41:23.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/fcrypt.ko 2023-03-29T10:41:23.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/khazad.ko 2023-03-29T10:41:23.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/lz4.ko 2023-03-29T10:41:23.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/lz4hc.ko 2023-03-29T10:41:23.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/md4.ko 2023-03-29T10:41:23.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/michael_mic.ko 2023-03-29T10:41:23.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/pcbc.ko 2023-03-29T10:41:23.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/pcrypt.ko 2023-03-29T10:41:23.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/rmd160.ko 2023-03-29T10:41:23.920 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/seed.ko 2023-03-29T10:41:23.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/serpent_generic.ko 2023-03-29T10:41:23.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/tcrypt.ko 2023-03-29T10:41:23.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/tea.ko 2023-03-29T10:41:23.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/twofish_common.ko 2023-03-29T10:41:23.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/twofish_generic.ko 2023-03-29T10:41:23.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/vmac.ko 2023-03-29T10:41:23.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/wp512.ko 2023-03-29T10:41:23.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/xcbc.ko 2023-03-29T10:41:23.921 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/xor.ko 2023-03-29T10:41:23.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/xxhash_generic.ko 2023-03-29T10:41:23.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers 2023-03-29T10:41:23.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi 2023-03-29T10:41:23.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/acpi_ipmi.ko 2023-03-29T10:41:23.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/acpi_pad.ko 2023-03-29T10:41:23.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/custom_method.ko 2023-03-29T10:41:23.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/ec_sys.ko 2023-03-29T10:41:23.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/platform_profile.ko 2023-03-29T10:41:23.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/sbs.ko 2023-03-29T10:41:23.922 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/sbshc.ko 2023-03-29T10:41:23.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/video.ko 2023-03-29T10:41:23.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata 2023-03-29T10:41:23.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/acard-ahci.ko 2023-03-29T10:41:23.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/ahci_platform.ko 2023-03-29T10:41:23.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/ata_generic.ko 2023-03-29T10:41:23.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/libahci_platform.ko 2023-03-29T10:41:23.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_acpi.ko 2023-03-29T10:41:23.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_ali.ko 2023-03-29T10:41:23.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_amd.ko 2023-03-29T10:41:23.923 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_artop.ko 2023-03-29T10:41:23.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_atiixp.ko 2023-03-29T10:41:23.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_atp867x.ko 2023-03-29T10:41:23.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_cmd640.ko 2023-03-29T10:41:23.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_cmd64x.ko 2023-03-29T10:41:23.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_cypress.ko 2023-03-29T10:41:23.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_efar.ko 2023-03-29T10:41:23.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_hpt366.ko 2023-03-29T10:41:23.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_hpt37x.ko 2023-03-29T10:41:23.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_hpt3x2n.ko 2023-03-29T10:41:23.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_hpt3x3.ko 2023-03-29T10:41:23.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_it8213.ko 2023-03-29T10:41:23.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_it821x.ko 2023-03-29T10:41:23.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_jmicron.ko 2023-03-29T10:41:23.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_marvell.ko 2023-03-29T10:41:23.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_mpiix.ko 2023-03-29T10:41:23.924 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_netcell.ko 2023-03-29T10:41:23.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_ninja32.ko 2023-03-29T10:41:23.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_ns87410.ko 2023-03-29T10:41:23.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_ns87415.ko 2023-03-29T10:41:23.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_oldpiix.ko 2023-03-29T10:41:23.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_opti.ko 2023-03-29T10:41:23.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_optidma.ko 2023-03-29T10:41:23.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_pcmcia.ko 2023-03-29T10:41:23.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_pdc2027x.ko 2023-03-29T10:41:23.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-03-29T10:41:23.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_piccolo.ko 2023-03-29T10:41:23.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_rdc.ko 2023-03-29T10:41:23.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_sch.ko 2023-03-29T10:41:23.925 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_serverworks.ko 2023-03-29T10:41:23.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_sil680.ko 2023-03-29T10:41:23.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_sis.ko 2023-03-29T10:41:23.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_sl82c105.ko 2023-03-29T10:41:23.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_triflex.ko 2023-03-29T10:41:23.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_via.ko 2023-03-29T10:41:23.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pdc_adma.ko 2023-03-29T10:41:23.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_inic162x.ko 2023-03-29T10:41:23.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_mv.ko 2023-03-29T10:41:23.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_nv.ko 2023-03-29T10:41:23.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_promise.ko 2023-03-29T10:41:23.926 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_qstor.ko 2023-03-29T10:41:23.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_sil.ko 2023-03-29T10:41:23.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_sil24.ko 2023-03-29T10:41:23.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_sis.ko 2023-03-29T10:41:23.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_svw.ko 2023-03-29T10:41:23.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_sx4.ko 2023-03-29T10:41:23.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_uli.ko 2023-03-29T10:41:23.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_via.ko 2023-03-29T10:41:23.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_vsc.ko 2023-03-29T10:41:23.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/auxdisplay 2023-03-29T10:41:23.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/auxdisplay/ks0108.ko 2023-03-29T10:41:23.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/base 2023-03-29T10:41:23.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/base/regmap 2023-03-29T10:41:23.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/base/regmap/regmap-i2c.ko 2023-03-29T10:41:23.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/bcma 2023-03-29T10:41:23.927 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/bcma/bcma.ko 2023-03-29T10:41:23.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block 2023-03-29T10:41:23.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/aoe 2023-03-29T10:41:23.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/aoe/aoe.ko 2023-03-29T10:41:23.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/brd.ko 2023-03-29T10:41:23.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/drbd 2023-03-29T10:41:23.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/drbd/drbd.ko 2023-03-29T10:41:23.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/floppy.ko 2023-03-29T10:41:23.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/loop.ko 2023-03-29T10:41:23.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/mtip32xx 2023-03-29T10:41:23.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-03-29T10:41:23.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/nbd.ko 2023-03-29T10:41:23.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/null_blk 2023-03-29T10:41:23.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/null_blk/null_blk.ko 2023-03-29T10:41:23.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/pktcdvd.ko 2023-03-29T10:41:23.928 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/rbd.ko 2023-03-29T10:41:23.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/virtio_blk.ko 2023-03-29T10:41:23.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/xen-blkback 2023-03-29T10:41:23.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-03-29T10:41:23.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/xen-blkfront.ko 2023-03-29T10:41:23.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/zram 2023-03-29T10:41:23.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/zram/zram.ko 2023-03-29T10:41:23.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char 2023-03-29T10:41:23.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hangcheck-timer.ko 2023-03-29T10:41:23.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hw_random 2023-03-29T10:41:23.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hw_random/amd-rng.ko 2023-03-29T10:41:23.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hw_random/intel-rng.ko 2023-03-29T10:41:23.929 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-03-29T10:41:23.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hw_random/via-rng.ko 2023-03-29T10:41:23.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hw_random/virtio-rng.ko 2023-03-29T10:41:23.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi 2023-03-29T10:41:23.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-03-29T10:41:23.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-03-29T10:41:23.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-03-29T10:41:23.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi/ipmi_si.ko 2023-03-29T10:41:23.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-03-29T10:41:23.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-03-29T10:41:23.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/lp.ko 2023-03-29T10:41:23.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/mwave 2023-03-29T10:41:23.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/mwave/mwave.ko 2023-03-29T10:41:23.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/pcmcia 2023-03-29T10:41:23.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/pcmcia/cm4000_cs.ko 2023-03-29T10:41:23.930 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/pcmcia/cm4040_cs.ko 2023-03-29T10:41:23.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ppdev.ko 2023-03-29T10:41:23.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tlclk.ko 2023-03-29T10:41:23.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm 2023-03-29T10:41:23.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm/tpm.ko 2023-03-29T10:41:23.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm/tpm_atmel.ko 2023-03-29T10:41:23.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm/tpm_infineon.ko 2023-03-29T10:41:23.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm/tpm_nsc.ko 2023-03-29T10:41:23.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm/tpm_tis.ko 2023-03-29T10:41:23.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-03-29T10:41:23.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/uv_mmtimer.ko 2023-03-29T10:41:23.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/virtio_console.ko 2023-03-29T10:41:23.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq 2023-03-29T10:41:23.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-03-29T10:41:23.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-03-29T10:41:23.931 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq/p4-clockmod.ko 2023-03-29T10:41:23.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-03-29T10:41:23.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq/powernow-k8.ko 2023-03-29T10:41:23.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq/speedstep-lib.ko 2023-03-29T10:41:23.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto 2023-03-29T10:41:23.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/ccp 2023-03-29T10:41:23.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-03-29T10:41:23.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/ccp/ccp.ko 2023-03-29T10:41:23.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/padlock-aes.ko 2023-03-29T10:41:23.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/padlock-sha.ko 2023-03-29T10:41:23.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/qat 2023-03-29T10:41:23.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/qat/qat_common 2023-03-29T10:41:23.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/qat/qat_common/intel_qat.ko 2023-03-29T10:41:23.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/qat/qat_dh895xcc 2023-03-29T10:41:23.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-03-29T10:41:23.932 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/virtio 2023-03-29T10:41:23.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-03-29T10:41:23.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dca 2023-03-29T10:41:23.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dca/dca.ko 2023-03-29T10:41:23.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/devfreq 2023-03-29T10:41:23.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-03-29T10:41:23.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dma 2023-03-29T10:41:23.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dma/dw 2023-03-29T10:41:23.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dma/dw/dw_dmac.ko 2023-03-29T10:41:23.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dma/ioat 2023-03-29T10:41:23.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dma/ioat/ioatdma.ko 2023-03-29T10:41:23.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac 2023-03-29T10:41:23.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/amd64_edac.ko 2023-03-29T10:41:23.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/e752x_edac.ko 2023-03-29T10:41:23.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/edac_mce_amd.ko 2023-03-29T10:41:23.933 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i3000_edac.ko 2023-03-29T10:41:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i3200_edac.ko 2023-03-29T10:41:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i5100_edac.ko 2023-03-29T10:41:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i5400_edac.ko 2023-03-29T10:41:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i7300_edac.ko 2023-03-29T10:41:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i7core_edac.ko 2023-03-29T10:41:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i82975x_edac.ko 2023-03-29T10:41:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/ie31200_edac.ko 2023-03-29T10:41:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/sb_edac.ko 2023-03-29T10:41:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/x38_edac.ko 2023-03-29T10:41:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/firmware 2023-03-29T10:41:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/firmware/edd.ko 2023-03-29T10:41:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/firmware/iscsi_ibft.ko 2023-03-29T10:41:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/gpio 2023-03-29T10:41:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/gpio/gpio-ich.ko 2023-03-29T10:41:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/gpio/gpio-viperboard.ko 2023-03-29T10:41:23.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid 2023-03-29T10:41:23.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-appleir.ko 2023-03-29T10:41:23.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-aureal.ko 2023-03-29T10:41:23.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-axff.ko 2023-03-29T10:41:23.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-dr.ko 2023-03-29T10:41:23.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-elecom.ko 2023-03-29T10:41:23.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-elo.ko 2023-03-29T10:41:23.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-emsff.ko 2023-03-29T10:41:23.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-gaff.ko 2023-03-29T10:41:23.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-gt683r.ko 2023-03-29T10:41:23.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-gyration.ko 2023-03-29T10:41:23.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-holtek-kbd.ko 2023-03-29T10:41:23.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-holtek-mouse.ko 2023-03-29T10:41:23.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-holtekff.ko 2023-03-29T10:41:23.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-hyperv.ko 2023-03-29T10:41:23.935 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-icade.ko 2023-03-29T10:41:23.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-keytouch.ko 2023-03-29T10:41:23.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-kye.ko 2023-03-29T10:41:23.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-lcpower.ko 2023-03-29T10:41:23.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-led.ko 2023-03-29T10:41:23.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-lenovo.ko 2023-03-29T10:41:23.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-logitech-dj.ko 2023-03-29T10:41:23.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-03-29T10:41:23.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-multitouch.ko 2023-03-29T10:41:23.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-ortek.ko 2023-03-29T10:41:23.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-penmount.ko 2023-03-29T10:41:23.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-petalynx.ko 2023-03-29T10:41:23.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-picolcd.ko 2023-03-29T10:41:23.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-pl.ko 2023-03-29T10:41:23.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-plantronics.ko 2023-03-29T10:41:23.936 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-primax.ko 2023-03-29T10:41:23.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-rmi.ko 2023-03-29T10:41:23.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-arvo.ko 2023-03-29T10:41:23.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-common.ko 2023-03-29T10:41:23.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-isku.ko 2023-03-29T10:41:23.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-kone.ko 2023-03-29T10:41:23.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-03-29T10:41:23.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-konepure.ko 2023-03-29T10:41:23.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-03-29T10:41:23.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-lua.ko 2023-03-29T10:41:23.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-pyra.ko 2023-03-29T10:41:23.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-ryos.ko 2023-03-29T10:41:23.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-savu.ko 2023-03-29T10:41:23.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat.ko 2023-03-29T10:41:23.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-saitek.ko 2023-03-29T10:41:23.937 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-samsung.ko 2023-03-29T10:41:23.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-sensor-hub.ko 2023-03-29T10:41:23.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-sjoy.ko 2023-03-29T10:41:23.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-sony.ko 2023-03-29T10:41:23.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-speedlink.ko 2023-03-29T10:41:23.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-steelseries.ko 2023-03-29T10:41:23.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-sunplus.ko 2023-03-29T10:41:23.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-thrustmaster.ko 2023-03-29T10:41:23.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-tivo.ko 2023-03-29T10:41:23.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-tmff.ko 2023-03-29T10:41:23.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-topseed.ko 2023-03-29T10:41:23.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-twinhan.ko 2023-03-29T10:41:23.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-uclogic.ko 2023-03-29T10:41:23.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-waltop.ko 2023-03-29T10:41:23.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-wiimote.ko 2023-03-29T10:41:23.938 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-xinmo.ko 2023-03-29T10:41:23.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-zpff.ko 2023-03-29T10:41:23.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-zydacron.ko 2023-03-29T10:41:23.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/uhid.ko 2023-03-29T10:41:23.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/wacom.ko 2023-03-29T10:41:23.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hv 2023-03-29T10:41:23.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hv/hv_balloon.ko 2023-03-29T10:41:23.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hv/hv_utils.ko 2023-03-29T10:41:23.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hv/hv_vmbus.ko 2023-03-29T10:41:23.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon 2023-03-29T10:41:23.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/abituguru.ko 2023-03-29T10:41:23.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/abituguru3.ko 2023-03-29T10:41:23.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/acpi_power_meter.ko 2023-03-29T10:41:23.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ad7414.ko 2023-03-29T10:41:23.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ad7418.ko 2023-03-29T10:41:23.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adc128d818.ko 2023-03-29T10:41:23.939 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adm1025.ko 2023-03-29T10:41:23.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adm1026.ko 2023-03-29T10:41:23.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adm1029.ko 2023-03-29T10:41:23.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adm1031.ko 2023-03-29T10:41:23.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adm9240.ko 2023-03-29T10:41:23.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ads7828.ko 2023-03-29T10:41:23.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adt7410.ko 2023-03-29T10:41:23.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adt7411.ko 2023-03-29T10:41:23.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adt7462.ko 2023-03-29T10:41:23.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adt7470.ko 2023-03-29T10:41:23.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adt7475.ko 2023-03-29T10:41:23.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adt7x10.ko 2023-03-29T10:41:23.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/amc6821.ko 2023-03-29T10:41:23.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/applesmc.ko 2023-03-29T10:41:23.940 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/asb100.ko 2023-03-29T10:41:23.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/asc7621.ko 2023-03-29T10:41:23.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/asus_atk0110.ko 2023-03-29T10:41:23.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/atxp1.ko 2023-03-29T10:41:23.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/coretemp.ko 2023-03-29T10:41:23.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-03-29T10:41:23.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/dme1737.ko 2023-03-29T10:41:23.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ds1621.ko 2023-03-29T10:41:23.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ds620.ko 2023-03-29T10:41:23.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/emc1403.ko 2023-03-29T10:41:23.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/emc6w201.ko 2023-03-29T10:41:23.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/f71805f.ko 2023-03-29T10:41:23.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/f71882fg.ko 2023-03-29T10:41:23.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/f75375s.ko 2023-03-29T10:41:23.941 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/fam15h_power.ko 2023-03-29T10:41:23.942 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/fschmd.ko 2023-03-29T10:41:23.942 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/g760a.ko 2023-03-29T10:41:23.942 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/g762.ko 2023-03-29T10:41:23.942 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/gl518sm.ko 2023-03-29T10:41:23.942 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/gl520sm.ko 2023-03-29T10:41:23.942 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/hwmon-vid.ko 2023-03-29T10:41:23.942 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/i5500_temp.ko 2023-03-29T10:41:23.942 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/i5k_amb.ko 2023-03-29T10:41:23.942 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ibmaem.ko 2023-03-29T10:41:23.942 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ibmpex.ko 2023-03-29T10:41:23.942 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ina209.ko 2023-03-29T10:41:23.942 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ina2xx.ko 2023-03-29T10:41:23.942 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/it87.ko 2023-03-29T10:41:23.942 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/k10temp.ko 2023-03-29T10:41:23.942 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/k8temp.ko 2023-03-29T10:41:23.943 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lineage-pem.ko 2023-03-29T10:41:23.943 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm63.ko 2023-03-29T10:41:23.943 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm73.ko 2023-03-29T10:41:23.943 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm75.ko 2023-03-29T10:41:23.943 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm77.ko 2023-03-29T10:41:23.943 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm78.ko 2023-03-29T10:41:23.943 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm80.ko 2023-03-29T10:41:23.943 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm83.ko 2023-03-29T10:41:23.943 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm85.ko 2023-03-29T10:41:23.943 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm87.ko 2023-03-29T10:41:23.943 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm90.ko 2023-03-29T10:41:23.943 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm92.ko 2023-03-29T10:41:23.943 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm93.ko 2023-03-29T10:41:23.943 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm95234.ko 2023-03-29T10:41:23.943 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm95241.ko 2023-03-29T10:41:23.944 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm95245.ko 2023-03-29T10:41:23.944 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc2945.ko 2023-03-29T10:41:23.944 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc4151.ko 2023-03-29T10:41:23.944 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc4215.ko 2023-03-29T10:41:23.944 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc4222.ko 2023-03-29T10:41:23.944 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc4245.ko 2023-03-29T10:41:23.944 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc4260.ko 2023-03-29T10:41:23.944 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc4261.ko 2023-03-29T10:41:23.944 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max16065.ko 2023-03-29T10:41:23.944 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max1619.ko 2023-03-29T10:41:23.944 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max1668.ko 2023-03-29T10:41:23.944 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max197.ko 2023-03-29T10:41:23.944 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max6639.ko 2023-03-29T10:41:23.944 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max6650.ko 2023-03-29T10:41:23.944 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max6697.ko 2023-03-29T10:41:23.944 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/mcp3021.ko 2023-03-29T10:41:23.945 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/nct6683.ko 2023-03-29T10:41:23.945 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/nct6775-core.ko 2023-03-29T10:41:23.945 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/nct6775.ko 2023-03-29T10:41:23.945 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/nct7802.ko 2023-03-29T10:41:23.945 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pc87360.ko 2023-03-29T10:41:23.945 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pc87427.ko 2023-03-29T10:41:23.945 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pcf8591.ko 2023-03-29T10:41:23.945 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus 2023-03-29T10:41:23.945 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-03-29T10:41:23.945 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-03-29T10:41:23.945 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-03-29T10:41:23.946 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/max16064.ko 2023-03-29T10:41:23.946 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/max34440.ko 2023-03-29T10:41:23.946 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/max8688.ko 2023-03-29T10:41:23.946 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-03-29T10:41:23.946 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-03-29T10:41:23.946 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-03-29T10:41:23.946 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-03-29T10:41:23.946 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-03-29T10:41:23.946 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-03-29T10:41:23.946 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/powr1220.ko 2023-03-29T10:41:23.947 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/sch5627.ko 2023-03-29T10:41:23.947 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/sch5636.ko 2023-03-29T10:41:23.947 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/sch56xx-common.ko 2023-03-29T10:41:23.947 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/sht15.ko 2023-03-29T10:41:23.947 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/sht21.ko 2023-03-29T10:41:23.947 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/shtc1.ko 2023-03-29T10:41:23.947 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/sis5595.ko 2023-03-29T10:41:23.947 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/smsc47b397.ko 2023-03-29T10:41:23.947 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/smsc47m1.ko 2023-03-29T10:41:23.948 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/smsc47m192.ko 2023-03-29T10:41:23.948 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/thmc50.ko 2023-03-29T10:41:23.948 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/tmp102.ko 2023-03-29T10:41:23.948 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/tmp103.ko 2023-03-29T10:41:23.948 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/tmp401.ko 2023-03-29T10:41:23.948 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/tmp421.ko 2023-03-29T10:41:23.948 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/via-cputemp.ko 2023-03-29T10:41:23.948 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/via686a.ko 2023-03-29T10:41:23.948 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/vt1211.ko 2023-03-29T10:41:23.948 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/vt8231.ko 2023-03-29T10:41:23.949 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83627ehf.ko 2023-03-29T10:41:23.949 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83627hf.ko 2023-03-29T10:41:23.949 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83781d.ko 2023-03-29T10:41:23.949 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83791d.ko 2023-03-29T10:41:23.949 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83792d.ko 2023-03-29T10:41:23.949 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83793.ko 2023-03-29T10:41:23.949 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83795.ko 2023-03-29T10:41:23.949 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83l785ts.ko 2023-03-29T10:41:23.949 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83l786ng.ko 2023-03-29T10:41:23.950 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c 2023-03-29T10:41:23.950 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/algos 2023-03-29T10:41:23.950 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-03-29T10:41:23.950 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-03-29T10:41:23.950 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses 2023-03-29T10:41:23.950 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-03-29T10:41:23.950 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-03-29T10:41:23.950 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-03-29T10:41:23.950 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-03-29T10:41:23.951 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-03-29T10:41:23.951 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-03-29T10:41:23.951 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-03-29T10:41:23.951 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-03-29T10:41:23.951 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-i801.ko 2023-03-29T10:41:23.951 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-isch.ko 2023-03-29T10:41:23.951 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-03-29T10:41:23.951 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-03-29T10:41:23.951 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-03-29T10:41:23.952 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-parport.ko 2023-03-29T10:41:23.952 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-03-29T10:41:23.952 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-03-29T10:41:23.952 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-03-29T10:41:23.952 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-03-29T10:41:23.952 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-03-29T10:41:23.952 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-03-29T10:41:23.952 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-via.ko 2023-03-29T10:41:23.952 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-03-29T10:41:23.953 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-03-29T10:41:23.953 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/i2c-dev.ko 2023-03-29T10:41:23.953 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/i2c-mux.ko 2023-03-29T10:41:23.953 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-03-29T10:41:23.953 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/i2c-smbus.ko 2023-03-29T10:41:23.953 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/i2c-stub.ko 2023-03-29T10:41:23.953 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband 2023-03-29T10:41:23.953 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core 2023-03-29T10:41:23.953 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/ib_cm.ko 2023-03-29T10:41:23.953 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/ib_core.ko 2023-03-29T10:41:23.954 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/ib_umad.ko 2023-03-29T10:41:23.954 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-03-29T10:41:23.954 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/iw_cm.ko 2023-03-29T10:41:23.954 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/rdma_cm.ko 2023-03-29T10:41:23.954 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-03-29T10:41:23.954 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw 2023-03-29T10:41:23.954 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/cxgb4 2023-03-29T10:41:23.954 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-03-29T10:41:23.954 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/mlx4 2023-03-29T10:41:23.954 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-03-29T10:41:23.955 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/mlx5 2023-03-29T10:41:23.955 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-03-29T10:41:23.955 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/mthca 2023-03-29T10:41:23.955 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-03-29T10:41:23.955 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp 2023-03-29T10:41:23.955 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/ipoib 2023-03-29T10:41:23.955 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-03-29T10:41:23.955 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/iser 2023-03-29T10:41:23.955 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-03-29T10:41:23.956 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/isert 2023-03-29T10:41:23.956 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-03-29T10:41:23.956 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/srp 2023-03-29T10:41:23.956 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-03-29T10:41:23.956 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/srpt 2023-03-29T10:41:23.956 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-03-29T10:41:23.956 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input 2023-03-29T10:41:23.956 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/gameport 2023-03-29T10:41:23.956 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-03-29T10:41:23.957 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/gameport/fm801-gp.ko 2023-03-29T10:41:23.957 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/gameport/gameport.ko 2023-03-29T10:41:23.957 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/gameport/lightning.ko 2023-03-29T10:41:23.957 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/gameport/ns558.ko 2023-03-29T10:41:23.957 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/joydev.ko 2023-03-29T10:41:23.957 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/keyboard 2023-03-29T10:41:23.957 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/keyboard/gpio_keys.ko 2023-03-29T10:41:23.957 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse 2023-03-29T10:41:23.957 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/appletouch.ko 2023-03-29T10:41:23.957 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/bcm5974.ko 2023-03-29T10:41:23.958 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/cyapatp.ko 2023-03-29T10:41:23.958 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/elan_i2c.ko 2023-03-29T10:41:23.958 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/sermouse.ko 2023-03-29T10:41:23.958 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-03-29T10:41:23.958 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/synaptics_usb.ko 2023-03-29T10:41:23.958 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/vsxxxaa.ko 2023-03-29T10:41:23.958 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/rmi4 2023-03-29T10:41:23.958 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/rmi4/rmi_core.ko 2023-03-29T10:41:23.958 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/serio 2023-03-29T10:41:23.958 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/serio/altera_ps2.ko 2023-03-29T10:41:23.958 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/serio/arc_ps2.ko 2023-03-29T10:41:23.958 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-03-29T10:41:23.958 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/serio/serio_raw.ko 2023-03-29T10:41:23.959 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/sparse-keymap.ko 2023-03-29T10:41:23.959 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/iommu 2023-03-29T10:41:23.959 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/iommu/amd 2023-03-29T10:41:23.959 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/iommu/amd/iommu_v2.ko 2023-03-29T10:41:23.959 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds 2023-03-29T10:41:23.959 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/leds-blinkm.ko 2023-03-29T10:41:23.959 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/leds-lm3530.ko 2023-03-29T10:41:23.959 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/leds-lp3944.ko 2023-03-29T10:41:23.959 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/leds-ss4200.ko 2023-03-29T10:41:23.959 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger 2023-03-29T10:41:23.959 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-03-29T10:41:23.959 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-03-29T10:41:23.959 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-03-29T10:41:23.959 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-03-29T10:41:23.959 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2023-03-29T10:41:23.960 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-03-29T10:41:23.960 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-03-29T10:41:23.960 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-03-29T10:41:23.960 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-03-29T10:41:23.960 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md 2023-03-29T10:41:23.960 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/bcache 2023-03-29T10:41:23.960 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/bcache/bcache.ko 2023-03-29T10:41:23.960 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-bio-prison.ko 2023-03-29T10:41:23.960 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-cache-smq.ko 2023-03-29T10:41:23.960 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-cache.ko 2023-03-29T10:41:23.960 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-crypt.ko 2023-03-29T10:41:23.960 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-delay.ko 2023-03-29T10:41:23.960 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-flakey.ko 2023-03-29T10:41:23.960 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-log-userspace.ko 2023-03-29T10:41:23.960 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-multipath.ko 2023-03-29T10:41:23.960 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-queue-length.ko 2023-03-29T10:41:23.961 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-raid.ko 2023-03-29T10:41:23.961 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-round-robin.ko 2023-03-29T10:41:23.961 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-service-time.ko 2023-03-29T10:41:23.961 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-switch.ko 2023-03-29T10:41:23.961 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-thin-pool.ko 2023-03-29T10:41:23.961 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-verity.ko 2023-03-29T10:41:23.961 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/faulty.ko 2023-03-29T10:41:23.961 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/linear.ko 2023-03-29T10:41:23.961 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/multipath.ko 2023-03-29T10:41:23.961 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/persistent-data 2023-03-29T10:41:23.961 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-03-29T10:41:23.961 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/raid0.ko 2023-03-29T10:41:23.961 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/raid1.ko 2023-03-29T10:41:23.962 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/raid10.ko 2023-03-29T10:41:23.962 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/raid456.ko 2023-03-29T10:41:23.962 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message 2023-03-29T10:41:23.962 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion 2023-03-29T10:41:23.962 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptbase.ko 2023-03-29T10:41:23.962 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptctl.ko 2023-03-29T10:41:23.962 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptfc.ko 2023-03-29T10:41:23.962 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptlan.ko 2023-03-29T10:41:23.962 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptsas.ko 2023-03-29T10:41:23.962 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptscsih.ko 2023-03-29T10:41:23.962 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptspi.ko 2023-03-29T10:41:23.962 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd 2023-03-29T10:41:23.962 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/lpc_ich.ko 2023-03-29T10:41:23.962 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/lpc_sch.ko 2023-03-29T10:41:23.962 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/mfd-core.ko 2023-03-29T10:41:23.963 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/sm501.ko 2023-03-29T10:41:23.963 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/viperboard.ko 2023-03-29T10:41:23.963 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/vx855.ko 2023-03-29T10:41:23.963 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/wl1273-core.ko 2023-03-29T10:41:23.963 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc 2023-03-29T10:41:23.963 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/altera-stapl 2023-03-29T10:41:23.963 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-03-29T10:41:23.963 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/apds9802als.ko 2023-03-29T10:41:23.963 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/apds990x.ko 2023-03-29T10:41:23.963 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/bh1770glc.ko 2023-03-29T10:41:23.963 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/cb710 2023-03-29T10:41:23.963 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/cb710/cb710.ko 2023-03-29T10:41:23.963 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/eeprom 2023-03-29T10:41:23.963 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/eeprom/at24.ko 2023-03-29T10:41:23.964 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/eeprom/eeprom.ko 2023-03-29T10:41:23.964 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-03-29T10:41:23.964 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/eeprom/max6875.ko 2023-03-29T10:41:23.964 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/enclosure.ko 2023-03-29T10:41:23.964 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/hpilo.ko 2023-03-29T10:41:23.964 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/isl29003.ko 2023-03-29T10:41:23.964 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/isl29020.ko 2023-03-29T10:41:23.964 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/lis3lv02d 2023-03-29T10:41:23.964 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-03-29T10:41:23.964 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-03-29T10:41:23.964 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/mei 2023-03-29T10:41:23.964 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/mei/mei-me.ko 2023-03-29T10:41:23.964 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/mei/mei-txe.ko 2023-03-29T10:41:23.964 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/mei/mei.ko 2023-03-29T10:41:23.965 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/sgi-gru 2023-03-29T10:41:23.965 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/sgi-gru/gru.ko 2023-03-29T10:41:23.965 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/sgi-xp 2023-03-29T10:41:23.965 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/sgi-xp/xp.ko 2023-03-29T10:41:23.965 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/sgi-xp/xpc.ko 2023-03-29T10:41:23.965 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-03-29T10:41:23.965 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/tifm_7xx1.ko 2023-03-29T10:41:23.965 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/tifm_core.ko 2023-03-29T10:41:23.965 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/tsl2550.ko 2023-03-29T10:41:23.965 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/vmw_balloon.ko 2023-03-29T10:41:23.965 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/vmw_vmci 2023-03-29T10:41:23.965 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-03-29T10:41:23.965 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net 2023-03-29T10:41:23.965 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/bonding 2023-03-29T10:41:23.966 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/bonding/bonding.ko 2023-03-29T10:41:23.966 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa 2023-03-29T10:41:23.966 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa/b53 2023-03-29T10:41:23.966 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa/b53/b53_common.ko 2023-03-29T10:41:23.966 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa/bcm-sf2.ko 2023-03-29T10:41:23.966 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa/mv88e6060.ko 2023-03-29T10:41:23.966 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa/mv88e6xxx 2023-03-29T10:41:23.966 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-03-29T10:41:23.966 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dummy.ko 2023-03-29T10:41:23.966 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/eql.ko 2023-03-29T10:41:23.966 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet 2023-03-29T10:41:23.966 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/3com 2023-03-29T10:41:23.966 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-03-29T10:41:23.966 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-03-29T10:41:23.967 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-03-29T10:41:23.967 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-03-29T10:41:23.967 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/8390 2023-03-29T10:41:23.967 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/8390/8390.ko 2023-03-29T10:41:23.967 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-03-29T10:41:23.967 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-03-29T10:41:23.967 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-03-29T10:41:23.967 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/adaptec 2023-03-29T10:41:23.967 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-03-29T10:41:23.967 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/agere 2023-03-29T10:41:23.967 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/agere/et131x.ko 2023-03-29T10:41:23.967 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/alteon 2023-03-29T10:41:23.967 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-03-29T10:41:23.967 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/altera 2023-03-29T10:41:23.967 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-03-29T10:41:23.968 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/amd 2023-03-29T10:41:23.968 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-03-29T10:41:23.968 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-03-29T10:41:23.968 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-03-29T10:41:23.968 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros 2023-03-29T10:41:23.968 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/alx 2023-03-29T10:41:23.968 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-03-29T10:41:23.968 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atl1c 2023-03-29T10:41:23.968 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-03-29T10:41:23.968 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atl1e 2023-03-29T10:41:23.968 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-03-29T10:41:23.968 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atlx 2023-03-29T10:41:23.968 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-03-29T10:41:23.968 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-03-29T10:41:23.968 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom 2023-03-29T10:41:23.969 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-03-29T10:41:23.969 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-03-29T10:41:23.969 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-03-29T10:41:23.969 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-03-29T10:41:23.969 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-03-29T10:41:23.969 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/genet 2023-03-29T10:41:23.969 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-03-29T10:41:23.969 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-03-29T10:41:23.969 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/brocade 2023-03-29T10:41:23.969 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/brocade/bna 2023-03-29T10:41:23.969 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-03-29T10:41:23.969 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio 2023-03-29T10:41:23.969 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb 2023-03-29T10:41:23.969 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-03-29T10:41:23.969 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-03-29T10:41:23.970 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-03-29T10:41:23.970 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-03-29T10:41:23.970 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-03-29T10:41:23.970 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-03-29T10:41:23.970 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-03-29T10:41:23.970 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-03-29T10:41:23.970 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-03-29T10:41:23.970 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/cisco 2023-03-29T10:41:23.970 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/cisco/enic 2023-03-29T10:41:23.970 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-03-29T10:41:23.970 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec 2023-03-29T10:41:23.970 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip 2023-03-29T10:41:23.970 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-03-29T10:41:23.970 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-03-29T10:41:23.970 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-03-29T10:41:23.971 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-03-29T10:41:23.971 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-03-29T10:41:23.971 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-03-29T10:41:23.971 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dlink 2023-03-29T10:41:23.971 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-03-29T10:41:23.971 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-03-29T10:41:23.971 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dnet.ko 2023-03-29T10:41:23.971 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/emulex 2023-03-29T10:41:23.971 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/emulex/benet 2023-03-29T10:41:23.971 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-03-29T10:41:23.971 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/ethoc.ko 2023-03-29T10:41:23.972 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/fealnx.ko 2023-03-29T10:41:23.972 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel 2023-03-29T10:41:23.972 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/e100.ko 2023-03-29T10:41:23.972 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/e1000 2023-03-29T10:41:23.972 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-03-29T10:41:23.972 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/e1000e 2023-03-29T10:41:23.972 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-03-29T10:41:23.972 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/fm10k 2023-03-29T10:41:23.972 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-03-29T10:41:23.973 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/i40e 2023-03-29T10:41:23.973 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-03-29T10:41:23.973 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/iavf 2023-03-29T10:41:23.973 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-03-29T10:41:23.973 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/igb 2023-03-29T10:41:23.973 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-03-29T10:41:23.973 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/igbvf 2023-03-29T10:41:23.973 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-03-29T10:41:23.973 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/ixgb 2023-03-29T10:41:23.973 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/ixgb/ixgb.ko 2023-03-29T10:41:23.973 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/ixgbe 2023-03-29T10:41:23.973 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-03-29T10:41:23.973 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/ixgbevf 2023-03-29T10:41:23.973 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-03-29T10:41:23.974 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/jme.ko 2023-03-29T10:41:23.974 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/marvell 2023-03-29T10:41:23.974 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-03-29T10:41:23.974 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/marvell/skge.ko 2023-03-29T10:41:23.974 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-03-29T10:41:23.974 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox 2023-03-29T10:41:23.974 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox/mlx4 2023-03-29T10:41:23.974 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-03-29T10:41:23.975 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-03-29T10:41:23.975 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox/mlx5 2023-03-29T10:41:23.975 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-03-29T10:41:23.975 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-03-29T10:41:23.975 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/micrel 2023-03-29T10:41:23.975 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-03-29T10:41:23.975 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/myricom 2023-03-29T10:41:23.975 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/myricom/myri10ge 2023-03-29T10:41:23.975 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-03-29T10:41:23.976 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/natsemi 2023-03-29T10:41:23.976 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-03-29T10:41:23.976 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-03-29T10:41:23.976 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/neterion 2023-03-29T10:41:23.976 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-03-29T10:41:23.976 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/nvidia 2023-03-29T10:41:23.976 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-03-29T10:41:23.976 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/packetengines 2023-03-29T10:41:23.976 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-03-29T10:41:23.976 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-03-29T10:41:23.976 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/qlogic 2023-03-29T10:41:23.977 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/qlogic/netxen 2023-03-29T10:41:23.977 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-03-29T10:41:23.977 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-03-29T10:41:23.977 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-03-29T10:41:23.977 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-03-29T10:41:23.977 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/rdc 2023-03-29T10:41:23.977 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-03-29T10:41:23.977 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/realtek 2023-03-29T10:41:23.977 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-03-29T10:41:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-03-29T10:41:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/realtek/atp.ko 2023-03-29T10:41:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-03-29T10:41:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/rocker 2023-03-29T10:41:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-03-29T10:41:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sfc 2023-03-29T10:41:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-03-29T10:41:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/silan 2023-03-29T10:41:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-03-29T10:41:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sis 2023-03-29T10:41:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sis/sis190.ko 2023-03-29T10:41:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sis/sis900.ko 2023-03-29T10:41:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/smsc 2023-03-29T10:41:23.978 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-03-29T10:41:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-03-29T10:41:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-03-29T10:41:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-03-29T10:41:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/stmicro 2023-03-29T10:41:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/stmicro/stmmac 2023-03-29T10:41:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-03-29T10:41:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-03-29T10:41:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sun 2023-03-29T10:41:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sun/cassini.ko 2023-03-29T10:41:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sun/niu.ko 2023-03-29T10:41:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sun/sungem.ko 2023-03-29T10:41:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-03-29T10:41:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/tehuti 2023-03-29T10:41:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-03-29T10:41:23.979 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/ti 2023-03-29T10:41:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/ti/tlan.ko 2023-03-29T10:41:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/via 2023-03-29T10:41:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-03-29T10:41:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-03-29T10:41:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/wiznet 2023-03-29T10:41:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-03-29T10:41:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-03-29T10:41:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/xircom 2023-03-29T10:41:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-03-29T10:41:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/geneve.ko 2023-03-29T10:41:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/hyperv 2023-03-29T10:41:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-03-29T10:41:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ieee802154 2023-03-29T10:41:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ieee802154/fakelb.ko 2023-03-29T10:41:23.980 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ifb.ko 2023-03-29T10:41:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ipvlan 2023-03-29T10:41:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ipvlan/ipvlan.ko 2023-03-29T10:41:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/macvlan.ko 2023-03-29T10:41:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/macvtap.ko 2023-03-29T10:41:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/mdio 2023-03-29T10:41:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/mdio.ko 2023-03-29T10:41:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-03-29T10:41:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-03-29T10:41:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/mii.ko 2023-03-29T10:41:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/net_failover.ko 2023-03-29T10:41:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/netconsole.ko 2023-03-29T10:41:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/nlmon.ko 2023-03-29T10:41:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/pcs 2023-03-29T10:41:23.981 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/pcs/pcs-altera-tse.ko 2023-03-29T10:41:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-03-29T10:41:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy 2023-03-29T10:41:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/amd.ko 2023-03-29T10:41:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-03-29T10:41:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-03-29T10:41:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/bcm7xxx.ko 2023-03-29T10:41:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/bcm87xx.ko 2023-03-29T10:41:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/broadcom.ko 2023-03-29T10:41:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/cicada.ko 2023-03-29T10:41:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/davicom.ko 2023-03-29T10:41:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/dp83640.ko 2023-03-29T10:41:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/et1011c.ko 2023-03-29T10:41:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/icplus.ko 2023-03-29T10:41:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/lxt.ko 2023-03-29T10:41:23.982 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/marvell.ko 2023-03-29T10:41:23.983 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/micrel.ko 2023-03-29T10:41:23.983 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/national.ko 2023-03-29T10:41:23.983 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/phylink.ko 2023-03-29T10:41:23.983 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/qsemi.ko 2023-03-29T10:41:23.983 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/realtek.ko 2023-03-29T10:41:23.983 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/smsc.ko 2023-03-29T10:41:23.983 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/ste10Xp.ko 2023-03-29T10:41:23.983 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/vitesse.ko 2023-03-29T10:41:23.983 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp 2023-03-29T10:41:23.983 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/bsd_comp.ko 2023-03-29T10:41:23.983 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/ppp_async.ko 2023-03-29T10:41:23.983 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/ppp_deflate.ko 2023-03-29T10:41:23.983 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/ppp_generic.ko 2023-03-29T10:41:23.983 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/ppp_mppe.ko 2023-03-29T10:41:23.983 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/ppp_synctty.ko 2023-03-29T10:41:23.984 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/pppoe.ko 2023-03-29T10:41:23.984 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/pppox.ko 2023-03-29T10:41:23.984 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/pptp.ko 2023-03-29T10:41:23.984 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/slip 2023-03-29T10:41:23.984 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/slip/slhc.ko 2023-03-29T10:41:23.984 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/slip/slip.ko 2023-03-29T10:41:23.984 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/sungem_phy.ko 2023-03-29T10:41:23.984 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/tap.ko 2023-03-29T10:41:23.984 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team 2023-03-29T10:41:23.984 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team/team.ko 2023-03-29T10:41:23.984 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team/team_mode_activebackup.ko 2023-03-29T10:41:23.984 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team/team_mode_broadcast.ko 2023-03-29T10:41:23.984 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-03-29T10:41:23.984 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team/team_mode_random.ko 2023-03-29T10:41:23.984 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-03-29T10:41:23.985 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/tun.ko 2023-03-29T10:41:23.985 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/veth.ko 2023-03-29T10:41:23.985 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/virtio_net.ko 2023-03-29T10:41:23.985 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/vmxnet3 2023-03-29T10:41:23.985 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-03-29T10:41:23.985 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/vxlan 2023-03-29T10:41:23.985 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/vxlan/vxlan.ko 2023-03-29T10:41:23.985 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/xen-netback 2023-03-29T10:41:23.985 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/xen-netback/xen-netback.ko 2023-03-29T10:41:23.985 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/xen-netfront.ko 2023-03-29T10:41:23.985 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ntb 2023-03-29T10:41:23.985 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ntb/ntb.ko 2023-03-29T10:41:23.985 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/nvme 2023-03-29T10:41:23.985 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/nvme/host 2023-03-29T10:41:23.986 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/nvme/host/nvme-core.ko 2023-03-29T10:41:23.986 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/nvme/host/nvme.ko 2023-03-29T10:41:23.986 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/parport 2023-03-29T10:41:23.986 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/parport/parport.ko 2023-03-29T10:41:23.986 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/parport/parport_cs.ko 2023-03-29T10:41:23.986 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/parport/parport_pc.ko 2023-03-29T10:41:23.986 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/parport/parport_serial.ko 2023-03-29T10:41:23.986 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pci 2023-03-29T10:41:23.986 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pci/hotplug 2023-03-29T10:41:23.986 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-03-29T10:41:23.986 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pci/pcie 2023-03-29T10:41:23.986 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pci/pcie/aer_inject.ko 2023-03-29T10:41:23.986 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pci/xen-pcifront.ko 2023-03-29T10:41:23.986 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pcmcia 2023-03-29T10:41:23.986 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pcmcia/i82092.ko 2023-03-29T10:41:23.987 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pcmcia/pd6729.ko 2023-03-29T10:41:23.987 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pcmcia/yenta_socket.ko 2023-03-29T10:41:23.987 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pinctrl 2023-03-29T10:41:23.987 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pinctrl/intel 2023-03-29T10:41:23.987 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-03-29T10:41:23.987 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform 2023-03-29T10:41:23.987 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86 2023-03-29T10:41:23.987 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/acer-wmi.ko 2023-03-29T10:41:23.987 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/acerhdf.ko 2023-03-29T10:41:23.987 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/apple-gmux.ko 2023-03-29T10:41:23.987 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/asus-laptop.ko 2023-03-29T10:41:23.987 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-03-29T10:41:23.987 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/asus-wmi.ko 2023-03-29T10:41:23.987 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/classmate-laptop.ko 2023-03-29T10:41:23.987 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-03-29T10:41:23.987 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-03-29T10:41:23.988 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-03-29T10:41:23.988 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-03-29T10:41:23.988 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/hdaps.ko 2023-03-29T10:41:23.988 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/intel 2023-03-29T10:41:23.988 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-03-29T10:41:23.988 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/intel_ips.ko 2023-03-29T10:41:23.988 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/msi-wmi.ko 2023-03-29T10:41:23.988 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/mxm-wmi.ko 2023-03-29T10:41:23.988 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-03-29T10:41:23.988 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/samsung-laptop.ko 2023-03-29T10:41:23.988 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/samsung-q10.ko 2023-03-29T10:41:23.988 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/topstar-laptop.ko 2023-03-29T10:41:23.988 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-03-29T10:41:23.988 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/toshiba_haps.ko 2023-03-29T10:41:23.988 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/wmi-bmof.ko 2023-03-29T10:41:23.989 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/wmi.ko 2023-03-29T10:41:23.989 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/powercap 2023-03-29T10:41:23.989 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/powercap/intel_rapl_common.ko 2023-03-29T10:41:23.989 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/powercap/intel_rapl_msr.ko 2023-03-29T10:41:23.989 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pps 2023-03-29T10:41:23.989 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pps/clients 2023-03-29T10:41:23.989 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pps/clients/pps-gpio.ko 2023-03-29T10:41:23.989 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pps/clients/pps-ldisc.ko 2023-03-29T10:41:23.989 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pps/clients/pps_parport.ko 2023-03-29T10:41:23.989 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pps/pps_core.ko 2023-03-29T10:41:23.989 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ptp 2023-03-29T10:41:23.989 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ptp/ptp.ko 2023-03-29T10:41:23.989 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ptp/ptp_kvm.ko 2023-03-29T10:41:23.989 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc 2023-03-29T10:41:23.990 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-bq32k.ko 2023-03-29T10:41:23.990 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-bq4802.ko 2023-03-29T10:41:23.990 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1286.ko 2023-03-29T10:41:23.990 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1307.ko 2023-03-29T10:41:23.990 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1374.ko 2023-03-29T10:41:23.990 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1511.ko 2023-03-29T10:41:23.990 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1553.ko 2023-03-29T10:41:23.990 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1672.ko 2023-03-29T10:41:23.990 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1742.ko 2023-03-29T10:41:23.990 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds2404.ko 2023-03-29T10:41:23.990 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds3232.ko 2023-03-29T10:41:23.990 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-em3027.ko 2023-03-29T10:41:23.990 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-fm3130.ko 2023-03-29T10:41:23.990 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-isl12022.ko 2023-03-29T10:41:23.990 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-isl1208.ko 2023-03-29T10:41:23.991 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-m41t80.ko 2023-03-29T10:41:23.991 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-m48t35.ko 2023-03-29T10:41:23.991 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-m48t59.ko 2023-03-29T10:41:23.991 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-max6900.ko 2023-03-29T10:41:23.991 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-msm6242.ko 2023-03-29T10:41:23.991 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-pcf2127.ko 2023-03-29T10:41:23.991 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-pcf85063.ko 2023-03-29T10:41:23.991 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-pcf8523.ko 2023-03-29T10:41:23.991 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-pcf8563.ko 2023-03-29T10:41:23.991 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-pcf8583.ko 2023-03-29T10:41:23.991 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-rp5c01.ko 2023-03-29T10:41:23.991 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-rs5c372.ko 2023-03-29T10:41:23.991 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-03-29T10:41:23.991 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-rx8025.ko 2023-03-29T10:41:23.991 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-rx8581.ko 2023-03-29T10:41:23.991 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-03-29T10:41:23.992 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-x1205.ko 2023-03-29T10:41:23.992 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi 2023-03-29T10:41:23.992 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/3w-9xxx.ko 2023-03-29T10:41:23.992 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/3w-sas.ko 2023-03-29T10:41:23.992 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/3w-xxxx.ko 2023-03-29T10:41:23.992 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/BusLogic.ko 2023-03-29T10:41:23.992 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/a100u2w.ko 2023-03-29T10:41:23.992 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/aacraid 2023-03-29T10:41:23.992 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/aacraid/aacraid.ko 2023-03-29T10:41:23.992 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/advansys.ko 2023-03-29T10:41:23.992 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/aic7xxx 2023-03-29T10:41:23.992 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-03-29T10:41:23.992 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-03-29T10:41:23.992 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/am53c974.ko 2023-03-29T10:41:23.992 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/arcmsr 2023-03-29T10:41:23.993 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-03-29T10:41:23.993 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/atp870u.ko 2023-03-29T10:41:23.993 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/be2iscsi 2023-03-29T10:41:23.993 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-03-29T10:41:23.993 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/bfa 2023-03-29T10:41:23.993 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/bfa/bfa.ko 2023-03-29T10:41:23.993 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/bnx2fc 2023-03-29T10:41:23.993 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-03-29T10:41:23.993 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/bnx2i 2023-03-29T10:41:23.993 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-03-29T10:41:23.993 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/ch.ko 2023-03-29T10:41:23.993 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/csiostor 2023-03-29T10:41:23.993 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/csiostor/csiostor.ko 2023-03-29T10:41:23.993 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/cxgbi 2023-03-29T10:41:23.994 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/cxgbi/cxgb3i 2023-03-29T10:41:23.994 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-03-29T10:41:23.994 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/cxgbi/cxgb4i 2023-03-29T10:41:23.994 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-03-29T10:41:23.994 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-03-29T10:41:23.994 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/dc395x.ko 2023-03-29T10:41:23.994 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/device_handler 2023-03-29T10:41:23.994 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-03-29T10:41:23.994 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-03-29T10:41:23.994 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-03-29T10:41:23.994 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-03-29T10:41:23.994 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/dmx3191d.ko 2023-03-29T10:41:23.994 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/esas2r 2023-03-29T10:41:23.994 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/esas2r/esas2r.ko 2023-03-29T10:41:23.995 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/esp_scsi.ko 2023-03-29T10:41:23.995 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/fcoe 2023-03-29T10:41:23.995 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/fcoe/fcoe.ko 2023-03-29T10:41:23.995 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-03-29T10:41:23.995 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/fnic 2023-03-29T10:41:23.995 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/fnic/fnic.ko 2023-03-29T10:41:23.995 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/hpsa.ko 2023-03-29T10:41:23.995 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/hptiop.ko 2023-03-29T10:41:23.995 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/hv_storvsc.ko 2023-03-29T10:41:23.995 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/initio.ko 2023-03-29T10:41:23.995 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/ipr.ko 2023-03-29T10:41:23.995 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/ips.ko 2023-03-29T10:41:23.995 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/isci 2023-03-29T10:41:23.995 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/isci/isci.ko 2023-03-29T10:41:23.995 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-03-29T10:41:23.996 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/iscsi_tcp.ko 2023-03-29T10:41:23.996 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/libfc 2023-03-29T10:41:23.996 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/libfc/libfc.ko 2023-03-29T10:41:23.996 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/libiscsi.ko 2023-03-29T10:41:23.996 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/libiscsi_tcp.ko 2023-03-29T10:41:23.996 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/libsas 2023-03-29T10:41:23.996 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/libsas/libsas.ko 2023-03-29T10:41:23.996 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/lpfc 2023-03-29T10:41:23.996 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/lpfc/lpfc.ko 2023-03-29T10:41:23.996 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/megaraid 2023-03-29T10:41:23.996 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/megaraid.ko 2023-03-29T10:41:23.996 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-03-29T10:41:23.996 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-03-29T10:41:23.996 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-03-29T10:41:23.997 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/mpt3sas 2023-03-29T10:41:23.997 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-03-29T10:41:23.997 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/mvsas 2023-03-29T10:41:23.997 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/mvsas/mvsas.ko 2023-03-29T10:41:23.997 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/mvumi.ko 2023-03-29T10:41:23.997 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/pm8001 2023-03-29T10:41:23.997 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-03-29T10:41:23.997 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/pmcraid.ko 2023-03-29T10:41:23.997 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/qla1280.ko 2023-03-29T10:41:23.997 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/qla2xxx 2023-03-29T10:41:23.997 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-03-29T10:41:23.997 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-03-29T10:41:23.997 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/qla4xxx 2023-03-29T10:41:23.998 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-03-29T10:41:23.998 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/raid_class.ko 2023-03-29T10:41:23.998 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/scsi_debug.ko 2023-03-29T10:41:23.998 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/scsi_transport_fc.ko 2023-03-29T10:41:23.998 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-03-29T10:41:23.998 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/scsi_transport_sas.ko 2023-03-29T10:41:23.998 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/scsi_transport_spi.ko 2023-03-29T10:41:23.998 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/scsi_transport_srp.ko 2023-03-29T10:41:23.998 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/ses.ko 2023-03-29T10:41:23.998 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/st.ko 2023-03-29T10:41:23.998 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/stex.ko 2023-03-29T10:41:23.998 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/sym53c8xx_2 2023-03-29T10:41:23.998 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-03-29T10:41:23.999 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/virtio_scsi.ko 2023-03-29T10:41:23.999 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/vmw_pvscsi.ko 2023-03-29T10:41:23.999 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/wd719x.ko 2023-03-29T10:41:23.999 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ssb 2023-03-29T10:41:23.999 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ssb/ssb.ko 2023-03-29T10:41:23.999 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target 2023-03-29T10:41:23.999 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/iscsi 2023-03-29T10:41:23.999 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-03-29T10:41:23.999 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/loopback 2023-03-29T10:41:23.999 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/loopback/tcm_loop.ko 2023-03-29T10:41:23.999 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/target_core_file.ko 2023-03-29T10:41:23.999 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/target_core_iblock.ko 2023-03-29T10:41:23.999 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/target_core_mod.ko 2023-03-29T10:41:23.999 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/target_core_pscsi.ko 2023-03-29T10:41:24.000 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/target_core_user.ko 2023-03-29T10:41:24.000 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/tcm_fc 2023-03-29T10:41:24.000 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-03-29T10:41:24.000 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal 2023-03-29T10:41:24.000 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel 2023-03-29T10:41:24.000 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal 2023-03-29T10:41:24.000 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-03-29T10:41:24.000 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-03-29T10:41:24.000 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-03-29T10:41:24.000 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-03-29T10:41:24.000 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-03-29T10:41:24.000 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-03-29T10:41:24.000 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-03-29T10:41:24.000 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-03-29T10:41:24.000 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-03-29T10:41:24.001 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-03-29T10:41:24.001 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-03-29T10:41:24.001 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-03-29T10:41:24.001 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-03-29T10:41:24.001 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-03-29T10:41:24.001 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-03-29T10:41:24.001 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty 2023-03-29T10:41:24.001 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/ipwireless 2023-03-29T10:41:24.001 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-03-29T10:41:24.001 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/n_gsm.ko 2023-03-29T10:41:24.001 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/n_hdlc.ko 2023-03-29T10:41:24.001 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/nozomi.ko 2023-03-29T10:41:24.001 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/serial 2023-03-29T10:41:24.001 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/serial/8250 2023-03-29T10:41:24.002 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-03-29T10:41:24.002 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/serial/arc_uart.ko 2023-03-29T10:41:24.002 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/serial/jsm 2023-03-29T10:41:24.002 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/serial/jsm/jsm.ko 2023-03-29T10:41:24.002 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/synclink_gt.ko 2023-03-29T10:41:24.002 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ufs 2023-03-29T10:41:24.002 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ufs/core 2023-03-29T10:41:24.002 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ufs/core/ufshcd-core.ko 2023-03-29T10:41:24.002 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ufs/host 2023-03-29T10:41:24.002 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-03-29T10:41:24.002 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/uio 2023-03-29T10:41:24.002 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/uio/uio.ko 2023-03-29T10:41:24.002 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/uio/uio_aec.ko 2023-03-29T10:41:24.002 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/uio/uio_cif.ko 2023-03-29T10:41:24.002 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/uio/uio_pci_generic.ko 2023-03-29T10:41:24.003 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/uio/uio_sercos3.ko 2023-03-29T10:41:24.003 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb 2023-03-29T10:41:24.003 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/class 2023-03-29T10:41:24.003 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/class/cdc-acm.ko 2023-03-29T10:41:24.003 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/class/cdc-wdm.ko 2023-03-29T10:41:24.003 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/class/usblp.ko 2023-03-29T10:41:24.003 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/class/usbtmc.ko 2023-03-29T10:41:24.003 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/host 2023-03-29T10:41:24.003 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/host/sl811-hcd.ko 2023-03-29T10:41:24.003 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/host/u132-hcd.ko 2023-03-29T10:41:24.003 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/image 2023-03-29T10:41:24.003 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/image/mdc800.ko 2023-03-29T10:41:24.003 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/image/microtek.ko 2023-03-29T10:41:24.003 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc 2023-03-29T10:41:24.003 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/adutux.ko 2023-03-29T10:41:24.003 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/appledisplay.ko 2023-03-29T10:41:24.004 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/emi26.ko 2023-03-29T10:41:24.004 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/emi62.ko 2023-03-29T10:41:24.004 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/ezusb.ko 2023-03-29T10:41:24.004 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/ftdi-elan.ko 2023-03-29T10:41:24.004 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/idmouse.ko 2023-03-29T10:41:24.004 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/iowarrior.ko 2023-03-29T10:41:24.004 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/isight_firmware.ko 2023-03-29T10:41:24.004 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/ldusb.ko 2023-03-29T10:41:24.004 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/legousbtower.ko 2023-03-29T10:41:24.004 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/sisusbvga 2023-03-29T10:41:24.004 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-03-29T10:41:24.004 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/trancevibrator.ko 2023-03-29T10:41:24.004 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/usb3503.ko 2023-03-29T10:41:24.004 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/usblcd.ko 2023-03-29T10:41:24.005 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/usbsevseg.ko 2023-03-29T10:41:24.005 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/uss720.ko 2023-03-29T10:41:24.005 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/yurex.ko 2023-03-29T10:41:24.005 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/phy 2023-03-29T10:41:24.005 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/phy/phy-generic.ko 2023-03-29T10:41:24.005 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial 2023-03-29T10:41:24.005 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/aircable.ko 2023-03-29T10:41:24.005 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ark3116.ko 2023-03-29T10:41:24.005 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/belkin_sa.ko 2023-03-29T10:41:24.005 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ch341.ko 2023-03-29T10:41:24.005 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/cp210x.ko 2023-03-29T10:41:24.005 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/cyberjack.ko 2023-03-29T10:41:24.005 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/cypress_m8.ko 2023-03-29T10:41:24.005 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/digi_acceleport.ko 2023-03-29T10:41:24.005 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/empeg.ko 2023-03-29T10:41:24.006 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ftdi_sio.ko 2023-03-29T10:41:24.006 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/garmin_gps.ko 2023-03-29T10:41:24.006 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/io_edgeport.ko 2023-03-29T10:41:24.006 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/io_ti.ko 2023-03-29T10:41:24.006 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ipaq.ko 2023-03-29T10:41:24.006 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ipw.ko 2023-03-29T10:41:24.006 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ir-usb.ko 2023-03-29T10:41:24.006 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-03-29T10:41:24.006 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/keyspan.ko 2023-03-29T10:41:24.006 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/keyspan_pda.ko 2023-03-29T10:41:24.006 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/kl5kusb105.ko 2023-03-29T10:41:24.006 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/kobil_sct.ko 2023-03-29T10:41:24.006 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/mct_u232.ko 2023-03-29T10:41:24.006 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/mos7720.ko 2023-03-29T10:41:24.007 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/mos7840.ko 2023-03-29T10:41:24.007 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/navman.ko 2023-03-29T10:41:24.007 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/omninet.ko 2023-03-29T10:41:24.007 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/opticon.ko 2023-03-29T10:41:24.007 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/option.ko 2023-03-29T10:41:24.007 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/oti6858.ko 2023-03-29T10:41:24.007 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/pl2303.ko 2023-03-29T10:41:24.007 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/qcaux.ko 2023-03-29T10:41:24.007 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/qcserial.ko 2023-03-29T10:41:24.007 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/quatech2.ko 2023-03-29T10:41:24.007 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/safe_serial.ko 2023-03-29T10:41:24.007 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/sierra.ko 2023-03-29T10:41:24.007 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/spcp8x5.ko 2023-03-29T10:41:24.007 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ssu100.ko 2023-03-29T10:41:24.007 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/symbolserial.ko 2023-03-29T10:41:24.007 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-03-29T10:41:24.008 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-03-29T10:41:24.008 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/usb_debug.ko 2023-03-29T10:41:24.008 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/usb_wwan.ko 2023-03-29T10:41:24.008 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/visor.ko 2023-03-29T10:41:24.008 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/whiteheat.ko 2023-03-29T10:41:24.008 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/xsens_mt.ko 2023-03-29T10:41:24.008 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage 2023-03-29T10:41:24.008 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/uas.ko 2023-03-29T10:41:24.008 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-alauda.ko 2023-03-29T10:41:24.008 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-cypress.ko 2023-03-29T10:41:24.008 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-datafab.ko 2023-03-29T10:41:24.008 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-03-29T10:41:24.008 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-freecom.ko 2023-03-29T10:41:24.009 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-isd200.ko 2023-03-29T10:41:24.009 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-03-29T10:41:24.009 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-karma.ko 2023-03-29T10:41:24.009 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-onetouch.ko 2023-03-29T10:41:24.009 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-realtek.ko 2023-03-29T10:41:24.009 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-sddr09.ko 2023-03-29T10:41:24.009 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-sddr55.ko 2023-03-29T10:41:24.009 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-usbat.ko 2023-03-29T10:41:24.009 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/usb-storage.ko 2023-03-29T10:41:24.009 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/usbip 2023-03-29T10:41:24.009 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/usbip/usbip-core.ko 2023-03-29T10:41:24.009 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/usbip/usbip-host.ko 2023-03-29T10:41:24.009 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-03-29T10:41:24.009 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vfio 2023-03-29T10:41:24.009 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vfio/pci 2023-03-29T10:41:24.010 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-03-29T10:41:24.010 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vfio/pci/vfio-pci.ko 2023-03-29T10:41:24.010 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vfio/vfio.ko 2023-03-29T10:41:24.010 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-03-29T10:41:24.010 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vhost 2023-03-29T10:41:24.010 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vhost/vhost.ko 2023-03-29T10:41:24.010 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vhost/vhost_iotlb.ko 2023-03-29T10:41:24.010 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vhost/vhost_net.ko 2023-03-29T10:41:24.010 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vhost/vhost_scsi.ko 2023-03-29T10:41:24.010 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/video 2023-03-29T10:41:24.010 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/video/backlight 2023-03-29T10:41:24.010 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/video/backlight/apple_bl.ko 2023-03-29T10:41:24.010 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/video/backlight/lcd.ko 2023-03-29T10:41:24.010 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/video/backlight/platform_lcd.ko 2023-03-29T10:41:24.010 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio 2023-03-29T10:41:24.011 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio.ko 2023-03-29T10:41:24.011 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio_balloon.ko 2023-03-29T10:41:24.011 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio_mmio.ko 2023-03-29T10:41:24.011 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio_pci.ko 2023-03-29T10:41:24.011 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-03-29T10:41:24.011 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-03-29T10:41:24.011 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio_ring.ko 2023-03-29T10:41:24.011 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1 2023-03-29T10:41:24.011 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/masters 2023-03-29T10:41:24.011 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/masters/ds2482.ko 2023-03-29T10:41:24.011 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/masters/ds2490.ko 2023-03-29T10:41:24.011 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves 2023-03-29T10:41:24.011 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-03-29T10:41:24.011 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-03-29T10:41:24.011 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-03-29T10:41:24.012 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-03-29T10:41:24.012 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-03-29T10:41:24.012 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-03-29T10:41:24.012 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-03-29T10:41:24.012 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-03-29T10:41:24.012 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-03-29T10:41:24.012 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_smem.ko 2023-03-29T10:41:24.012 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_therm.ko 2023-03-29T10:41:24.012 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/wire.ko 2023-03-29T10:41:24.012 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog 2023-03-29T10:41:24.012 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/alim1535_wdt.ko 2023-03-29T10:41:24.012 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/alim7101_wdt.ko 2023-03-29T10:41:24.012 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/f71808e_wdt.ko 2023-03-29T10:41:24.013 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/hpwdt.ko 2023-03-29T10:41:24.013 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/i6300esb.ko 2023-03-29T10:41:24.013 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-03-29T10:41:24.013 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/iTCO_wdt.ko 2023-03-29T10:41:24.013 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/ib700wdt.ko 2023-03-29T10:41:24.013 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/ibmasr.ko 2023-03-29T10:41:24.013 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-03-29T10:41:24.013 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/it8712f_wdt.ko 2023-03-29T10:41:24.013 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/it87_wdt.ko 2023-03-29T10:41:24.013 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/machzwd.ko 2023-03-29T10:41:24.013 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/nv_tco.ko 2023-03-29T10:41:24.013 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/pcwd_pci.ko 2023-03-29T10:41:24.013 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/pcwd_usb.ko 2023-03-29T10:41:24.014 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-03-29T10:41:24.014 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/sch311x_wdt.ko 2023-03-29T10:41:24.014 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/softdog.ko 2023-03-29T10:41:24.014 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/sp5100_tco.ko 2023-03-29T10:41:24.014 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/via_wdt.ko 2023-03-29T10:41:24.014 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-03-29T10:41:24.014 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/w83877f_wdt.ko 2023-03-29T10:41:24.014 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/w83977f_wdt.ko 2023-03-29T10:41:24.014 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/wdt_pci.ko 2023-03-29T10:41:24.014 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/xen_wdt.ko 2023-03-29T10:41:24.014 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen 2023-03-29T10:41:24.014 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-acpi-processor.ko 2023-03-29T10:41:24.014 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-evtchn.ko 2023-03-29T10:41:24.014 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-gntalloc.ko 2023-03-29T10:41:24.014 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-gntdev.ko 2023-03-29T10:41:24.015 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-pciback 2023-03-29T10:41:24.015 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-03-29T10:41:24.015 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-privcmd.ko 2023-03-29T10:41:24.015 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xenfs 2023-03-29T10:41:24.015 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xenfs/xenfs.ko 2023-03-29T10:41:24.015 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs 2023-03-29T10:41:24.015 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/9p 2023-03-29T10:41:24.015 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/9p/9p.ko 2023-03-29T10:41:24.015 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/affs 2023-03-29T10:41:24.015 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/affs/affs.ko 2023-03-29T10:41:24.015 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/befs 2023-03-29T10:41:24.015 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/befs/befs.ko 2023-03-29T10:41:24.015 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/binfmt_misc.ko 2023-03-29T10:41:24.015 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/btrfs 2023-03-29T10:41:24.016 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/btrfs/btrfs.ko 2023-03-29T10:41:24.016 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/cachefiles 2023-03-29T10:41:24.016 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/cachefiles/cachefiles.ko 2023-03-29T10:41:24.016 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ceph 2023-03-29T10:41:24.016 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ceph/ceph.ko 2023-03-29T10:41:24.016 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/cifs 2023-03-29T10:41:24.016 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/cifs/cifs.ko 2023-03-29T10:41:24.016 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/coda 2023-03-29T10:41:24.016 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/coda/coda.ko 2023-03-29T10:41:24.016 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/cramfs 2023-03-29T10:41:24.016 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/cramfs/cramfs.ko 2023-03-29T10:41:24.016 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/dlm 2023-03-29T10:41:24.016 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/dlm/dlm.ko 2023-03-29T10:41:24.016 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ecryptfs 2023-03-29T10:41:24.016 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ecryptfs/ecryptfs.ko 2023-03-29T10:41:24.017 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/f2fs 2023-03-29T10:41:24.017 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/f2fs/f2fs.ko 2023-03-29T10:41:24.017 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fat 2023-03-29T10:41:24.017 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fat/fat.ko 2023-03-29T10:41:24.017 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fat/msdos.ko 2023-03-29T10:41:24.017 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fat/vfat.ko 2023-03-29T10:41:24.017 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fscache 2023-03-29T10:41:24.017 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fscache/fscache.ko 2023-03-29T10:41:24.017 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fuse 2023-03-29T10:41:24.017 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fuse/cuse.ko 2023-03-29T10:41:24.017 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fuse/fuse.ko 2023-03-29T10:41:24.017 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/gfs2 2023-03-29T10:41:24.017 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/gfs2/gfs2.ko 2023-03-29T10:41:24.017 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/hfs 2023-03-29T10:41:24.018 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/hfs/hfs.ko 2023-03-29T10:41:24.018 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/hfsplus 2023-03-29T10:41:24.018 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/hfsplus/hfsplus.ko 2023-03-29T10:41:24.018 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/isofs 2023-03-29T10:41:24.018 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/isofs/isofs.ko 2023-03-29T10:41:24.018 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/jfs 2023-03-29T10:41:24.018 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/jfs/jfs.ko 2023-03-29T10:41:24.018 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/lockd 2023-03-29T10:41:24.018 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/lockd/lockd.ko 2023-03-29T10:41:24.018 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/minix 2023-03-29T10:41:24.018 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/minix/minix.ko 2023-03-29T10:41:24.018 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/netfs 2023-03-29T10:41:24.018 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/netfs/netfs.ko 2023-03-29T10:41:24.018 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs 2023-03-29T10:41:24.018 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/blocklayout 2023-03-29T10:41:24.019 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-03-29T10:41:24.019 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/filelayout 2023-03-29T10:41:24.019 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-03-29T10:41:24.019 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/flexfilelayout 2023-03-29T10:41:24.019 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-03-29T10:41:24.019 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/nfs.ko 2023-03-29T10:41:24.019 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/nfsv3.ko 2023-03-29T10:41:24.019 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/nfsv4.ko 2023-03-29T10:41:24.019 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs_common 2023-03-29T10:41:24.019 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs_common/grace.ko 2023-03-29T10:41:24.019 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs_common/nfs_acl.ko 2023-03-29T10:41:24.019 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfsd 2023-03-29T10:41:24.019 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfsd/nfsd.ko 2023-03-29T10:41:24.019 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nilfs2 2023-03-29T10:41:24.019 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nilfs2/nilfs2.ko 2023-03-29T10:41:24.019 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls 2023-03-29T10:41:24.020 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-celtic.ko 2023-03-29T10:41:24.020 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-centeuro.ko 2023-03-29T10:41:24.020 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-croatian.ko 2023-03-29T10:41:24.020 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-cyrillic.ko 2023-03-29T10:41:24.020 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-gaelic.ko 2023-03-29T10:41:24.020 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-greek.ko 2023-03-29T10:41:24.020 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-iceland.ko 2023-03-29T10:41:24.020 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-inuit.ko 2023-03-29T10:41:24.020 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-roman.ko 2023-03-29T10:41:24.020 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-romanian.ko 2023-03-29T10:41:24.020 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-turkish.ko 2023-03-29T10:41:24.020 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp1250.ko 2023-03-29T10:41:24.020 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp1251.ko 2023-03-29T10:41:24.020 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp1255.ko 2023-03-29T10:41:24.020 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp737.ko 2023-03-29T10:41:24.021 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp775.ko 2023-03-29T10:41:24.021 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp850.ko 2023-03-29T10:41:24.021 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp852.ko 2023-03-29T10:41:24.021 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp855.ko 2023-03-29T10:41:24.021 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp857.ko 2023-03-29T10:41:24.021 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp860.ko 2023-03-29T10:41:24.021 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp861.ko 2023-03-29T10:41:24.021 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp862.ko 2023-03-29T10:41:24.021 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp863.ko 2023-03-29T10:41:24.021 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp864.ko 2023-03-29T10:41:24.021 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp865.ko 2023-03-29T10:41:24.021 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp866.ko 2023-03-29T10:41:24.021 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp869.ko 2023-03-29T10:41:24.021 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp874.ko 2023-03-29T10:41:24.022 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp932.ko 2023-03-29T10:41:24.022 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp936.ko 2023-03-29T10:41:24.022 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp949.ko 2023-03-29T10:41:24.022 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp950.ko 2023-03-29T10:41:24.022 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_euc-jp.ko 2023-03-29T10:41:24.022 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-1.ko 2023-03-29T10:41:24.022 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-13.ko 2023-03-29T10:41:24.022 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-14.ko 2023-03-29T10:41:24.022 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-15.ko 2023-03-29T10:41:24.022 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-2.ko 2023-03-29T10:41:24.022 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-3.ko 2023-03-29T10:41:24.022 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-4.ko 2023-03-29T10:41:24.022 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-5.ko 2023-03-29T10:41:24.022 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-6.ko 2023-03-29T10:41:24.022 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-7.ko 2023-03-29T10:41:24.023 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-9.ko 2023-03-29T10:41:24.023 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_koi8-r.ko 2023-03-29T10:41:24.023 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_koi8-ru.ko 2023-03-29T10:41:24.023 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_koi8-u.ko 2023-03-29T10:41:24.023 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_utf8.ko 2023-03-29T10:41:24.023 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2 2023-03-29T10:41:24.023 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/cluster 2023-03-29T10:41:24.023 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-03-29T10:41:24.023 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/dlm 2023-03-29T10:41:24.023 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-03-29T10:41:24.023 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/dlmfs 2023-03-29T10:41:24.023 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-03-29T10:41:24.023 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/ocfs2.ko 2023-03-29T10:41:24.023 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-03-29T10:41:24.023 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-03-29T10:41:24.024 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-03-29T10:41:24.024 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/overlayfs 2023-03-29T10:41:24.024 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/overlayfs/overlay.ko 2023-03-29T10:41:24.024 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/pstore 2023-03-29T10:41:24.024 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/pstore/ramoops.ko 2023-03-29T10:41:24.024 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/reiserfs 2023-03-29T10:41:24.024 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/reiserfs/reiserfs.ko 2023-03-29T10:41:24.024 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/romfs 2023-03-29T10:41:24.024 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/romfs/romfs.ko 2023-03-29T10:41:24.024 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/smbfs_common 2023-03-29T10:41:24.024 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/smbfs_common/cifs_arc4.ko 2023-03-29T10:41:24.024 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/smbfs_common/cifs_md4.ko 2023-03-29T10:41:24.024 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/squashfs 2023-03-29T10:41:24.024 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/squashfs/squashfs.ko 2023-03-29T10:41:24.024 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/sysv 2023-03-29T10:41:24.024 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/sysv/sysv.ko 2023-03-29T10:41:24.025 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/udf 2023-03-29T10:41:24.025 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/udf/udf.ko 2023-03-29T10:41:24.025 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ufs 2023-03-29T10:41:24.025 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ufs/ufs.ko 2023-03-29T10:41:24.025 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/xfs 2023-03-29T10:41:24.025 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/xfs/xfs.ko 2023-03-29T10:41:24.025 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/kernel 2023-03-29T10:41:24.025 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/kernel/rcu 2023-03-29T10:41:24.025 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/kernel/rcu/rcutorture.ko 2023-03-29T10:41:24.025 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/kernel/torture.ko 2023-03-29T10:41:24.025 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/kernel/trace 2023-03-29T10:41:24.025 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-03-29T10:41:24.025 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib 2023-03-29T10:41:24.025 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/asn1_encoder.ko 2023-03-29T10:41:24.026 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/crc-itu-t.ko 2023-03-29T10:41:24.026 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/crc8.ko 2023-03-29T10:41:24.026 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/crypto 2023-03-29T10:41:24.026 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/crypto/libarc4.ko 2023-03-29T10:41:24.026 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/crypto/libdes.ko 2023-03-29T10:41:24.026 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/libcrc32c.ko 2023-03-29T10:41:24.026 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/lru_cache.ko 2023-03-29T10:41:24.026 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/lz4 2023-03-29T10:41:24.026 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/lz4/lz4_compress.ko 2023-03-29T10:41:24.026 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/lz4/lz4hc_compress.ko 2023-03-29T10:41:24.026 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/math 2023-03-29T10:41:24.026 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/math/cordic.ko 2023-03-29T10:41:24.026 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/raid6 2023-03-29T10:41:24.026 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/raid6/raid6_pq.ko 2023-03-29T10:41:24.027 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/reed_solomon 2023-03-29T10:41:24.027 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/reed_solomon/reed_solomon.ko 2023-03-29T10:41:24.027 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/ts_bm.ko 2023-03-29T10:41:24.027 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/ts_fsm.ko 2023-03-29T10:41:24.027 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/ts_kmp.ko 2023-03-29T10:41:24.027 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/zstd 2023-03-29T10:41:24.027 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/zstd/zstd_compress.ko 2023-03-29T10:41:24.027 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/mm 2023-03-29T10:41:24.027 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/mm/hwpoison-inject.ko 2023-03-29T10:41:24.027 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net 2023-03-29T10:41:24.027 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan 2023-03-29T10:41:24.027 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/6lowpan.ko 2023-03-29T10:41:24.027 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_dest.ko 2023-03-29T10:41:24.027 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_fragment.ko 2023-03-29T10:41:24.027 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_hop.ko 2023-03-29T10:41:24.028 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_ipv6.ko 2023-03-29T10:41:24.028 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_mobility.ko 2023-03-29T10:41:24.028 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_routing.ko 2023-03-29T10:41:24.028 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_udp.ko 2023-03-29T10:41:24.028 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/802 2023-03-29T10:41:24.028 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/802/garp.ko 2023-03-29T10:41:24.028 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/802/mrp.ko 2023-03-29T10:41:24.028 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/802/p8022.ko 2023-03-29T10:41:24.028 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/802/psnap.ko 2023-03-29T10:41:24.028 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/802/stp.ko 2023-03-29T10:41:24.028 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/8021q 2023-03-29T10:41:24.028 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/8021q/8021q.ko 2023-03-29T10:41:24.028 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/9p 2023-03-29T10:41:24.028 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/9p/9pnet.ko 2023-03-29T10:41:24.028 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/9p/9pnet_fd.ko 2023-03-29T10:41:24.029 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/9p/9pnet_rdma.ko 2023-03-29T10:41:24.029 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/9p/9pnet_virtio.ko 2023-03-29T10:41:24.029 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/batman-adv 2023-03-29T10:41:24.029 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/batman-adv/batman-adv.ko 2023-03-29T10:41:24.029 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge 2023-03-29T10:41:24.029 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/br_netfilter.ko 2023-03-29T10:41:24.029 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/bridge.ko 2023-03-29T10:41:24.029 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter 2023-03-29T10:41:24.029 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-03-29T10:41:24.029 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_among.ko 2023-03-29T10:41:24.029 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_arp.ko 2023-03-29T10:41:24.029 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-03-29T10:41:24.029 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-03-29T10:41:24.029 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_ip.ko 2023-03-29T10:41:24.029 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-03-29T10:41:24.030 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_limit.ko 2023-03-29T10:41:24.030 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_log.ko 2023-03-29T10:41:24.030 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_mark.ko 2023-03-29T10:41:24.030 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-03-29T10:41:24.030 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-03-29T10:41:24.030 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-03-29T10:41:24.030 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-03-29T10:41:24.030 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_snat.ko 2023-03-29T10:41:24.030 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_stp.ko 2023-03-29T10:41:24.030 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-03-29T10:41:24.030 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-03-29T10:41:24.030 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-03-29T10:41:24.030 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-03-29T10:41:24.030 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebtables.ko 2023-03-29T10:41:24.031 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-03-29T10:41:24.031 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-03-29T10:41:24.031 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-03-29T10:41:24.031 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ceph 2023-03-29T10:41:24.031 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ceph/libceph.ko 2023-03-29T10:41:24.031 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/core 2023-03-29T10:41:24.031 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/core/failover.ko 2023-03-29T10:41:24.031 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/core/pktgen.ko 2023-03-29T10:41:24.031 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dccp 2023-03-29T10:41:24.031 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dccp/dccp.ko 2023-03-29T10:41:24.031 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dccp/dccp_diag.ko 2023-03-29T10:41:24.031 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dccp/dccp_ipv4.ko 2023-03-29T10:41:24.031 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dccp/dccp_ipv6.ko 2023-03-29T10:41:24.031 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dns_resolver 2023-03-29T10:41:24.031 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dns_resolver/dns_resolver.ko 2023-03-29T10:41:24.032 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dsa 2023-03-29T10:41:24.032 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dsa/dsa_core.ko 2023-03-29T10:41:24.032 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dsa/tag_brcm.ko 2023-03-29T10:41:24.032 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dsa/tag_dsa.ko 2023-03-29T10:41:24.032 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dsa/tag_none.ko 2023-03-29T10:41:24.032 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dsa/tag_trailer.ko 2023-03-29T10:41:24.032 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ieee802154 2023-03-29T10:41:24.032 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ieee802154/6lowpan 2023-03-29T10:41:24.032 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-03-29T10:41:24.032 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ieee802154/ieee802154.ko 2023-03-29T10:41:24.032 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ieee802154/ieee802154_socket.ko 2023-03-29T10:41:24.032 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4 2023-03-29T10:41:24.032 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/ah4.ko 2023-03-29T10:41:24.032 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/esp4.ko 2023-03-29T10:41:24.032 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/fou.ko 2023-03-29T10:41:24.033 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/gre.ko 2023-03-29T10:41:24.033 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/inet_diag.ko 2023-03-29T10:41:24.033 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/ip_gre.ko 2023-03-29T10:41:24.033 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/ip_tunnel.ko 2023-03-29T10:41:24.033 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/ip_vti.ko 2023-03-29T10:41:24.033 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/ipcomp.ko 2023-03-29T10:41:24.033 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/ipip.ko 2023-03-29T10:41:24.033 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter 2023-03-29T10:41:24.033 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/arp_tables.ko 2023-03-29T10:41:24.033 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-03-29T10:41:24.033 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-03-29T10:41:24.033 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-03-29T10:41:24.033 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-03-29T10:41:24.033 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-03-29T10:41:24.033 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-03-29T10:41:24.034 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-03-29T10:41:24.034 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-03-29T10:41:24.034 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-03-29T10:41:24.034 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/iptable_security.ko 2023-03-29T10:41:24.034 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-03-29T10:41:24.034 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-03-29T10:41:24.034 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-03-29T10:41:24.034 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-03-29T10:41:24.034 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-03-29T10:41:24.034 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-03-29T10:41:24.034 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-03-29T10:41:24.034 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-03-29T10:41:24.034 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-03-29T10:41:24.034 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-03-29T10:41:24.034 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_bic.ko 2023-03-29T10:41:24.035 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_dctcp.ko 2023-03-29T10:41:24.035 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_diag.ko 2023-03-29T10:41:24.035 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_highspeed.ko 2023-03-29T10:41:24.035 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_htcp.ko 2023-03-29T10:41:24.035 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_hybla.ko 2023-03-29T10:41:24.035 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_illinois.ko 2023-03-29T10:41:24.035 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_lp.ko 2023-03-29T10:41:24.035 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_scalable.ko 2023-03-29T10:41:24.035 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_vegas.ko 2023-03-29T10:41:24.035 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_veno.ko 2023-03-29T10:41:24.035 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_westwood.ko 2023-03-29T10:41:24.035 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_yeah.ko 2023-03-29T10:41:24.035 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tunnel4.ko 2023-03-29T10:41:24.035 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/udp_diag.ko 2023-03-29T10:41:24.035 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/udp_tunnel.ko 2023-03-29T10:41:24.035 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/xfrm4_tunnel.ko 2023-03-29T10:41:24.036 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6 2023-03-29T10:41:24.036 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/ah6.ko 2023-03-29T10:41:24.036 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/esp6.ko 2023-03-29T10:41:24.036 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/fou6.ko 2023-03-29T10:41:24.036 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/ip6_tunnel.ko 2023-03-29T10:41:24.036 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-03-29T10:41:24.036 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/ip6_vti.ko 2023-03-29T10:41:24.036 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/ipcomp6.ko 2023-03-29T10:41:24.036 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter 2023-03-29T10:41:24.036 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-03-29T10:41:24.036 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-03-29T10:41:24.036 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-03-29T10:41:24.036 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-03-29T10:41:24.036 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-03-29T10:41:24.036 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-03-29T10:41:24.037 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-03-29T10:41:24.037 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-03-29T10:41:24.037 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-03-29T10:41:24.037 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-03-29T10:41:24.037 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-03-29T10:41:24.037 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-03-29T10:41:24.037 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-03-29T10:41:24.037 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-03-29T10:41:24.037 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-03-29T10:41:24.037 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-03-29T10:41:24.037 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-03-29T10:41:24.037 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-03-29T10:41:24.037 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-03-29T10:41:24.037 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-03-29T10:41:24.037 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-03-29T10:41:24.038 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-03-29T10:41:24.038 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-03-29T10:41:24.038 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-03-29T10:41:24.038 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/sit.ko 2023-03-29T10:41:24.038 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/tunnel6.ko 2023-03-29T10:41:24.038 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/xfrm6_tunnel.ko 2023-03-29T10:41:24.038 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/key 2023-03-29T10:41:24.038 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/key/af_key.ko 2023-03-29T10:41:24.038 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp 2023-03-29T10:41:24.038 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_core.ko 2023-03-29T10:41:24.038 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_debugfs.ko 2023-03-29T10:41:24.038 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_eth.ko 2023-03-29T10:41:24.038 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_ip.ko 2023-03-29T10:41:24.038 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_ip6.ko 2023-03-29T10:41:24.038 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_netlink.ko 2023-03-29T10:41:24.039 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_ppp.ko 2023-03-29T10:41:24.039 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/llc 2023-03-29T10:41:24.039 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/llc/llc.ko 2023-03-29T10:41:24.039 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/mac802154 2023-03-29T10:41:24.039 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/mac802154/mac802154.ko 2023-03-29T10:41:24.039 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/mpls 2023-03-29T10:41:24.039 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/mpls/mpls_gso.ko 2023-03-29T10:41:24.039 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter 2023-03-29T10:41:24.039 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset 2023-03-29T10:41:24.039 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set.ko 2023-03-29T10:41:24.039 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-03-29T10:41:24.039 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-03-29T10:41:24.039 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-03-29T10:41:24.039 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-03-29T10:41:24.039 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-03-29T10:41:24.040 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-03-29T10:41:24.040 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-03-29T10:41:24.040 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-03-29T10:41:24.040 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-03-29T10:41:24.040 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-03-29T10:41:24.040 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-03-29T10:41:24.040 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-03-29T10:41:24.040 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-03-29T10:41:24.040 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-03-29T10:41:24.040 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-03-29T10:41:24.040 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs 2023-03-29T10:41:24.040 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs.ko 2023-03-29T10:41:24.040 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-03-29T10:41:24.040 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-03-29T10:41:24.040 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-03-29T10:41:24.041 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-03-29T10:41:24.041 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-03-29T10:41:24.041 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-03-29T10:41:24.041 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-03-29T10:41:24.041 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-03-29T10:41:24.041 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-03-29T10:41:24.041 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-03-29T10:41:24.041 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-03-29T10:41:24.041 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-03-29T10:41:24.041 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-03-29T10:41:24.041 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conncount.ko 2023-03-29T10:41:24.041 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack.ko 2023-03-29T10:41:24.041 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-03-29T10:41:24.041 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-03-29T10:41:24.041 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-03-29T10:41:24.042 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_h323.ko 2023-03-29T10:41:24.042 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_irc.ko 2023-03-29T10:41:24.042 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-03-29T10:41:24.042 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-03-29T10:41:24.042 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-03-29T10:41:24.042 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_sane.ko 2023-03-29T10:41:24.042 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_sip.ko 2023-03-29T10:41:24.042 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-03-29T10:41:24.042 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-03-29T10:41:24.042 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_dup_netdev.ko 2023-03-29T10:41:24.042 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_flow_table.ko 2023-03-29T10:41:24.042 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_flow_table_inet.ko 2023-03-29T10:41:24.042 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_log_syslog.ko 2023-03-29T10:41:24.042 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_nat.ko 2023-03-29T10:41:24.042 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_nat_amanda.ko 2023-03-29T10:41:24.043 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_nat_ftp.ko 2023-03-29T10:41:24.043 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_nat_irc.ko 2023-03-29T10:41:24.043 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_nat_sip.ko 2023-03-29T10:41:24.043 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_nat_tftp.ko 2023-03-29T10:41:24.043 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_synproxy_core.ko 2023-03-29T10:41:24.043 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_tables.ko 2023-03-29T10:41:24.043 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nfnetlink.ko 2023-03-29T10:41:24.043 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nfnetlink_acct.ko 2023-03-29T10:41:24.043 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nfnetlink_log.ko 2023-03-29T10:41:24.043 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nfnetlink_osf.ko 2023-03-29T10:41:24.043 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nfnetlink_queue.ko 2023-03-29T10:41:24.043 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_chain_nat.ko 2023-03-29T10:41:24.043 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_compat.ko 2023-03-29T10:41:24.043 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_ct.ko 2023-03-29T10:41:24.043 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_dup_netdev.ko 2023-03-29T10:41:24.044 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_fib.ko 2023-03-29T10:41:24.044 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_fwd_netdev.ko 2023-03-29T10:41:24.044 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_hash.ko 2023-03-29T10:41:24.044 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_limit.ko 2023-03-29T10:41:24.044 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_log.ko 2023-03-29T10:41:24.044 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_masq.ko 2023-03-29T10:41:24.044 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_nat.ko 2023-03-29T10:41:24.044 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_numgen.ko 2023-03-29T10:41:24.044 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_queue.ko 2023-03-29T10:41:24.044 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_quota.ko 2023-03-29T10:41:24.044 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_redir.ko 2023-03-29T10:41:24.044 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_reject.ko 2023-03-29T10:41:24.044 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_reject_inet.ko 2023-03-29T10:41:24.044 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_socket.ko 2023-03-29T10:41:24.044 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_tunnel.ko 2023-03-29T10:41:24.045 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_AUDIT.ko 2023-03-29T10:41:24.045 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_CHECKSUM.ko 2023-03-29T10:41:24.045 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_CLASSIFY.ko 2023-03-29T10:41:24.045 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-03-29T10:41:24.045 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_CT.ko 2023-03-29T10:41:24.045 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_DSCP.ko 2023-03-29T10:41:24.045 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_HL.ko 2023-03-29T10:41:24.045 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_HMARK.ko 2023-03-29T10:41:24.045 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_IDLETIMER.ko 2023-03-29T10:41:24.045 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_LED.ko 2023-03-29T10:41:24.045 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_LOG.ko 2023-03-29T10:41:24.045 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_MASQUERADE.ko 2023-03-29T10:41:24.045 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_NETMAP.ko 2023-03-29T10:41:24.045 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_NFLOG.ko 2023-03-29T10:41:24.045 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_NFQUEUE.ko 2023-03-29T10:41:24.046 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_RATEEST.ko 2023-03-29T10:41:24.046 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_REDIRECT.ko 2023-03-29T10:41:24.046 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_SECMARK.ko 2023-03-29T10:41:24.046 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_TCPMSS.ko 2023-03-29T10:41:24.046 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-03-29T10:41:24.046 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_TEE.ko 2023-03-29T10:41:24.046 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_TPROXY.ko 2023-03-29T10:41:24.046 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_TRACE.ko 2023-03-29T10:41:24.046 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_addrtype.ko 2023-03-29T10:41:24.046 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_bpf.ko 2023-03-29T10:41:24.046 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_cgroup.ko 2023-03-29T10:41:24.046 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_cluster.ko 2023-03-29T10:41:24.046 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_comment.ko 2023-03-29T10:41:24.046 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_connbytes.ko 2023-03-29T10:41:24.046 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_connlabel.ko 2023-03-29T10:41:24.047 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_connlimit.ko 2023-03-29T10:41:24.047 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_connmark.ko 2023-03-29T10:41:24.047 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_conntrack.ko 2023-03-29T10:41:24.047 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_cpu.ko 2023-03-29T10:41:24.047 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_dccp.ko 2023-03-29T10:41:24.047 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_devgroup.ko 2023-03-29T10:41:24.047 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_dscp.ko 2023-03-29T10:41:24.047 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_ecn.ko 2023-03-29T10:41:24.047 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_esp.ko 2023-03-29T10:41:24.047 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_hashlimit.ko 2023-03-29T10:41:24.047 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_helper.ko 2023-03-29T10:41:24.047 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_hl.ko 2023-03-29T10:41:24.047 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_ipcomp.ko 2023-03-29T10:41:24.047 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_iprange.ko 2023-03-29T10:41:24.047 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_ipvs.ko 2023-03-29T10:41:24.048 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_l2tp.ko 2023-03-29T10:41:24.048 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_length.ko 2023-03-29T10:41:24.048 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_limit.ko 2023-03-29T10:41:24.048 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_mac.ko 2023-03-29T10:41:24.048 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_mark.ko 2023-03-29T10:41:24.048 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_multiport.ko 2023-03-29T10:41:24.048 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_nat.ko 2023-03-29T10:41:24.048 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_nfacct.ko 2023-03-29T10:41:24.048 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_osf.ko 2023-03-29T10:41:24.048 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_owner.ko 2023-03-29T10:41:24.048 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_physdev.ko 2023-03-29T10:41:24.048 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_pkttype.ko 2023-03-29T10:41:24.048 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_policy.ko 2023-03-29T10:41:24.048 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_quota.ko 2023-03-29T10:41:24.048 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_rateest.ko 2023-03-29T10:41:24.049 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_realm.ko 2023-03-29T10:41:24.049 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_recent.ko 2023-03-29T10:41:24.049 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_sctp.ko 2023-03-29T10:41:24.049 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_set.ko 2023-03-29T10:41:24.049 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_socket.ko 2023-03-29T10:41:24.049 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_state.ko 2023-03-29T10:41:24.049 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_statistic.ko 2023-03-29T10:41:24.049 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_string.ko 2023-03-29T10:41:24.049 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_tcpmss.ko 2023-03-29T10:41:24.049 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_time.ko 2023-03-29T10:41:24.049 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_u32.ko 2023-03-29T10:41:24.049 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netlink 2023-03-29T10:41:24.049 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netlink/netlink_diag.ko 2023-03-29T10:41:24.049 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/nsh 2023-03-29T10:41:24.050 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/nsh/nsh.ko 2023-03-29T10:41:24.050 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/openvswitch 2023-03-29T10:41:24.050 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/openvswitch/openvswitch.ko 2023-03-29T10:41:24.050 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/openvswitch/vport-geneve.ko 2023-03-29T10:41:24.050 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/openvswitch/vport-gre.ko 2023-03-29T10:41:24.050 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/openvswitch/vport-vxlan.ko 2023-03-29T10:41:24.050 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/packet 2023-03-29T10:41:24.050 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/packet/af_packet_diag.ko 2023-03-29T10:41:24.050 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/rds 2023-03-29T10:41:24.050 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/rds/rds.ko 2023-03-29T10:41:24.050 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/rds/rds_rdma.ko 2023-03-29T10:41:24.050 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/rds/rds_tcp.ko 2023-03-29T10:41:24.050 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched 2023-03-29T10:41:24.050 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_csum.ko 2023-03-29T10:41:24.050 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_gact.ko 2023-03-29T10:41:24.050 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_ipt.ko 2023-03-29T10:41:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_mirred.ko 2023-03-29T10:41:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_nat.ko 2023-03-29T10:41:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_pedit.ko 2023-03-29T10:41:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_police.ko 2023-03-29T10:41:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_simple.ko 2023-03-29T10:41:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_skbedit.ko 2023-03-29T10:41:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_vlan.ko 2023-03-29T10:41:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/cls_basic.ko 2023-03-29T10:41:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/cls_bpf.ko 2023-03-29T10:41:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/cls_flow.ko 2023-03-29T10:41:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/cls_fw.ko 2023-03-29T10:41:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/cls_route.ko 2023-03-29T10:41:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/cls_u32.ko 2023-03-29T10:41:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/em_cmp.ko 2023-03-29T10:41:24.051 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/em_ipset.ko 2023-03-29T10:41:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/em_meta.ko 2023-03-29T10:41:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/em_nbyte.ko 2023-03-29T10:41:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/em_text.ko 2023-03-29T10:41:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/em_u32.ko 2023-03-29T10:41:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_choke.ko 2023-03-29T10:41:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_codel.ko 2023-03-29T10:41:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_drr.ko 2023-03-29T10:41:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_fq.ko 2023-03-29T10:41:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_gred.ko 2023-03-29T10:41:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_hfsc.ko 2023-03-29T10:41:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_hhf.ko 2023-03-29T10:41:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_htb.ko 2023-03-29T10:41:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_ingress.ko 2023-03-29T10:41:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_mqprio.ko 2023-03-29T10:41:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_mqprio_lib.ko 2023-03-29T10:41:24.052 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_multiq.ko 2023-03-29T10:41:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_netem.ko 2023-03-29T10:41:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_pie.ko 2023-03-29T10:41:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_plug.ko 2023-03-29T10:41:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_prio.ko 2023-03-29T10:41:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_qfq.ko 2023-03-29T10:41:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_red.ko 2023-03-29T10:41:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_sfb.ko 2023-03-29T10:41:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_sfq.ko 2023-03-29T10:41:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_tbf.ko 2023-03-29T10:41:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_teql.ko 2023-03-29T10:41:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sctp 2023-03-29T10:41:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sctp/sctp.ko 2023-03-29T10:41:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sctp/sctp_diag.ko 2023-03-29T10:41:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc 2023-03-29T10:41:24.053 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc/auth_gss 2023-03-29T10:41:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-03-29T10:41:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-03-29T10:41:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc/sunrpc.ko 2023-03-29T10:41:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc/xprtrdma 2023-03-29T10:41:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-03-29T10:41:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/tipc 2023-03-29T10:41:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/tipc/diag.ko 2023-03-29T10:41:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/tipc/tipc.ko 2023-03-29T10:41:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/unix 2023-03-29T10:41:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/unix/unix_diag.ko 2023-03-29T10:41:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/vmw_vsock 2023-03-29T10:41:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-03-29T10:41:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-03-29T10:41:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/vmw_vsock/vsock.ko 2023-03-29T10:41:24.054 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/vmw_vsock/vsock_diag.ko 2023-03-29T10:41:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/vmw_vsock/vsock_loopback.ko 2023-03-29T10:41:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/xfrm 2023-03-29T10:41:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/xfrm/xfrm_ipcomp.ko 2023-03-29T10:41:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/security 2023-03-29T10:41:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/security/keys 2023-03-29T10:41:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/security/keys/encrypted-keys 2023-03-29T10:41:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-03-29T10:41:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/security/keys/trusted-keys 2023-03-29T10:41:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/security/keys/trusted-keys/trusted.ko 2023-03-29T10:41:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/virt 2023-03-29T10:41:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/virt/lib 2023-03-29T10:41:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/virt/lib/irqbypass.ko 2023-03-29T10:41:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.alias 2023-03-29T10:41:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.alias.bin 2023-03-29T10:41:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.builtin 2023-03-29T10:41:24.055 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.builtin.bin 2023-03-29T10:41:24.056 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.builtin.modinfo 2023-03-29T10:41:24.056 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.dep 2023-03-29T10:41:24.056 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.dep.bin 2023-03-29T10:41:24.056 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.devname 2023-03-29T10:41:24.056 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.order 2023-03-29T10:41:24.056 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.softdep 2023-03-29T10:41:24.056 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.symbols 2023-03-29T10:41:24.056 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.symbols.bin 2023-03-29T10:41:24.057 DEBUG:teuthology.task.kernel:get_image_version: 6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:24.057 DEBUG:teuthology.orchestra.run.smithi119:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-03-29T10:41:24.082 INFO:teuthology.orchestra.run.smithi119.stdout:postinstall scriptlet (using /bin/sh): 2023-03-29T10:41:24.082 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.3.0-rc3-g1776e8f494e9 -a -r /boot/System.map-6.3.0-rc3-g1776e8f494e9 ]; then 2023-03-29T10:41:24.082 INFO:teuthology.orchestra.run.smithi119.stdout:cp /boot/vmlinuz-6.3.0-rc3-g1776e8f494e9 /boot/.vmlinuz-6.3.0-rc3-g1776e8f494e9-rpm 2023-03-29T10:41:24.082 INFO:teuthology.orchestra.run.smithi119.stdout:cp /boot/System.map-6.3.0-rc3-g1776e8f494e9 /boot/.System.map-6.3.0-rc3-g1776e8f494e9-rpm 2023-03-29T10:41:24.083 INFO:teuthology.orchestra.run.smithi119.stdout:rm -f /boot/vmlinuz-6.3.0-rc3-g1776e8f494e9 /boot/System.map-6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:24.083 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/installkernel 6.3.0-rc3-g1776e8f494e9 /boot/.vmlinuz-6.3.0-rc3-g1776e8f494e9-rpm /boot/.System.map-6.3.0-rc3-g1776e8f494e9-rpm 2023-03-29T10:41:24.083 INFO:teuthology.orchestra.run.smithi119.stdout:rm -f /boot/.vmlinuz-6.3.0-rc3-g1776e8f494e9-rpm /boot/.System.map-6.3.0-rc3-g1776e8f494e9-rpm 2023-03-29T10:41:24.083 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2023-03-29T10:41:24.083 INFO:teuthology.orchestra.run.smithi119.stdout:preuninstall scriptlet (using /bin/sh): 2023-03-29T10:41:24.083 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-03-29T10:41:24.083 INFO:teuthology.orchestra.run.smithi119.stdout:new-kernel-pkg --remove 6.3.0-rc3-g1776e8f494e9 --rminitrd --initrdfile=/boot/initramfs-6.3.0-rc3-g1776e8f494e9.img 2023-03-29T10:41:24.083 INFO:teuthology.orchestra.run.smithi119.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-03-29T10:41:24.083 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-install remove 6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:24.083 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2023-03-29T10:41:24.083 INFO:teuthology.orchestra.run.smithi119.stdout:postuninstall scriptlet (using /bin/sh): 2023-03-29T10:41:24.083 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /sbin/update-bootloader ]; then 2023-03-29T10:41:24.083 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/update-bootloader --remove 6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:24.083 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2023-03-29T10:41:24.084 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -qi grub2-tools 2023-03-29T10:41:24.242 INFO:teuthology.orchestra.run.smithi119.stdout:Name : grub2-tools 2023-03-29T10:41:24.242 INFO:teuthology.orchestra.run.smithi119.stdout:Epoch : 1 2023-03-29T10:41:24.242 INFO:teuthology.orchestra.run.smithi119.stdout:Version : 2.02 2023-03-29T10:41:24.242 INFO:teuthology.orchestra.run.smithi119.stdout:Release : 129.el8 2023-03-29T10:41:24.242 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: x86_64 2023-03-29T10:41:24.243 INFO:teuthology.orchestra.run.smithi119.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-03-29T10:41:24.243 INFO:teuthology.orchestra.run.smithi119.stdout:Group : System Environment/Base 2023-03-29T10:41:24.243 INFO:teuthology.orchestra.run.smithi119.stdout:Size : 9265586 2023-03-29T10:41:24.243 INFO:teuthology.orchestra.run.smithi119.stdout:License : GPLv3+ 2023-03-29T10:41:24.243 INFO:teuthology.orchestra.run.smithi119.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-03-29T10:41:24.243 INFO:teuthology.orchestra.run.smithi119.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-03-29T10:41:24.243 INFO:teuthology.orchestra.run.smithi119.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-03-29T10:41:24.243 INFO:teuthology.orchestra.run.smithi119.stdout:Build Host : kbuilder.bsys.centos.org 2023-03-29T10:41:24.243 INFO:teuthology.orchestra.run.smithi119.stdout:Relocations : (not relocatable) 2023-03-29T10:41:24.243 INFO:teuthology.orchestra.run.smithi119.stdout:Packager : CentOS BuildSystem 2023-03-29T10:41:24.243 INFO:teuthology.orchestra.run.smithi119.stdout:Vendor : CentOS 2023-03-29T10:41:24.243 INFO:teuthology.orchestra.run.smithi119.stdout:URL : http://www.gnu.org/software/grub/ 2023-03-29T10:41:24.243 INFO:teuthology.orchestra.run.smithi119.stdout:Summary : Support tools for GRUB. 2023-03-29T10:41:24.243 INFO:teuthology.orchestra.run.smithi119.stdout:Description : 2023-03-29T10:41:24.243 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:41:24.244 INFO:teuthology.orchestra.run.smithi119.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-03-29T10:41:24.244 INFO:teuthology.orchestra.run.smithi119.stdout:customizable bootloader with modular architecture. It supports a rich 2023-03-29T10:41:24.244 INFO:teuthology.orchestra.run.smithi119.stdout:variety of kernel formats, file systems, computer architectures and 2023-03-29T10:41:24.244 INFO:teuthology.orchestra.run.smithi119.stdout:hardware devices. 2023-03-29T10:41:24.244 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:41:24.244 INFO:teuthology.orchestra.run.smithi119.stdout:This subpackage provides tools for support of all platforms. 2023-03-29T10:41:24.245 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-03-29T10:41:24.245 INFO:teuthology.task.kernel:Updating grub on smithi119 to boot 6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:24.245 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-03-29T10:41:26.563 INFO:teuthology.orchestra.run.smithi119.stderr:Generating grub configuration file ... 2023-03-29T10:41:28.228 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi111.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-03-29T10:41:28.228 DEBUG:teuthology.orchestra.run.smithi111:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-03-29T10:41:28.280 INFO:teuthology.orchestra.run.smithi111.stdout:/boot/System.map-6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:28.280 INFO:teuthology.orchestra.run.smithi111.stdout:/boot/config-6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:28.280 INFO:teuthology.orchestra.run.smithi111.stdout:/boot/vmlinuz-6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:28.280 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:28.280 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel 2023-03-29T10:41:28.280 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch 2023-03-29T10:41:28.280 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86 2023-03-29T10:41:28.280 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto 2023-03-29T10:41:28.280 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-03-29T10:41:28.280 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-03-29T10:41:28.280 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-03-29T10:41:28.281 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-03-29T10:41:28.281 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-03-29T10:41:28.281 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-03-29T10:41:28.281 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-03-29T10:41:28.281 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/crc32c-intel.ko 2023-03-29T10:41:28.281 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-03-29T10:41:28.281 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-03-29T10:41:28.281 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-03-29T10:41:28.281 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-03-29T10:41:28.281 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/serpent-avx2.ko 2023-03-29T10:41:28.281 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-03-29T10:41:28.281 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-03-29T10:41:28.281 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-03-29T10:41:28.281 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-03-29T10:41:28.281 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-03-29T10:41:28.281 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-03-29T10:41:28.281 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-03-29T10:41:28.282 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kernel 2023-03-29T10:41:28.282 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kernel/cpu 2023-03-29T10:41:28.282 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kernel/cpu/mce 2023-03-29T10:41:28.282 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-03-29T10:41:28.282 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kvm 2023-03-29T10:41:28.282 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kvm/kvm-amd.ko 2023-03-29T10:41:28.282 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kvm/kvm-intel.ko 2023-03-29T10:41:28.282 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/arch/x86/kvm/kvm.ko 2023-03-29T10:41:28.282 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto 2023-03-29T10:41:28.282 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/ansi_cprng.ko 2023-03-29T10:41:28.282 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/anubis.ko 2023-03-29T10:41:28.282 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/arc4.ko 2023-03-29T10:41:28.282 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx 2023-03-29T10:41:28.282 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx/async_memcpy.ko 2023-03-29T10:41:28.282 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx/async_pq.ko 2023-03-29T10:41:28.282 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx/async_raid6_recov.ko 2023-03-29T10:41:28.282 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx/async_tx.ko 2023-03-29T10:41:28.283 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx/async_xor.ko 2023-03-29T10:41:28.283 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/async_tx/raid6test.ko 2023-03-29T10:41:28.283 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/authenc.ko 2023-03-29T10:41:28.283 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/authencesn.ko 2023-03-29T10:41:28.283 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/blake2b_generic.ko 2023-03-29T10:41:28.283 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/blowfish_common.ko 2023-03-29T10:41:28.283 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/blowfish_generic.ko 2023-03-29T10:41:28.283 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/camellia_generic.ko 2023-03-29T10:41:28.283 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/cast5_generic.ko 2023-03-29T10:41:28.283 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/cast6_generic.ko 2023-03-29T10:41:28.283 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/cast_common.ko 2023-03-29T10:41:28.283 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/ccm.ko 2023-03-29T10:41:28.283 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/cmac.ko 2023-03-29T10:41:28.283 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/crc32_generic.ko 2023-03-29T10:41:28.283 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/crypto_engine.ko 2023-03-29T10:41:28.283 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/crypto_user.ko 2023-03-29T10:41:28.283 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/cts.ko 2023-03-29T10:41:28.284 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/des_generic.ko 2023-03-29T10:41:28.284 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/dh_generic.ko 2023-03-29T10:41:28.284 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/echainiv.ko 2023-03-29T10:41:28.284 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/essiv.ko 2023-03-29T10:41:28.284 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/fcrypt.ko 2023-03-29T10:41:28.284 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/khazad.ko 2023-03-29T10:41:28.284 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/lz4.ko 2023-03-29T10:41:28.284 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/lz4hc.ko 2023-03-29T10:41:28.284 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/md4.ko 2023-03-29T10:41:28.284 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/michael_mic.ko 2023-03-29T10:41:28.284 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/pcbc.ko 2023-03-29T10:41:28.284 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/pcrypt.ko 2023-03-29T10:41:28.284 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/rmd160.ko 2023-03-29T10:41:28.284 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/seed.ko 2023-03-29T10:41:28.284 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/serpent_generic.ko 2023-03-29T10:41:28.284 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/tcrypt.ko 2023-03-29T10:41:28.284 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/tea.ko 2023-03-29T10:41:28.285 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/twofish_common.ko 2023-03-29T10:41:28.285 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/twofish_generic.ko 2023-03-29T10:41:28.285 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/vmac.ko 2023-03-29T10:41:28.285 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/wp512.ko 2023-03-29T10:41:28.285 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/xcbc.ko 2023-03-29T10:41:28.285 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/xor.ko 2023-03-29T10:41:28.285 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/crypto/xxhash_generic.ko 2023-03-29T10:41:28.285 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers 2023-03-29T10:41:28.285 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi 2023-03-29T10:41:28.285 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/acpi_ipmi.ko 2023-03-29T10:41:28.285 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/acpi_pad.ko 2023-03-29T10:41:28.285 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/custom_method.ko 2023-03-29T10:41:28.285 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/ec_sys.ko 2023-03-29T10:41:28.285 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/platform_profile.ko 2023-03-29T10:41:28.285 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/sbs.ko 2023-03-29T10:41:28.285 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/sbshc.ko 2023-03-29T10:41:28.285 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/acpi/video.ko 2023-03-29T10:41:28.286 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata 2023-03-29T10:41:28.286 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/acard-ahci.ko 2023-03-29T10:41:28.286 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/ahci_platform.ko 2023-03-29T10:41:28.286 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/ata_generic.ko 2023-03-29T10:41:28.286 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/libahci_platform.ko 2023-03-29T10:41:28.286 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_acpi.ko 2023-03-29T10:41:28.286 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_ali.ko 2023-03-29T10:41:28.286 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_amd.ko 2023-03-29T10:41:28.286 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_artop.ko 2023-03-29T10:41:28.286 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_atiixp.ko 2023-03-29T10:41:28.286 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_atp867x.ko 2023-03-29T10:41:28.286 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_cmd640.ko 2023-03-29T10:41:28.286 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_cmd64x.ko 2023-03-29T10:41:28.286 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_cypress.ko 2023-03-29T10:41:28.286 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_efar.ko 2023-03-29T10:41:28.286 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_hpt366.ko 2023-03-29T10:41:28.286 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_hpt37x.ko 2023-03-29T10:41:28.286 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_hpt3x2n.ko 2023-03-29T10:41:28.287 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_hpt3x3.ko 2023-03-29T10:41:28.287 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_it8213.ko 2023-03-29T10:41:28.287 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_it821x.ko 2023-03-29T10:41:28.287 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_jmicron.ko 2023-03-29T10:41:28.287 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_marvell.ko 2023-03-29T10:41:28.287 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_mpiix.ko 2023-03-29T10:41:28.287 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_netcell.ko 2023-03-29T10:41:28.287 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_ninja32.ko 2023-03-29T10:41:28.287 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_ns87410.ko 2023-03-29T10:41:28.287 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_ns87415.ko 2023-03-29T10:41:28.287 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_oldpiix.ko 2023-03-29T10:41:28.287 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_opti.ko 2023-03-29T10:41:28.287 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_optidma.ko 2023-03-29T10:41:28.287 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_pcmcia.ko 2023-03-29T10:41:28.287 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_pdc2027x.ko 2023-03-29T10:41:28.287 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-03-29T10:41:28.287 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_piccolo.ko 2023-03-29T10:41:28.288 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_rdc.ko 2023-03-29T10:41:28.288 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_sch.ko 2023-03-29T10:41:28.288 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_serverworks.ko 2023-03-29T10:41:28.288 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_sil680.ko 2023-03-29T10:41:28.288 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_sis.ko 2023-03-29T10:41:28.288 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_sl82c105.ko 2023-03-29T10:41:28.288 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_triflex.ko 2023-03-29T10:41:28.288 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pata_via.ko 2023-03-29T10:41:28.288 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/pdc_adma.ko 2023-03-29T10:41:28.288 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_inic162x.ko 2023-03-29T10:41:28.288 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_mv.ko 2023-03-29T10:41:28.288 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_nv.ko 2023-03-29T10:41:28.288 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_promise.ko 2023-03-29T10:41:28.288 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_qstor.ko 2023-03-29T10:41:28.288 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_sil.ko 2023-03-29T10:41:28.288 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_sil24.ko 2023-03-29T10:41:28.288 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_sis.ko 2023-03-29T10:41:28.289 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_svw.ko 2023-03-29T10:41:28.289 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_sx4.ko 2023-03-29T10:41:28.289 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_uli.ko 2023-03-29T10:41:28.289 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_via.ko 2023-03-29T10:41:28.289 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ata/sata_vsc.ko 2023-03-29T10:41:28.289 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/auxdisplay 2023-03-29T10:41:28.289 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/auxdisplay/ks0108.ko 2023-03-29T10:41:28.289 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/base 2023-03-29T10:41:28.289 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/base/regmap 2023-03-29T10:41:28.289 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/base/regmap/regmap-i2c.ko 2023-03-29T10:41:28.289 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/bcma 2023-03-29T10:41:28.289 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/bcma/bcma.ko 2023-03-29T10:41:28.289 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block 2023-03-29T10:41:28.289 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/aoe 2023-03-29T10:41:28.289 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/aoe/aoe.ko 2023-03-29T10:41:28.289 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/brd.ko 2023-03-29T10:41:28.289 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/drbd 2023-03-29T10:41:28.290 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/drbd/drbd.ko 2023-03-29T10:41:28.290 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/floppy.ko 2023-03-29T10:41:28.290 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/loop.ko 2023-03-29T10:41:28.290 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/mtip32xx 2023-03-29T10:41:28.290 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-03-29T10:41:28.290 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/nbd.ko 2023-03-29T10:41:28.290 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/null_blk 2023-03-29T10:41:28.290 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/null_blk/null_blk.ko 2023-03-29T10:41:28.290 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/pktcdvd.ko 2023-03-29T10:41:28.290 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/rbd.ko 2023-03-29T10:41:28.290 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/virtio_blk.ko 2023-03-29T10:41:28.290 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/xen-blkback 2023-03-29T10:41:28.290 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-03-29T10:41:28.290 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/xen-blkfront.ko 2023-03-29T10:41:28.290 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/zram 2023-03-29T10:41:28.290 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/block/zram/zram.ko 2023-03-29T10:41:28.290 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char 2023-03-29T10:41:28.290 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hangcheck-timer.ko 2023-03-29T10:41:28.291 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hw_random 2023-03-29T10:41:28.291 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hw_random/amd-rng.ko 2023-03-29T10:41:28.291 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hw_random/intel-rng.ko 2023-03-29T10:41:28.291 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-03-29T10:41:28.291 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hw_random/via-rng.ko 2023-03-29T10:41:28.291 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/hw_random/virtio-rng.ko 2023-03-29T10:41:28.291 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi 2023-03-29T10:41:28.291 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-03-29T10:41:28.291 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-03-29T10:41:28.291 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-03-29T10:41:28.291 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi/ipmi_si.ko 2023-03-29T10:41:28.291 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-03-29T10:41:28.291 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-03-29T10:41:28.291 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/lp.ko 2023-03-29T10:41:28.291 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/mwave 2023-03-29T10:41:28.291 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/mwave/mwave.ko 2023-03-29T10:41:28.291 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/pcmcia 2023-03-29T10:41:28.292 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/pcmcia/cm4000_cs.ko 2023-03-29T10:41:28.292 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/pcmcia/cm4040_cs.ko 2023-03-29T10:41:28.292 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/ppdev.ko 2023-03-29T10:41:28.292 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tlclk.ko 2023-03-29T10:41:28.292 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm 2023-03-29T10:41:28.292 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm/tpm.ko 2023-03-29T10:41:28.292 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm/tpm_atmel.ko 2023-03-29T10:41:28.292 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm/tpm_infineon.ko 2023-03-29T10:41:28.292 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm/tpm_nsc.ko 2023-03-29T10:41:28.292 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm/tpm_tis.ko 2023-03-29T10:41:28.292 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-03-29T10:41:28.292 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/uv_mmtimer.ko 2023-03-29T10:41:28.292 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/char/virtio_console.ko 2023-03-29T10:41:28.292 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq 2023-03-29T10:41:28.292 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-03-29T10:41:28.292 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-03-29T10:41:28.292 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq/p4-clockmod.ko 2023-03-29T10:41:28.293 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-03-29T10:41:28.293 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq/powernow-k8.ko 2023-03-29T10:41:28.293 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/cpufreq/speedstep-lib.ko 2023-03-29T10:41:28.293 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto 2023-03-29T10:41:28.293 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/ccp 2023-03-29T10:41:28.293 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-03-29T10:41:28.293 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/ccp/ccp.ko 2023-03-29T10:41:28.293 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/padlock-aes.ko 2023-03-29T10:41:28.293 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/padlock-sha.ko 2023-03-29T10:41:28.293 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/qat 2023-03-29T10:41:28.293 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/qat/qat_common 2023-03-29T10:41:28.293 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/qat/qat_common/intel_qat.ko 2023-03-29T10:41:28.293 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/qat/qat_dh895xcc 2023-03-29T10:41:28.293 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-03-29T10:41:28.293 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/virtio 2023-03-29T10:41:28.293 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-03-29T10:41:28.294 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dca 2023-03-29T10:41:28.294 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dca/dca.ko 2023-03-29T10:41:28.294 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/devfreq 2023-03-29T10:41:28.294 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-03-29T10:41:28.294 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dma 2023-03-29T10:41:28.294 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dma/dw 2023-03-29T10:41:28.294 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dma/dw/dw_dmac.ko 2023-03-29T10:41:28.294 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dma/ioat 2023-03-29T10:41:28.294 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/dma/ioat/ioatdma.ko 2023-03-29T10:41:28.294 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac 2023-03-29T10:41:28.294 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/amd64_edac.ko 2023-03-29T10:41:28.294 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/e752x_edac.ko 2023-03-29T10:41:28.294 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/edac_mce_amd.ko 2023-03-29T10:41:28.294 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i3000_edac.ko 2023-03-29T10:41:28.294 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i3200_edac.ko 2023-03-29T10:41:28.294 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i5100_edac.ko 2023-03-29T10:41:28.295 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i5400_edac.ko 2023-03-29T10:41:28.295 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i7300_edac.ko 2023-03-29T10:41:28.295 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i7core_edac.ko 2023-03-29T10:41:28.295 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/i82975x_edac.ko 2023-03-29T10:41:28.295 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/ie31200_edac.ko 2023-03-29T10:41:28.295 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/sb_edac.ko 2023-03-29T10:41:28.295 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/edac/x38_edac.ko 2023-03-29T10:41:28.295 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/firmware 2023-03-29T10:41:28.295 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/firmware/edd.ko 2023-03-29T10:41:28.295 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/firmware/iscsi_ibft.ko 2023-03-29T10:41:28.295 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/gpio 2023-03-29T10:41:28.295 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/gpio/gpio-ich.ko 2023-03-29T10:41:28.295 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/gpio/gpio-viperboard.ko 2023-03-29T10:41:28.295 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid 2023-03-29T10:41:28.295 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-appleir.ko 2023-03-29T10:41:28.295 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-aureal.ko 2023-03-29T10:41:28.295 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-axff.ko 2023-03-29T10:41:28.296 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-dr.ko 2023-03-29T10:41:28.296 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-elecom.ko 2023-03-29T10:41:28.296 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-elo.ko 2023-03-29T10:41:28.296 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-emsff.ko 2023-03-29T10:41:28.296 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-gaff.ko 2023-03-29T10:41:28.296 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-gt683r.ko 2023-03-29T10:41:28.296 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-gyration.ko 2023-03-29T10:41:28.296 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-holtek-kbd.ko 2023-03-29T10:41:28.296 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-holtek-mouse.ko 2023-03-29T10:41:28.296 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-holtekff.ko 2023-03-29T10:41:28.296 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-hyperv.ko 2023-03-29T10:41:28.296 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-icade.ko 2023-03-29T10:41:28.296 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-keytouch.ko 2023-03-29T10:41:28.296 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-kye.ko 2023-03-29T10:41:28.296 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-lcpower.ko 2023-03-29T10:41:28.296 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-led.ko 2023-03-29T10:41:28.296 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-lenovo.ko 2023-03-29T10:41:28.296 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-logitech-dj.ko 2023-03-29T10:41:28.297 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-03-29T10:41:28.297 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-multitouch.ko 2023-03-29T10:41:28.297 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-ortek.ko 2023-03-29T10:41:28.297 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-penmount.ko 2023-03-29T10:41:28.297 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-petalynx.ko 2023-03-29T10:41:28.297 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-picolcd.ko 2023-03-29T10:41:28.297 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-pl.ko 2023-03-29T10:41:28.297 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-plantronics.ko 2023-03-29T10:41:28.297 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-primax.ko 2023-03-29T10:41:28.297 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-rmi.ko 2023-03-29T10:41:28.297 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-arvo.ko 2023-03-29T10:41:28.297 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-common.ko 2023-03-29T10:41:28.297 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-isku.ko 2023-03-29T10:41:28.297 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-kone.ko 2023-03-29T10:41:28.297 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-03-29T10:41:28.297 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-konepure.ko 2023-03-29T10:41:28.297 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-03-29T10:41:28.298 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-lua.ko 2023-03-29T10:41:28.298 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-pyra.ko 2023-03-29T10:41:28.298 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-ryos.ko 2023-03-29T10:41:28.298 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat-savu.ko 2023-03-29T10:41:28.298 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-roccat.ko 2023-03-29T10:41:28.298 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-saitek.ko 2023-03-29T10:41:28.298 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-samsung.ko 2023-03-29T10:41:28.298 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-sensor-hub.ko 2023-03-29T10:41:28.298 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-sjoy.ko 2023-03-29T10:41:28.298 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-sony.ko 2023-03-29T10:41:28.298 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-speedlink.ko 2023-03-29T10:41:28.298 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-steelseries.ko 2023-03-29T10:41:28.298 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-sunplus.ko 2023-03-29T10:41:28.298 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-thrustmaster.ko 2023-03-29T10:41:28.298 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-tivo.ko 2023-03-29T10:41:28.298 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-tmff.ko 2023-03-29T10:41:28.298 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-topseed.ko 2023-03-29T10:41:28.299 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-twinhan.ko 2023-03-29T10:41:28.299 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-uclogic.ko 2023-03-29T10:41:28.299 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-waltop.ko 2023-03-29T10:41:28.299 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-wiimote.ko 2023-03-29T10:41:28.299 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-xinmo.ko 2023-03-29T10:41:28.299 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-zpff.ko 2023-03-29T10:41:28.299 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/hid-zydacron.ko 2023-03-29T10:41:28.299 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/uhid.ko 2023-03-29T10:41:28.299 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hid/wacom.ko 2023-03-29T10:41:28.299 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hv 2023-03-29T10:41:28.299 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hv/hv_balloon.ko 2023-03-29T10:41:28.299 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hv/hv_utils.ko 2023-03-29T10:41:28.299 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hv/hv_vmbus.ko 2023-03-29T10:41:28.299 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon 2023-03-29T10:41:28.299 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/abituguru.ko 2023-03-29T10:41:28.299 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/abituguru3.ko 2023-03-29T10:41:28.300 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/acpi_power_meter.ko 2023-03-29T10:41:28.300 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ad7414.ko 2023-03-29T10:41:28.300 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ad7418.ko 2023-03-29T10:41:28.300 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adc128d818.ko 2023-03-29T10:41:28.300 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adm1025.ko 2023-03-29T10:41:28.300 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adm1026.ko 2023-03-29T10:41:28.300 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adm1029.ko 2023-03-29T10:41:28.300 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adm1031.ko 2023-03-29T10:41:28.300 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adm9240.ko 2023-03-29T10:41:28.300 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ads7828.ko 2023-03-29T10:41:28.300 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adt7410.ko 2023-03-29T10:41:28.300 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adt7411.ko 2023-03-29T10:41:28.300 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adt7462.ko 2023-03-29T10:41:28.300 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adt7470.ko 2023-03-29T10:41:28.300 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adt7475.ko 2023-03-29T10:41:28.300 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/adt7x10.ko 2023-03-29T10:41:28.300 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/amc6821.ko 2023-03-29T10:41:28.300 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/applesmc.ko 2023-03-29T10:41:28.301 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/asb100.ko 2023-03-29T10:41:28.301 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/asc7621.ko 2023-03-29T10:41:28.301 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/asus_atk0110.ko 2023-03-29T10:41:28.301 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/atxp1.ko 2023-03-29T10:41:28.301 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/coretemp.ko 2023-03-29T10:41:28.301 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-03-29T10:41:28.301 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/dme1737.ko 2023-03-29T10:41:28.301 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ds1621.ko 2023-03-29T10:41:28.301 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ds620.ko 2023-03-29T10:41:28.301 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/emc1403.ko 2023-03-29T10:41:28.301 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/emc6w201.ko 2023-03-29T10:41:28.301 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/f71805f.ko 2023-03-29T10:41:28.301 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/f71882fg.ko 2023-03-29T10:41:28.302 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/f75375s.ko 2023-03-29T10:41:28.302 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/fam15h_power.ko 2023-03-29T10:41:28.302 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/fschmd.ko 2023-03-29T10:41:28.302 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/g760a.ko 2023-03-29T10:41:28.302 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/g762.ko 2023-03-29T10:41:28.302 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/gl518sm.ko 2023-03-29T10:41:28.302 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/gl520sm.ko 2023-03-29T10:41:28.302 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/hwmon-vid.ko 2023-03-29T10:41:28.302 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/i5500_temp.ko 2023-03-29T10:41:28.302 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/i5k_amb.ko 2023-03-29T10:41:28.302 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ibmaem.ko 2023-03-29T10:41:28.302 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ibmpex.ko 2023-03-29T10:41:28.302 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ina209.ko 2023-03-29T10:41:28.302 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ina2xx.ko 2023-03-29T10:41:28.302 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/it87.ko 2023-03-29T10:41:28.302 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/k10temp.ko 2023-03-29T10:41:28.302 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/k8temp.ko 2023-03-29T10:41:28.303 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lineage-pem.ko 2023-03-29T10:41:28.303 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm63.ko 2023-03-29T10:41:28.303 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm73.ko 2023-03-29T10:41:28.303 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm75.ko 2023-03-29T10:41:28.303 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm77.ko 2023-03-29T10:41:28.303 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm78.ko 2023-03-29T10:41:28.303 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm80.ko 2023-03-29T10:41:28.303 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm83.ko 2023-03-29T10:41:28.303 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm85.ko 2023-03-29T10:41:28.303 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm87.ko 2023-03-29T10:41:28.303 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm90.ko 2023-03-29T10:41:28.303 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm92.ko 2023-03-29T10:41:28.303 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm93.ko 2023-03-29T10:41:28.303 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm95234.ko 2023-03-29T10:41:28.303 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm95241.ko 2023-03-29T10:41:28.303 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/lm95245.ko 2023-03-29T10:41:28.303 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc2945.ko 2023-03-29T10:41:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc4151.ko 2023-03-29T10:41:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc4215.ko 2023-03-29T10:41:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc4222.ko 2023-03-29T10:41:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc4245.ko 2023-03-29T10:41:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc4260.ko 2023-03-29T10:41:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/ltc4261.ko 2023-03-29T10:41:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max16065.ko 2023-03-29T10:41:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max1619.ko 2023-03-29T10:41:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max1668.ko 2023-03-29T10:41:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max197.ko 2023-03-29T10:41:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max6639.ko 2023-03-29T10:41:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max6650.ko 2023-03-29T10:41:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/max6697.ko 2023-03-29T10:41:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/mcp3021.ko 2023-03-29T10:41:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/nct6683.ko 2023-03-29T10:41:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/nct6775-core.ko 2023-03-29T10:41:28.304 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/nct6775.ko 2023-03-29T10:41:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/nct7802.ko 2023-03-29T10:41:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pc87360.ko 2023-03-29T10:41:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pc87427.ko 2023-03-29T10:41:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pcf8591.ko 2023-03-29T10:41:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus 2023-03-29T10:41:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-03-29T10:41:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-03-29T10:41:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-03-29T10:41:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/max16064.ko 2023-03-29T10:41:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/max34440.ko 2023-03-29T10:41:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/max8688.ko 2023-03-29T10:41:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-03-29T10:41:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-03-29T10:41:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-03-29T10:41:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-03-29T10:41:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-03-29T10:41:28.305 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-03-29T10:41:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/powr1220.ko 2023-03-29T10:41:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/sch5627.ko 2023-03-29T10:41:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/sch5636.ko 2023-03-29T10:41:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/sch56xx-common.ko 2023-03-29T10:41:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/sht15.ko 2023-03-29T10:41:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/sht21.ko 2023-03-29T10:41:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/shtc1.ko 2023-03-29T10:41:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/sis5595.ko 2023-03-29T10:41:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/smsc47b397.ko 2023-03-29T10:41:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/smsc47m1.ko 2023-03-29T10:41:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/smsc47m192.ko 2023-03-29T10:41:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/thmc50.ko 2023-03-29T10:41:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/tmp102.ko 2023-03-29T10:41:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/tmp103.ko 2023-03-29T10:41:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/tmp401.ko 2023-03-29T10:41:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/tmp421.ko 2023-03-29T10:41:28.306 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/via-cputemp.ko 2023-03-29T10:41:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/via686a.ko 2023-03-29T10:41:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/vt1211.ko 2023-03-29T10:41:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/vt8231.ko 2023-03-29T10:41:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83627ehf.ko 2023-03-29T10:41:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83627hf.ko 2023-03-29T10:41:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83781d.ko 2023-03-29T10:41:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83791d.ko 2023-03-29T10:41:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83792d.ko 2023-03-29T10:41:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83793.ko 2023-03-29T10:41:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83795.ko 2023-03-29T10:41:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83l785ts.ko 2023-03-29T10:41:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/hwmon/w83l786ng.ko 2023-03-29T10:41:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c 2023-03-29T10:41:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/algos 2023-03-29T10:41:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-03-29T10:41:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-03-29T10:41:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses 2023-03-29T10:41:28.307 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-03-29T10:41:28.308 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-03-29T10:41:28.308 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-03-29T10:41:28.308 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-03-29T10:41:28.308 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-03-29T10:41:28.308 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-03-29T10:41:28.308 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-03-29T10:41:28.308 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-03-29T10:41:28.308 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-i801.ko 2023-03-29T10:41:28.308 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-isch.ko 2023-03-29T10:41:28.308 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-03-29T10:41:28.308 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-03-29T10:41:28.308 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-03-29T10:41:28.308 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-parport.ko 2023-03-29T10:41:28.308 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-03-29T10:41:28.308 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-03-29T10:41:28.308 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-03-29T10:41:28.309 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-03-29T10:41:28.309 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-03-29T10:41:28.309 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-03-29T10:41:28.309 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-via.ko 2023-03-29T10:41:28.309 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-03-29T10:41:28.309 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-03-29T10:41:28.309 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/i2c-dev.ko 2023-03-29T10:41:28.309 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/i2c-mux.ko 2023-03-29T10:41:28.309 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-03-29T10:41:28.309 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/i2c-smbus.ko 2023-03-29T10:41:28.309 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/i2c/i2c-stub.ko 2023-03-29T10:41:28.309 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband 2023-03-29T10:41:28.309 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core 2023-03-29T10:41:28.309 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/ib_cm.ko 2023-03-29T10:41:28.309 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/ib_core.ko 2023-03-29T10:41:28.309 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/ib_umad.ko 2023-03-29T10:41:28.309 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-03-29T10:41:28.309 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/iw_cm.ko 2023-03-29T10:41:28.310 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/rdma_cm.ko 2023-03-29T10:41:28.310 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-03-29T10:41:28.310 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw 2023-03-29T10:41:28.310 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/cxgb4 2023-03-29T10:41:28.310 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-03-29T10:41:28.310 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/mlx4 2023-03-29T10:41:28.310 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-03-29T10:41:28.310 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/mlx5 2023-03-29T10:41:28.310 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-03-29T10:41:28.310 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/mthca 2023-03-29T10:41:28.310 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-03-29T10:41:28.310 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp 2023-03-29T10:41:28.310 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/ipoib 2023-03-29T10:41:28.310 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-03-29T10:41:28.310 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/iser 2023-03-29T10:41:28.310 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-03-29T10:41:28.310 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/isert 2023-03-29T10:41:28.311 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-03-29T10:41:28.311 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/srp 2023-03-29T10:41:28.311 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-03-29T10:41:28.311 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/srpt 2023-03-29T10:41:28.311 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-03-29T10:41:28.311 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input 2023-03-29T10:41:28.311 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/gameport 2023-03-29T10:41:28.311 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-03-29T10:41:28.311 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/gameport/fm801-gp.ko 2023-03-29T10:41:28.311 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/gameport/gameport.ko 2023-03-29T10:41:28.311 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/gameport/lightning.ko 2023-03-29T10:41:28.311 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/gameport/ns558.ko 2023-03-29T10:41:28.311 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/joydev.ko 2023-03-29T10:41:28.311 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/keyboard 2023-03-29T10:41:28.311 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/keyboard/gpio_keys.ko 2023-03-29T10:41:28.311 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse 2023-03-29T10:41:28.311 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/appletouch.ko 2023-03-29T10:41:28.311 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/bcm5974.ko 2023-03-29T10:41:28.312 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/cyapatp.ko 2023-03-29T10:41:28.312 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/elan_i2c.ko 2023-03-29T10:41:28.312 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/sermouse.ko 2023-03-29T10:41:28.312 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-03-29T10:41:28.312 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/synaptics_usb.ko 2023-03-29T10:41:28.312 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/mouse/vsxxxaa.ko 2023-03-29T10:41:28.312 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/rmi4 2023-03-29T10:41:28.312 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/rmi4/rmi_core.ko 2023-03-29T10:41:28.312 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/serio 2023-03-29T10:41:28.312 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/serio/altera_ps2.ko 2023-03-29T10:41:28.312 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/serio/arc_ps2.ko 2023-03-29T10:41:28.312 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-03-29T10:41:28.312 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/serio/serio_raw.ko 2023-03-29T10:41:28.312 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/input/sparse-keymap.ko 2023-03-29T10:41:28.312 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/iommu 2023-03-29T10:41:28.312 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/iommu/amd 2023-03-29T10:41:28.312 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/iommu/amd/iommu_v2.ko 2023-03-29T10:41:28.312 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds 2023-03-29T10:41:28.313 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/leds-blinkm.ko 2023-03-29T10:41:28.313 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/leds-lm3530.ko 2023-03-29T10:41:28.313 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/leds-lp3944.ko 2023-03-29T10:41:28.313 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/leds-ss4200.ko 2023-03-29T10:41:28.313 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger 2023-03-29T10:41:28.313 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-03-29T10:41:28.313 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-03-29T10:41:28.313 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-03-29T10:41:28.313 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-03-29T10:41:28.313 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2023-03-29T10:41:28.313 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-03-29T10:41:28.313 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-03-29T10:41:28.313 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-03-29T10:41:28.313 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-03-29T10:41:28.313 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md 2023-03-29T10:41:28.313 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/bcache 2023-03-29T10:41:28.313 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/bcache/bcache.ko 2023-03-29T10:41:28.314 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-bio-prison.ko 2023-03-29T10:41:28.314 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-cache-smq.ko 2023-03-29T10:41:28.314 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-cache.ko 2023-03-29T10:41:28.314 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-crypt.ko 2023-03-29T10:41:28.314 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-delay.ko 2023-03-29T10:41:28.314 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-flakey.ko 2023-03-29T10:41:28.314 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-log-userspace.ko 2023-03-29T10:41:28.314 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-multipath.ko 2023-03-29T10:41:28.314 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-queue-length.ko 2023-03-29T10:41:28.314 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-raid.ko 2023-03-29T10:41:28.314 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-round-robin.ko 2023-03-29T10:41:28.314 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-service-time.ko 2023-03-29T10:41:28.314 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-switch.ko 2023-03-29T10:41:28.314 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-thin-pool.ko 2023-03-29T10:41:28.314 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/dm-verity.ko 2023-03-29T10:41:28.314 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/faulty.ko 2023-03-29T10:41:28.314 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/linear.ko 2023-03-29T10:41:28.314 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/multipath.ko 2023-03-29T10:41:28.315 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/persistent-data 2023-03-29T10:41:28.315 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-03-29T10:41:28.315 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/raid0.ko 2023-03-29T10:41:28.315 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/raid1.ko 2023-03-29T10:41:28.315 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/raid10.ko 2023-03-29T10:41:28.315 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/md/raid456.ko 2023-03-29T10:41:28.315 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message 2023-03-29T10:41:28.315 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion 2023-03-29T10:41:28.315 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptbase.ko 2023-03-29T10:41:28.315 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptctl.ko 2023-03-29T10:41:28.315 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptfc.ko 2023-03-29T10:41:28.315 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptlan.ko 2023-03-29T10:41:28.315 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptsas.ko 2023-03-29T10:41:28.315 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptscsih.ko 2023-03-29T10:41:28.315 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/message/fusion/mptspi.ko 2023-03-29T10:41:28.315 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd 2023-03-29T10:41:28.315 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/lpc_ich.ko 2023-03-29T10:41:28.315 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/lpc_sch.ko 2023-03-29T10:41:28.316 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/mfd-core.ko 2023-03-29T10:41:28.316 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/sm501.ko 2023-03-29T10:41:28.316 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/viperboard.ko 2023-03-29T10:41:28.316 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/vx855.ko 2023-03-29T10:41:28.316 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/mfd/wl1273-core.ko 2023-03-29T10:41:28.316 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc 2023-03-29T10:41:28.316 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/altera-stapl 2023-03-29T10:41:28.316 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-03-29T10:41:28.316 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/apds9802als.ko 2023-03-29T10:41:28.316 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/apds990x.ko 2023-03-29T10:41:28.316 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/bh1770glc.ko 2023-03-29T10:41:28.316 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/cb710 2023-03-29T10:41:28.316 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/cb710/cb710.ko 2023-03-29T10:41:28.316 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/eeprom 2023-03-29T10:41:28.316 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/eeprom/at24.ko 2023-03-29T10:41:28.316 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/eeprom/eeprom.ko 2023-03-29T10:41:28.316 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-03-29T10:41:28.316 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/eeprom/max6875.ko 2023-03-29T10:41:28.317 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/enclosure.ko 2023-03-29T10:41:28.317 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/hpilo.ko 2023-03-29T10:41:28.317 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/isl29003.ko 2023-03-29T10:41:28.317 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/isl29020.ko 2023-03-29T10:41:28.317 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/lis3lv02d 2023-03-29T10:41:28.317 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-03-29T10:41:28.317 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-03-29T10:41:28.317 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/mei 2023-03-29T10:41:28.317 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/mei/mei-me.ko 2023-03-29T10:41:28.317 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/mei/mei-txe.ko 2023-03-29T10:41:28.317 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/mei/mei.ko 2023-03-29T10:41:28.317 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/sgi-gru 2023-03-29T10:41:28.317 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/sgi-gru/gru.ko 2023-03-29T10:41:28.317 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/sgi-xp 2023-03-29T10:41:28.317 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/sgi-xp/xp.ko 2023-03-29T10:41:28.317 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/sgi-xp/xpc.ko 2023-03-29T10:41:28.317 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-03-29T10:41:28.318 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/tifm_7xx1.ko 2023-03-29T10:41:28.318 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/tifm_core.ko 2023-03-29T10:41:28.318 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/tsl2550.ko 2023-03-29T10:41:28.318 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/vmw_balloon.ko 2023-03-29T10:41:28.318 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/vmw_vmci 2023-03-29T10:41:28.318 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-03-29T10:41:28.318 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net 2023-03-29T10:41:28.318 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/bonding 2023-03-29T10:41:28.318 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/bonding/bonding.ko 2023-03-29T10:41:28.318 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa 2023-03-29T10:41:28.318 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa/b53 2023-03-29T10:41:28.318 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa/b53/b53_common.ko 2023-03-29T10:41:28.318 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa/bcm-sf2.ko 2023-03-29T10:41:28.318 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa/mv88e6060.ko 2023-03-29T10:41:28.318 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa/mv88e6xxx 2023-03-29T10:41:28.318 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-03-29T10:41:28.318 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/dummy.ko 2023-03-29T10:41:28.318 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/eql.ko 2023-03-29T10:41:28.319 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet 2023-03-29T10:41:28.319 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/3com 2023-03-29T10:41:28.319 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-03-29T10:41:28.319 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-03-29T10:41:28.319 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-03-29T10:41:28.319 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-03-29T10:41:28.319 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/8390 2023-03-29T10:41:28.319 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/8390/8390.ko 2023-03-29T10:41:28.319 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-03-29T10:41:28.319 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-03-29T10:41:28.319 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-03-29T10:41:28.319 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/adaptec 2023-03-29T10:41:28.319 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-03-29T10:41:28.319 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/agere 2023-03-29T10:41:28.319 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/agere/et131x.ko 2023-03-29T10:41:28.319 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/alteon 2023-03-29T10:41:28.319 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-03-29T10:41:28.319 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/altera 2023-03-29T10:41:28.320 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-03-29T10:41:28.320 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/amd 2023-03-29T10:41:28.320 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-03-29T10:41:28.320 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-03-29T10:41:28.320 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-03-29T10:41:28.320 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros 2023-03-29T10:41:28.320 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/alx 2023-03-29T10:41:28.320 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-03-29T10:41:28.320 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atl1c 2023-03-29T10:41:28.320 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-03-29T10:41:28.320 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atl1e 2023-03-29T10:41:28.320 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-03-29T10:41:28.320 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atlx 2023-03-29T10:41:28.320 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-03-29T10:41:28.320 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-03-29T10:41:28.320 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom 2023-03-29T10:41:28.320 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-03-29T10:41:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-03-29T10:41:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-03-29T10:41:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-03-29T10:41:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-03-29T10:41:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/genet 2023-03-29T10:41:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-03-29T10:41:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-03-29T10:41:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/brocade 2023-03-29T10:41:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/brocade/bna 2023-03-29T10:41:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-03-29T10:41:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio 2023-03-29T10:41:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb 2023-03-29T10:41:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-03-29T10:41:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-03-29T10:41:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-03-29T10:41:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-03-29T10:41:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-03-29T10:41:28.321 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-03-29T10:41:28.322 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-03-29T10:41:28.322 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-03-29T10:41:28.322 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-03-29T10:41:28.322 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/cisco 2023-03-29T10:41:28.322 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/cisco/enic 2023-03-29T10:41:28.322 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-03-29T10:41:28.322 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec 2023-03-29T10:41:28.322 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip 2023-03-29T10:41:28.322 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-03-29T10:41:28.322 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-03-29T10:41:28.322 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-03-29T10:41:28.322 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-03-29T10:41:28.322 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-03-29T10:41:28.322 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-03-29T10:41:28.322 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dlink 2023-03-29T10:41:28.322 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-03-29T10:41:28.322 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-03-29T10:41:28.322 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/dnet.ko 2023-03-29T10:41:28.322 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/emulex 2023-03-29T10:41:28.323 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/emulex/benet 2023-03-29T10:41:28.323 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-03-29T10:41:28.323 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/ethoc.ko 2023-03-29T10:41:28.323 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/fealnx.ko 2023-03-29T10:41:28.323 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel 2023-03-29T10:41:28.323 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/e100.ko 2023-03-29T10:41:28.323 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/e1000 2023-03-29T10:41:28.323 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-03-29T10:41:28.323 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/e1000e 2023-03-29T10:41:28.323 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-03-29T10:41:28.323 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/fm10k 2023-03-29T10:41:28.323 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-03-29T10:41:28.323 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/i40e 2023-03-29T10:41:28.323 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-03-29T10:41:28.323 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/iavf 2023-03-29T10:41:28.323 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-03-29T10:41:28.323 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/igb 2023-03-29T10:41:28.323 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-03-29T10:41:28.324 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/igbvf 2023-03-29T10:41:28.324 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-03-29T10:41:28.324 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/ixgb 2023-03-29T10:41:28.324 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/ixgb/ixgb.ko 2023-03-29T10:41:28.324 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/ixgbe 2023-03-29T10:41:28.324 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-03-29T10:41:28.324 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/ixgbevf 2023-03-29T10:41:28.324 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-03-29T10:41:28.324 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/jme.ko 2023-03-29T10:41:28.324 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/marvell 2023-03-29T10:41:28.324 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-03-29T10:41:28.324 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/marvell/skge.ko 2023-03-29T10:41:28.324 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-03-29T10:41:28.324 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox 2023-03-29T10:41:28.324 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox/mlx4 2023-03-29T10:41:28.324 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-03-29T10:41:28.324 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-03-29T10:41:28.325 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox/mlx5 2023-03-29T10:41:28.325 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-03-29T10:41:28.325 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-03-29T10:41:28.325 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/micrel 2023-03-29T10:41:28.325 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-03-29T10:41:28.325 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/myricom 2023-03-29T10:41:28.325 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/myricom/myri10ge 2023-03-29T10:41:28.325 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-03-29T10:41:28.325 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/natsemi 2023-03-29T10:41:28.325 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-03-29T10:41:28.325 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-03-29T10:41:28.325 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/neterion 2023-03-29T10:41:28.325 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-03-29T10:41:28.325 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/nvidia 2023-03-29T10:41:28.325 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-03-29T10:41:28.325 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/packetengines 2023-03-29T10:41:28.325 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-03-29T10:41:28.326 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-03-29T10:41:28.326 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/qlogic 2023-03-29T10:41:28.326 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/qlogic/netxen 2023-03-29T10:41:28.326 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-03-29T10:41:28.326 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-03-29T10:41:28.326 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-03-29T10:41:28.326 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-03-29T10:41:28.326 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/rdc 2023-03-29T10:41:28.326 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-03-29T10:41:28.326 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/realtek 2023-03-29T10:41:28.326 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-03-29T10:41:28.326 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-03-29T10:41:28.326 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/realtek/atp.ko 2023-03-29T10:41:28.326 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-03-29T10:41:28.326 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/rocker 2023-03-29T10:41:28.326 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-03-29T10:41:28.326 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sfc 2023-03-29T10:41:28.326 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-03-29T10:41:28.327 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/silan 2023-03-29T10:41:28.327 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-03-29T10:41:28.327 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sis 2023-03-29T10:41:28.327 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sis/sis190.ko 2023-03-29T10:41:28.327 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sis/sis900.ko 2023-03-29T10:41:28.327 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/smsc 2023-03-29T10:41:28.327 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-03-29T10:41:28.327 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-03-29T10:41:28.327 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-03-29T10:41:28.327 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-03-29T10:41:28.327 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/stmicro 2023-03-29T10:41:28.327 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/stmicro/stmmac 2023-03-29T10:41:28.327 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-03-29T10:41:28.327 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-03-29T10:41:28.327 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sun 2023-03-29T10:41:28.327 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sun/cassini.ko 2023-03-29T10:41:28.327 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sun/niu.ko 2023-03-29T10:41:28.327 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sun/sungem.ko 2023-03-29T10:41:28.328 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-03-29T10:41:28.328 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/tehuti 2023-03-29T10:41:28.328 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-03-29T10:41:28.328 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/ti 2023-03-29T10:41:28.328 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/ti/tlan.ko 2023-03-29T10:41:28.328 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/via 2023-03-29T10:41:28.328 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-03-29T10:41:28.328 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-03-29T10:41:28.328 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/wiznet 2023-03-29T10:41:28.328 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-03-29T10:41:28.328 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-03-29T10:41:28.328 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/xircom 2023-03-29T10:41:28.328 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-03-29T10:41:28.328 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/geneve.ko 2023-03-29T10:41:28.328 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/hyperv 2023-03-29T10:41:28.328 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-03-29T10:41:28.328 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ieee802154 2023-03-29T10:41:28.329 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ieee802154/fakelb.ko 2023-03-29T10:41:28.329 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ifb.ko 2023-03-29T10:41:28.329 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ipvlan 2023-03-29T10:41:28.329 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ipvlan/ipvlan.ko 2023-03-29T10:41:28.329 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/macvlan.ko 2023-03-29T10:41:28.329 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/macvtap.ko 2023-03-29T10:41:28.329 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/mdio 2023-03-29T10:41:28.329 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/mdio.ko 2023-03-29T10:41:28.329 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-03-29T10:41:28.329 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-03-29T10:41:28.329 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/mii.ko 2023-03-29T10:41:28.329 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/net_failover.ko 2023-03-29T10:41:28.329 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/netconsole.ko 2023-03-29T10:41:28.329 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/nlmon.ko 2023-03-29T10:41:28.329 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/pcs 2023-03-29T10:41:28.329 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/pcs/pcs-altera-tse.ko 2023-03-29T10:41:28.329 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-03-29T10:41:28.330 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy 2023-03-29T10:41:28.330 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/amd.ko 2023-03-29T10:41:28.330 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-03-29T10:41:28.330 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-03-29T10:41:28.330 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/bcm7xxx.ko 2023-03-29T10:41:28.330 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/bcm87xx.ko 2023-03-29T10:41:28.330 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/broadcom.ko 2023-03-29T10:41:28.330 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/cicada.ko 2023-03-29T10:41:28.330 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/davicom.ko 2023-03-29T10:41:28.330 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/dp83640.ko 2023-03-29T10:41:28.330 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/et1011c.ko 2023-03-29T10:41:28.330 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/icplus.ko 2023-03-29T10:41:28.330 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/lxt.ko 2023-03-29T10:41:28.330 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/marvell.ko 2023-03-29T10:41:28.330 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/micrel.ko 2023-03-29T10:41:28.330 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/national.ko 2023-03-29T10:41:28.330 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/phylink.ko 2023-03-29T10:41:28.330 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/qsemi.ko 2023-03-29T10:41:28.331 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/realtek.ko 2023-03-29T10:41:28.331 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/smsc.ko 2023-03-29T10:41:28.331 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/ste10Xp.ko 2023-03-29T10:41:28.331 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/phy/vitesse.ko 2023-03-29T10:41:28.331 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp 2023-03-29T10:41:28.331 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/bsd_comp.ko 2023-03-29T10:41:28.331 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/ppp_async.ko 2023-03-29T10:41:28.331 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/ppp_deflate.ko 2023-03-29T10:41:28.331 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/ppp_generic.ko 2023-03-29T10:41:28.331 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/ppp_mppe.ko 2023-03-29T10:41:28.331 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/ppp_synctty.ko 2023-03-29T10:41:28.331 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/pppoe.ko 2023-03-29T10:41:28.331 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/pppox.ko 2023-03-29T10:41:28.331 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/ppp/pptp.ko 2023-03-29T10:41:28.331 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/slip 2023-03-29T10:41:28.331 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/slip/slhc.ko 2023-03-29T10:41:28.332 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/slip/slip.ko 2023-03-29T10:41:28.332 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/sungem_phy.ko 2023-03-29T10:41:28.332 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/tap.ko 2023-03-29T10:41:28.332 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team 2023-03-29T10:41:28.332 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team/team.ko 2023-03-29T10:41:28.332 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team/team_mode_activebackup.ko 2023-03-29T10:41:28.332 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team/team_mode_broadcast.ko 2023-03-29T10:41:28.332 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-03-29T10:41:28.332 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team/team_mode_random.ko 2023-03-29T10:41:28.332 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-03-29T10:41:28.332 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/tun.ko 2023-03-29T10:41:28.332 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/veth.ko 2023-03-29T10:41:28.332 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/virtio_net.ko 2023-03-29T10:41:28.332 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/vmxnet3 2023-03-29T10:41:28.332 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-03-29T10:41:28.332 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/vxlan 2023-03-29T10:41:28.332 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/vxlan/vxlan.ko 2023-03-29T10:41:28.332 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/xen-netback 2023-03-29T10:41:28.333 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/xen-netback/xen-netback.ko 2023-03-29T10:41:28.333 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/net/xen-netfront.ko 2023-03-29T10:41:28.333 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ntb 2023-03-29T10:41:28.333 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ntb/ntb.ko 2023-03-29T10:41:28.333 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/nvme 2023-03-29T10:41:28.333 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/nvme/host 2023-03-29T10:41:28.333 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/nvme/host/nvme-core.ko 2023-03-29T10:41:28.333 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/nvme/host/nvme.ko 2023-03-29T10:41:28.333 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/parport 2023-03-29T10:41:28.333 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/parport/parport.ko 2023-03-29T10:41:28.333 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/parport/parport_cs.ko 2023-03-29T10:41:28.333 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/parport/parport_pc.ko 2023-03-29T10:41:28.333 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/parport/parport_serial.ko 2023-03-29T10:41:28.333 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pci 2023-03-29T10:41:28.333 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pci/hotplug 2023-03-29T10:41:28.333 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-03-29T10:41:28.334 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pci/pcie 2023-03-29T10:41:28.334 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pci/pcie/aer_inject.ko 2023-03-29T10:41:28.334 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pci/xen-pcifront.ko 2023-03-29T10:41:28.334 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pcmcia 2023-03-29T10:41:28.334 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pcmcia/i82092.ko 2023-03-29T10:41:28.334 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pcmcia/pd6729.ko 2023-03-29T10:41:28.334 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pcmcia/yenta_socket.ko 2023-03-29T10:41:28.334 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pinctrl 2023-03-29T10:41:28.334 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pinctrl/intel 2023-03-29T10:41:28.334 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-03-29T10:41:28.334 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform 2023-03-29T10:41:28.334 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86 2023-03-29T10:41:28.334 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/acer-wmi.ko 2023-03-29T10:41:28.334 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/acerhdf.ko 2023-03-29T10:41:28.334 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/apple-gmux.ko 2023-03-29T10:41:28.334 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/asus-laptop.ko 2023-03-29T10:41:28.334 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-03-29T10:41:28.334 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/asus-wmi.ko 2023-03-29T10:41:28.335 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/classmate-laptop.ko 2023-03-29T10:41:28.335 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-03-29T10:41:28.335 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-03-29T10:41:28.335 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-03-29T10:41:28.335 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-03-29T10:41:28.335 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/hdaps.ko 2023-03-29T10:41:28.335 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/intel 2023-03-29T10:41:28.335 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-03-29T10:41:28.335 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/intel_ips.ko 2023-03-29T10:41:28.335 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/msi-wmi.ko 2023-03-29T10:41:28.335 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/mxm-wmi.ko 2023-03-29T10:41:28.335 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-03-29T10:41:28.335 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/samsung-laptop.ko 2023-03-29T10:41:28.335 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/samsung-q10.ko 2023-03-29T10:41:28.335 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/topstar-laptop.ko 2023-03-29T10:41:28.335 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-03-29T10:41:28.335 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/toshiba_haps.ko 2023-03-29T10:41:28.335 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/wmi-bmof.ko 2023-03-29T10:41:28.336 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/platform/x86/wmi.ko 2023-03-29T10:41:28.336 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/powercap 2023-03-29T10:41:28.336 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/powercap/intel_rapl_common.ko 2023-03-29T10:41:28.336 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/powercap/intel_rapl_msr.ko 2023-03-29T10:41:28.336 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pps 2023-03-29T10:41:28.336 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pps/clients 2023-03-29T10:41:28.336 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pps/clients/pps-gpio.ko 2023-03-29T10:41:28.336 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pps/clients/pps-ldisc.ko 2023-03-29T10:41:28.336 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pps/clients/pps_parport.ko 2023-03-29T10:41:28.336 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/pps/pps_core.ko 2023-03-29T10:41:28.336 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ptp 2023-03-29T10:41:28.336 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ptp/ptp.ko 2023-03-29T10:41:28.336 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ptp/ptp_kvm.ko 2023-03-29T10:41:28.336 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc 2023-03-29T10:41:28.336 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-bq32k.ko 2023-03-29T10:41:28.336 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-bq4802.ko 2023-03-29T10:41:28.336 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1286.ko 2023-03-29T10:41:28.337 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1307.ko 2023-03-29T10:41:28.337 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1374.ko 2023-03-29T10:41:28.337 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1511.ko 2023-03-29T10:41:28.337 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1553.ko 2023-03-29T10:41:28.337 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1672.ko 2023-03-29T10:41:28.337 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds1742.ko 2023-03-29T10:41:28.337 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds2404.ko 2023-03-29T10:41:28.337 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-ds3232.ko 2023-03-29T10:41:28.337 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-em3027.ko 2023-03-29T10:41:28.337 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-fm3130.ko 2023-03-29T10:41:28.337 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-isl12022.ko 2023-03-29T10:41:28.337 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-isl1208.ko 2023-03-29T10:41:28.337 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-m41t80.ko 2023-03-29T10:41:28.337 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-m48t35.ko 2023-03-29T10:41:28.337 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-m48t59.ko 2023-03-29T10:41:28.337 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-max6900.ko 2023-03-29T10:41:28.337 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-msm6242.ko 2023-03-29T10:41:28.337 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-pcf2127.ko 2023-03-29T10:41:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-pcf85063.ko 2023-03-29T10:41:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-pcf8523.ko 2023-03-29T10:41:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-pcf8563.ko 2023-03-29T10:41:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-pcf8583.ko 2023-03-29T10:41:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-rp5c01.ko 2023-03-29T10:41:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-rs5c372.ko 2023-03-29T10:41:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-03-29T10:41:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-rx8025.ko 2023-03-29T10:41:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-rx8581.ko 2023-03-29T10:41:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-03-29T10:41:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/rtc/rtc-x1205.ko 2023-03-29T10:41:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi 2023-03-29T10:41:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/3w-9xxx.ko 2023-03-29T10:41:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/3w-sas.ko 2023-03-29T10:41:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/3w-xxxx.ko 2023-03-29T10:41:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/BusLogic.ko 2023-03-29T10:41:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/a100u2w.ko 2023-03-29T10:41:28.338 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/aacraid 2023-03-29T10:41:28.339 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/aacraid/aacraid.ko 2023-03-29T10:41:28.339 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/advansys.ko 2023-03-29T10:41:28.339 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/aic7xxx 2023-03-29T10:41:28.339 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-03-29T10:41:28.339 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-03-29T10:41:28.339 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/am53c974.ko 2023-03-29T10:41:28.339 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/arcmsr 2023-03-29T10:41:28.339 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-03-29T10:41:28.339 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/atp870u.ko 2023-03-29T10:41:28.339 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/be2iscsi 2023-03-29T10:41:28.339 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-03-29T10:41:28.339 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/bfa 2023-03-29T10:41:28.339 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/bfa/bfa.ko 2023-03-29T10:41:28.339 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/bnx2fc 2023-03-29T10:41:28.339 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-03-29T10:41:28.339 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/bnx2i 2023-03-29T10:41:28.339 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-03-29T10:41:28.340 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/ch.ko 2023-03-29T10:41:28.340 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/csiostor 2023-03-29T10:41:28.340 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/csiostor/csiostor.ko 2023-03-29T10:41:28.340 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/cxgbi 2023-03-29T10:41:28.340 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/cxgbi/cxgb3i 2023-03-29T10:41:28.340 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-03-29T10:41:28.340 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/cxgbi/cxgb4i 2023-03-29T10:41:28.340 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-03-29T10:41:28.340 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-03-29T10:41:28.340 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/dc395x.ko 2023-03-29T10:41:28.340 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/device_handler 2023-03-29T10:41:28.340 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-03-29T10:41:28.340 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-03-29T10:41:28.340 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-03-29T10:41:28.340 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-03-29T10:41:28.340 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/dmx3191d.ko 2023-03-29T10:41:28.340 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/esas2r 2023-03-29T10:41:28.340 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/esas2r/esas2r.ko 2023-03-29T10:41:28.341 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/esp_scsi.ko 2023-03-29T10:41:28.341 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/fcoe 2023-03-29T10:41:28.341 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/fcoe/fcoe.ko 2023-03-29T10:41:28.341 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-03-29T10:41:28.341 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/fnic 2023-03-29T10:41:28.341 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/fnic/fnic.ko 2023-03-29T10:41:28.341 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/hpsa.ko 2023-03-29T10:41:28.341 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/hptiop.ko 2023-03-29T10:41:28.341 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/hv_storvsc.ko 2023-03-29T10:41:28.341 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/initio.ko 2023-03-29T10:41:28.341 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/ipr.ko 2023-03-29T10:41:28.341 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/ips.ko 2023-03-29T10:41:28.341 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/isci 2023-03-29T10:41:28.341 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/isci/isci.ko 2023-03-29T10:41:28.341 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-03-29T10:41:28.341 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/iscsi_tcp.ko 2023-03-29T10:41:28.341 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/libfc 2023-03-29T10:41:28.342 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/libfc/libfc.ko 2023-03-29T10:41:28.342 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/libiscsi.ko 2023-03-29T10:41:28.342 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/libiscsi_tcp.ko 2023-03-29T10:41:28.342 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/libsas 2023-03-29T10:41:28.342 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/libsas/libsas.ko 2023-03-29T10:41:28.342 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/lpfc 2023-03-29T10:41:28.342 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/lpfc/lpfc.ko 2023-03-29T10:41:28.342 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/megaraid 2023-03-29T10:41:28.342 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/megaraid.ko 2023-03-29T10:41:28.342 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-03-29T10:41:28.342 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-03-29T10:41:28.342 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-03-29T10:41:28.342 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/mpt3sas 2023-03-29T10:41:28.342 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-03-29T10:41:28.342 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/mvsas 2023-03-29T10:41:28.342 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/mvsas/mvsas.ko 2023-03-29T10:41:28.342 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/mvumi.ko 2023-03-29T10:41:28.342 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/pm8001 2023-03-29T10:41:28.343 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-03-29T10:41:28.343 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/pmcraid.ko 2023-03-29T10:41:28.343 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/qla1280.ko 2023-03-29T10:41:28.343 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/qla2xxx 2023-03-29T10:41:28.343 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-03-29T10:41:28.343 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-03-29T10:41:28.343 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/qla4xxx 2023-03-29T10:41:28.343 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-03-29T10:41:28.343 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/raid_class.ko 2023-03-29T10:41:28.343 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/scsi_debug.ko 2023-03-29T10:41:28.343 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/scsi_transport_fc.ko 2023-03-29T10:41:28.343 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-03-29T10:41:28.343 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/scsi_transport_sas.ko 2023-03-29T10:41:28.343 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/scsi_transport_spi.ko 2023-03-29T10:41:28.343 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/scsi_transport_srp.ko 2023-03-29T10:41:28.343 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/ses.ko 2023-03-29T10:41:28.343 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/st.ko 2023-03-29T10:41:28.343 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/stex.ko 2023-03-29T10:41:28.344 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/sym53c8xx_2 2023-03-29T10:41:28.344 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-03-29T10:41:28.344 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/virtio_scsi.ko 2023-03-29T10:41:28.344 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/vmw_pvscsi.ko 2023-03-29T10:41:28.344 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/scsi/wd719x.ko 2023-03-29T10:41:28.344 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ssb 2023-03-29T10:41:28.344 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ssb/ssb.ko 2023-03-29T10:41:28.344 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target 2023-03-29T10:41:28.344 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/iscsi 2023-03-29T10:41:28.344 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-03-29T10:41:28.344 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/loopback 2023-03-29T10:41:28.344 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/loopback/tcm_loop.ko 2023-03-29T10:41:28.344 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/target_core_file.ko 2023-03-29T10:41:28.344 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/target_core_iblock.ko 2023-03-29T10:41:28.344 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/target_core_mod.ko 2023-03-29T10:41:28.344 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/target_core_pscsi.ko 2023-03-29T10:41:28.344 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/target_core_user.ko 2023-03-29T10:41:28.344 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/tcm_fc 2023-03-29T10:41:28.345 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-03-29T10:41:28.345 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal 2023-03-29T10:41:28.345 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel 2023-03-29T10:41:28.345 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal 2023-03-29T10:41:28.345 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-03-29T10:41:28.345 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-03-29T10:41:28.345 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-03-29T10:41:28.345 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-03-29T10:41:28.345 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-03-29T10:41:28.345 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-03-29T10:41:28.345 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-03-29T10:41:28.345 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-03-29T10:41:28.345 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-03-29T10:41:28.345 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-03-29T10:41:28.345 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-03-29T10:41:28.345 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-03-29T10:41:28.345 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-03-29T10:41:28.346 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-03-29T10:41:28.346 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-03-29T10:41:28.346 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty 2023-03-29T10:41:28.346 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/ipwireless 2023-03-29T10:41:28.346 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-03-29T10:41:28.346 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/n_gsm.ko 2023-03-29T10:41:28.346 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/n_hdlc.ko 2023-03-29T10:41:28.346 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/nozomi.ko 2023-03-29T10:41:28.346 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/serial 2023-03-29T10:41:28.346 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/serial/8250 2023-03-29T10:41:28.346 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-03-29T10:41:28.346 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/serial/arc_uart.ko 2023-03-29T10:41:28.346 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/serial/jsm 2023-03-29T10:41:28.346 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/serial/jsm/jsm.ko 2023-03-29T10:41:28.346 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/tty/synclink_gt.ko 2023-03-29T10:41:28.346 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ufs 2023-03-29T10:41:28.346 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ufs/core 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ufs/core/ufshcd-core.ko 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ufs/host 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/uio 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/uio/uio.ko 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/uio/uio_aec.ko 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/uio/uio_cif.ko 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/uio/uio_pci_generic.ko 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/uio/uio_sercos3.ko 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/class 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/class/cdc-acm.ko 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/class/cdc-wdm.ko 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/class/usblp.ko 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/class/usbtmc.ko 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/host 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/host/sl811-hcd.ko 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/host/u132-hcd.ko 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/image 2023-03-29T10:41:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/image/mdc800.ko 2023-03-29T10:41:28.348 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/image/microtek.ko 2023-03-29T10:41:28.348 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc 2023-03-29T10:41:28.348 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/adutux.ko 2023-03-29T10:41:28.348 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/appledisplay.ko 2023-03-29T10:41:28.348 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/emi26.ko 2023-03-29T10:41:28.348 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/emi62.ko 2023-03-29T10:41:28.348 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/ezusb.ko 2023-03-29T10:41:28.348 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/ftdi-elan.ko 2023-03-29T10:41:28.348 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/idmouse.ko 2023-03-29T10:41:28.348 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/iowarrior.ko 2023-03-29T10:41:28.348 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/isight_firmware.ko 2023-03-29T10:41:28.348 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/ldusb.ko 2023-03-29T10:41:28.348 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/legousbtower.ko 2023-03-29T10:41:28.348 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/sisusbvga 2023-03-29T10:41:28.348 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-03-29T10:41:28.348 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/trancevibrator.ko 2023-03-29T10:41:28.348 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/usb3503.ko 2023-03-29T10:41:28.349 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/usblcd.ko 2023-03-29T10:41:28.349 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/usbsevseg.ko 2023-03-29T10:41:28.349 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/uss720.ko 2023-03-29T10:41:28.349 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/misc/yurex.ko 2023-03-29T10:41:28.349 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/phy 2023-03-29T10:41:28.349 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/phy/phy-generic.ko 2023-03-29T10:41:28.349 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial 2023-03-29T10:41:28.349 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/aircable.ko 2023-03-29T10:41:28.349 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ark3116.ko 2023-03-29T10:41:28.349 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/belkin_sa.ko 2023-03-29T10:41:28.349 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ch341.ko 2023-03-29T10:41:28.349 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/cp210x.ko 2023-03-29T10:41:28.349 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/cyberjack.ko 2023-03-29T10:41:28.349 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/cypress_m8.ko 2023-03-29T10:41:28.349 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/digi_acceleport.ko 2023-03-29T10:41:28.349 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/empeg.ko 2023-03-29T10:41:28.349 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ftdi_sio.ko 2023-03-29T10:41:28.349 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/garmin_gps.ko 2023-03-29T10:41:28.350 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/io_edgeport.ko 2023-03-29T10:41:28.350 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/io_ti.ko 2023-03-29T10:41:28.350 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ipaq.ko 2023-03-29T10:41:28.350 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ipw.ko 2023-03-29T10:41:28.350 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ir-usb.ko 2023-03-29T10:41:28.350 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-03-29T10:41:28.350 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/keyspan.ko 2023-03-29T10:41:28.350 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/keyspan_pda.ko 2023-03-29T10:41:28.350 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/kl5kusb105.ko 2023-03-29T10:41:28.350 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/kobil_sct.ko 2023-03-29T10:41:28.350 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/mct_u232.ko 2023-03-29T10:41:28.350 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/mos7720.ko 2023-03-29T10:41:28.350 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/mos7840.ko 2023-03-29T10:41:28.350 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/navman.ko 2023-03-29T10:41:28.350 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/omninet.ko 2023-03-29T10:41:28.350 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/opticon.ko 2023-03-29T10:41:28.350 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/option.ko 2023-03-29T10:41:28.350 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/oti6858.ko 2023-03-29T10:41:28.351 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/pl2303.ko 2023-03-29T10:41:28.351 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/qcaux.ko 2023-03-29T10:41:28.351 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/qcserial.ko 2023-03-29T10:41:28.351 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/quatech2.ko 2023-03-29T10:41:28.351 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/safe_serial.ko 2023-03-29T10:41:28.351 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/sierra.ko 2023-03-29T10:41:28.351 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/spcp8x5.ko 2023-03-29T10:41:28.351 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ssu100.ko 2023-03-29T10:41:28.351 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/symbolserial.ko 2023-03-29T10:41:28.351 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-03-29T10:41:28.351 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-03-29T10:41:28.351 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/usb_debug.ko 2023-03-29T10:41:28.351 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/usb_wwan.ko 2023-03-29T10:41:28.351 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/visor.ko 2023-03-29T10:41:28.351 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/whiteheat.ko 2023-03-29T10:41:28.351 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/serial/xsens_mt.ko 2023-03-29T10:41:28.351 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage 2023-03-29T10:41:28.351 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/uas.ko 2023-03-29T10:41:28.352 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-alauda.ko 2023-03-29T10:41:28.352 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-cypress.ko 2023-03-29T10:41:28.352 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-datafab.ko 2023-03-29T10:41:28.352 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-03-29T10:41:28.352 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-freecom.ko 2023-03-29T10:41:28.352 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-isd200.ko 2023-03-29T10:41:28.352 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-03-29T10:41:28.352 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-karma.ko 2023-03-29T10:41:28.352 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-onetouch.ko 2023-03-29T10:41:28.352 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-realtek.ko 2023-03-29T10:41:28.352 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-sddr09.ko 2023-03-29T10:41:28.352 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-sddr55.ko 2023-03-29T10:41:28.352 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/ums-usbat.ko 2023-03-29T10:41:28.352 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/storage/usb-storage.ko 2023-03-29T10:41:28.352 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/usbip 2023-03-29T10:41:28.353 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/usbip/usbip-core.ko 2023-03-29T10:41:28.353 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/usbip/usbip-host.ko 2023-03-29T10:41:28.353 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-03-29T10:41:28.353 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vfio 2023-03-29T10:41:28.353 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vfio/pci 2023-03-29T10:41:28.353 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-03-29T10:41:28.353 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vfio/pci/vfio-pci.ko 2023-03-29T10:41:28.353 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vfio/vfio.ko 2023-03-29T10:41:28.353 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-03-29T10:41:28.353 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vhost 2023-03-29T10:41:28.353 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vhost/vhost.ko 2023-03-29T10:41:28.353 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vhost/vhost_iotlb.ko 2023-03-29T10:41:28.353 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vhost/vhost_net.ko 2023-03-29T10:41:28.353 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/vhost/vhost_scsi.ko 2023-03-29T10:41:28.353 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/video 2023-03-29T10:41:28.353 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/video/backlight 2023-03-29T10:41:28.353 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/video/backlight/apple_bl.ko 2023-03-29T10:41:28.354 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/video/backlight/lcd.ko 2023-03-29T10:41:28.354 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/video/backlight/platform_lcd.ko 2023-03-29T10:41:28.354 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio 2023-03-29T10:41:28.354 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio.ko 2023-03-29T10:41:28.354 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio_balloon.ko 2023-03-29T10:41:28.354 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio_mmio.ko 2023-03-29T10:41:28.354 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio_pci.ko 2023-03-29T10:41:28.354 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-03-29T10:41:28.354 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-03-29T10:41:28.354 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/virtio/virtio_ring.ko 2023-03-29T10:41:28.354 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1 2023-03-29T10:41:28.354 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/masters 2023-03-29T10:41:28.354 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/masters/ds2482.ko 2023-03-29T10:41:28.354 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/masters/ds2490.ko 2023-03-29T10:41:28.354 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves 2023-03-29T10:41:28.354 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-03-29T10:41:28.355 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-03-29T10:41:28.355 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-03-29T10:41:28.355 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-03-29T10:41:28.355 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-03-29T10:41:28.355 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-03-29T10:41:28.355 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-03-29T10:41:28.356 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-03-29T10:41:28.356 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-03-29T10:41:28.356 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_smem.ko 2023-03-29T10:41:28.356 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/slaves/w1_therm.ko 2023-03-29T10:41:28.356 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/w1/wire.ko 2023-03-29T10:41:28.356 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog 2023-03-29T10:41:28.356 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/alim1535_wdt.ko 2023-03-29T10:41:28.356 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/alim7101_wdt.ko 2023-03-29T10:41:28.356 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/f71808e_wdt.ko 2023-03-29T10:41:28.356 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/hpwdt.ko 2023-03-29T10:41:28.356 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/i6300esb.ko 2023-03-29T10:41:28.356 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-03-29T10:41:28.356 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/iTCO_wdt.ko 2023-03-29T10:41:28.356 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/ib700wdt.ko 2023-03-29T10:41:28.357 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/ibmasr.ko 2023-03-29T10:41:28.357 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-03-29T10:41:28.357 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/it8712f_wdt.ko 2023-03-29T10:41:28.357 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/it87_wdt.ko 2023-03-29T10:41:28.357 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/machzwd.ko 2023-03-29T10:41:28.357 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/nv_tco.ko 2023-03-29T10:41:28.357 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/pcwd_pci.ko 2023-03-29T10:41:28.357 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/pcwd_usb.ko 2023-03-29T10:41:28.357 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-03-29T10:41:28.357 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/sch311x_wdt.ko 2023-03-29T10:41:28.357 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/softdog.ko 2023-03-29T10:41:28.357 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/sp5100_tco.ko 2023-03-29T10:41:28.357 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/via_wdt.ko 2023-03-29T10:41:28.357 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-03-29T10:41:28.357 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/w83877f_wdt.ko 2023-03-29T10:41:28.357 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/w83977f_wdt.ko 2023-03-29T10:41:28.357 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/wdt_pci.ko 2023-03-29T10:41:28.358 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/watchdog/xen_wdt.ko 2023-03-29T10:41:28.358 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen 2023-03-29T10:41:28.358 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-acpi-processor.ko 2023-03-29T10:41:28.358 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-evtchn.ko 2023-03-29T10:41:28.358 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-gntalloc.ko 2023-03-29T10:41:28.358 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-gntdev.ko 2023-03-29T10:41:28.358 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-pciback 2023-03-29T10:41:28.358 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-03-29T10:41:28.358 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xen-privcmd.ko 2023-03-29T10:41:28.358 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xenfs 2023-03-29T10:41:28.358 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/drivers/xen/xenfs/xenfs.ko 2023-03-29T10:41:28.358 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs 2023-03-29T10:41:28.358 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/9p 2023-03-29T10:41:28.358 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/9p/9p.ko 2023-03-29T10:41:28.358 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/affs 2023-03-29T10:41:28.358 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/affs/affs.ko 2023-03-29T10:41:28.358 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/befs 2023-03-29T10:41:28.358 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/befs/befs.ko 2023-03-29T10:41:28.359 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/binfmt_misc.ko 2023-03-29T10:41:28.359 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/btrfs 2023-03-29T10:41:28.359 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/btrfs/btrfs.ko 2023-03-29T10:41:28.359 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/cachefiles 2023-03-29T10:41:28.359 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/cachefiles/cachefiles.ko 2023-03-29T10:41:28.359 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ceph 2023-03-29T10:41:28.359 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ceph/ceph.ko 2023-03-29T10:41:28.359 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/cifs 2023-03-29T10:41:28.359 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/cifs/cifs.ko 2023-03-29T10:41:28.359 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/coda 2023-03-29T10:41:28.359 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/coda/coda.ko 2023-03-29T10:41:28.359 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/cramfs 2023-03-29T10:41:28.359 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/cramfs/cramfs.ko 2023-03-29T10:41:28.359 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/dlm 2023-03-29T10:41:28.359 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/dlm/dlm.ko 2023-03-29T10:41:28.359 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ecryptfs 2023-03-29T10:41:28.359 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ecryptfs/ecryptfs.ko 2023-03-29T10:41:28.360 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/f2fs 2023-03-29T10:41:28.360 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/f2fs/f2fs.ko 2023-03-29T10:41:28.360 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fat 2023-03-29T10:41:28.360 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fat/fat.ko 2023-03-29T10:41:28.360 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fat/msdos.ko 2023-03-29T10:41:28.360 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fat/vfat.ko 2023-03-29T10:41:28.360 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fscache 2023-03-29T10:41:28.360 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fscache/fscache.ko 2023-03-29T10:41:28.360 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fuse 2023-03-29T10:41:28.360 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fuse/cuse.ko 2023-03-29T10:41:28.360 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/fuse/fuse.ko 2023-03-29T10:41:28.360 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/gfs2 2023-03-29T10:41:28.360 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/gfs2/gfs2.ko 2023-03-29T10:41:28.360 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/hfs 2023-03-29T10:41:28.360 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/hfs/hfs.ko 2023-03-29T10:41:28.360 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/hfsplus 2023-03-29T10:41:28.360 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/hfsplus/hfsplus.ko 2023-03-29T10:41:28.360 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/isofs 2023-03-29T10:41:28.361 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/isofs/isofs.ko 2023-03-29T10:41:28.361 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/jfs 2023-03-29T10:41:28.361 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/jfs/jfs.ko 2023-03-29T10:41:28.361 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/lockd 2023-03-29T10:41:28.361 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/lockd/lockd.ko 2023-03-29T10:41:28.361 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/minix 2023-03-29T10:41:28.361 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/minix/minix.ko 2023-03-29T10:41:28.361 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/netfs 2023-03-29T10:41:28.361 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/netfs/netfs.ko 2023-03-29T10:41:28.361 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs 2023-03-29T10:41:28.361 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/blocklayout 2023-03-29T10:41:28.361 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-03-29T10:41:28.361 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/filelayout 2023-03-29T10:41:28.361 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-03-29T10:41:28.361 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/flexfilelayout 2023-03-29T10:41:28.361 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-03-29T10:41:28.361 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/nfs.ko 2023-03-29T10:41:28.362 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/nfsv3.ko 2023-03-29T10:41:28.362 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs/nfsv4.ko 2023-03-29T10:41:28.362 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs_common 2023-03-29T10:41:28.362 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs_common/grace.ko 2023-03-29T10:41:28.362 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfs_common/nfs_acl.ko 2023-03-29T10:41:28.362 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfsd 2023-03-29T10:41:28.362 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nfsd/nfsd.ko 2023-03-29T10:41:28.362 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nilfs2 2023-03-29T10:41:28.362 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nilfs2/nilfs2.ko 2023-03-29T10:41:28.362 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls 2023-03-29T10:41:28.362 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-celtic.ko 2023-03-29T10:41:28.362 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-centeuro.ko 2023-03-29T10:41:28.362 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-croatian.ko 2023-03-29T10:41:28.362 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-cyrillic.ko 2023-03-29T10:41:28.362 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-gaelic.ko 2023-03-29T10:41:28.362 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-greek.ko 2023-03-29T10:41:28.362 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-iceland.ko 2023-03-29T10:41:28.362 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-inuit.ko 2023-03-29T10:41:28.363 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-roman.ko 2023-03-29T10:41:28.363 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-romanian.ko 2023-03-29T10:41:28.363 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/mac-turkish.ko 2023-03-29T10:41:28.363 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp1250.ko 2023-03-29T10:41:28.363 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp1251.ko 2023-03-29T10:41:28.363 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp1255.ko 2023-03-29T10:41:28.363 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp737.ko 2023-03-29T10:41:28.363 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp775.ko 2023-03-29T10:41:28.363 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp850.ko 2023-03-29T10:41:28.363 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp852.ko 2023-03-29T10:41:28.363 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp855.ko 2023-03-29T10:41:28.363 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp857.ko 2023-03-29T10:41:28.363 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp860.ko 2023-03-29T10:41:28.363 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp861.ko 2023-03-29T10:41:28.363 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp862.ko 2023-03-29T10:41:28.363 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp863.ko 2023-03-29T10:41:28.363 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp864.ko 2023-03-29T10:41:28.364 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp865.ko 2023-03-29T10:41:28.364 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp866.ko 2023-03-29T10:41:28.364 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp869.ko 2023-03-29T10:41:28.364 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp874.ko 2023-03-29T10:41:28.364 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp932.ko 2023-03-29T10:41:28.364 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp936.ko 2023-03-29T10:41:28.364 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp949.ko 2023-03-29T10:41:28.364 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_cp950.ko 2023-03-29T10:41:28.364 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_euc-jp.ko 2023-03-29T10:41:28.364 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-1.ko 2023-03-29T10:41:28.364 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-13.ko 2023-03-29T10:41:28.364 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-14.ko 2023-03-29T10:41:28.364 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-15.ko 2023-03-29T10:41:28.364 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-2.ko 2023-03-29T10:41:28.364 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-3.ko 2023-03-29T10:41:28.364 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-4.ko 2023-03-29T10:41:28.364 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-5.ko 2023-03-29T10:41:28.365 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-6.ko 2023-03-29T10:41:28.365 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-7.ko 2023-03-29T10:41:28.365 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_iso8859-9.ko 2023-03-29T10:41:28.365 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_koi8-r.ko 2023-03-29T10:41:28.365 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_koi8-ru.ko 2023-03-29T10:41:28.365 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_koi8-u.ko 2023-03-29T10:41:28.365 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/nls/nls_utf8.ko 2023-03-29T10:41:28.365 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2 2023-03-29T10:41:28.365 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/cluster 2023-03-29T10:41:28.365 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-03-29T10:41:28.365 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/dlm 2023-03-29T10:41:28.365 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-03-29T10:41:28.365 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/dlmfs 2023-03-29T10:41:28.365 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-03-29T10:41:28.365 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/ocfs2.ko 2023-03-29T10:41:28.365 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-03-29T10:41:28.365 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-03-29T10:41:28.365 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-03-29T10:41:28.366 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/overlayfs 2023-03-29T10:41:28.366 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/overlayfs/overlay.ko 2023-03-29T10:41:28.366 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/pstore 2023-03-29T10:41:28.366 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/pstore/ramoops.ko 2023-03-29T10:41:28.366 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/reiserfs 2023-03-29T10:41:28.366 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/reiserfs/reiserfs.ko 2023-03-29T10:41:28.366 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/romfs 2023-03-29T10:41:28.366 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/romfs/romfs.ko 2023-03-29T10:41:28.366 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/smbfs_common 2023-03-29T10:41:28.366 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/smbfs_common/cifs_arc4.ko 2023-03-29T10:41:28.366 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/smbfs_common/cifs_md4.ko 2023-03-29T10:41:28.366 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/squashfs 2023-03-29T10:41:28.366 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/squashfs/squashfs.ko 2023-03-29T10:41:28.366 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/sysv 2023-03-29T10:41:28.366 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/sysv/sysv.ko 2023-03-29T10:41:28.366 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/udf 2023-03-29T10:41:28.366 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/udf/udf.ko 2023-03-29T10:41:28.367 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ufs 2023-03-29T10:41:28.367 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/ufs/ufs.ko 2023-03-29T10:41:28.367 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/xfs 2023-03-29T10:41:28.367 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/fs/xfs/xfs.ko 2023-03-29T10:41:28.367 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/kernel 2023-03-29T10:41:28.367 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/kernel/rcu 2023-03-29T10:41:28.367 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/kernel/rcu/rcutorture.ko 2023-03-29T10:41:28.367 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/kernel/torture.ko 2023-03-29T10:41:28.367 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/kernel/trace 2023-03-29T10:41:28.367 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-03-29T10:41:28.367 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib 2023-03-29T10:41:28.367 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/asn1_encoder.ko 2023-03-29T10:41:28.367 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/crc-itu-t.ko 2023-03-29T10:41:28.367 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/crc8.ko 2023-03-29T10:41:28.367 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/crypto 2023-03-29T10:41:28.367 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/crypto/libarc4.ko 2023-03-29T10:41:28.367 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/crypto/libdes.ko 2023-03-29T10:41:28.367 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/libcrc32c.ko 2023-03-29T10:41:28.368 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/lru_cache.ko 2023-03-29T10:41:28.368 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/lz4 2023-03-29T10:41:28.368 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/lz4/lz4_compress.ko 2023-03-29T10:41:28.368 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/lz4/lz4hc_compress.ko 2023-03-29T10:41:28.368 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/math 2023-03-29T10:41:28.368 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/math/cordic.ko 2023-03-29T10:41:28.368 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/raid6 2023-03-29T10:41:28.368 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/raid6/raid6_pq.ko 2023-03-29T10:41:28.368 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/reed_solomon 2023-03-29T10:41:28.368 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/reed_solomon/reed_solomon.ko 2023-03-29T10:41:28.368 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/ts_bm.ko 2023-03-29T10:41:28.368 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/ts_fsm.ko 2023-03-29T10:41:28.368 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/ts_kmp.ko 2023-03-29T10:41:28.368 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/zstd 2023-03-29T10:41:28.368 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/lib/zstd/zstd_compress.ko 2023-03-29T10:41:28.368 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/mm 2023-03-29T10:41:28.368 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/mm/hwpoison-inject.ko 2023-03-29T10:41:28.369 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net 2023-03-29T10:41:28.369 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan 2023-03-29T10:41:28.369 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/6lowpan.ko 2023-03-29T10:41:28.369 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_dest.ko 2023-03-29T10:41:28.369 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_fragment.ko 2023-03-29T10:41:28.369 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_hop.ko 2023-03-29T10:41:28.369 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_ipv6.ko 2023-03-29T10:41:28.369 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_mobility.ko 2023-03-29T10:41:28.369 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_routing.ko 2023-03-29T10:41:28.369 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/6lowpan/nhc_udp.ko 2023-03-29T10:41:28.369 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/802 2023-03-29T10:41:28.369 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/802/garp.ko 2023-03-29T10:41:28.369 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/802/mrp.ko 2023-03-29T10:41:28.369 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/802/p8022.ko 2023-03-29T10:41:28.369 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/802/psnap.ko 2023-03-29T10:41:28.369 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/802/stp.ko 2023-03-29T10:41:28.369 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/8021q 2023-03-29T10:41:28.370 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/8021q/8021q.ko 2023-03-29T10:41:28.370 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/9p 2023-03-29T10:41:28.370 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/9p/9pnet.ko 2023-03-29T10:41:28.370 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/9p/9pnet_fd.ko 2023-03-29T10:41:28.370 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/9p/9pnet_rdma.ko 2023-03-29T10:41:28.370 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/9p/9pnet_virtio.ko 2023-03-29T10:41:28.370 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/batman-adv 2023-03-29T10:41:28.370 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/batman-adv/batman-adv.ko 2023-03-29T10:41:28.370 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge 2023-03-29T10:41:28.370 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/br_netfilter.ko 2023-03-29T10:41:28.370 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/bridge.ko 2023-03-29T10:41:28.370 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter 2023-03-29T10:41:28.370 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-03-29T10:41:28.370 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_among.ko 2023-03-29T10:41:28.370 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_arp.ko 2023-03-29T10:41:28.370 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-03-29T10:41:28.370 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-03-29T10:41:28.370 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_ip.ko 2023-03-29T10:41:28.371 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-03-29T10:41:28.371 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_limit.ko 2023-03-29T10:41:28.371 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_log.ko 2023-03-29T10:41:28.371 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_mark.ko 2023-03-29T10:41:28.371 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-03-29T10:41:28.371 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-03-29T10:41:28.371 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-03-29T10:41:28.371 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-03-29T10:41:28.371 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_snat.ko 2023-03-29T10:41:28.371 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_stp.ko 2023-03-29T10:41:28.371 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-03-29T10:41:28.371 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-03-29T10:41:28.371 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-03-29T10:41:28.371 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-03-29T10:41:28.371 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/ebtables.ko 2023-03-29T10:41:28.371 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-03-29T10:41:28.371 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-03-29T10:41:28.371 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-03-29T10:41:28.371 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ceph 2023-03-29T10:41:28.372 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ceph/libceph.ko 2023-03-29T10:41:28.372 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/core 2023-03-29T10:41:28.372 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/core/failover.ko 2023-03-29T10:41:28.372 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/core/pktgen.ko 2023-03-29T10:41:28.372 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dccp 2023-03-29T10:41:28.372 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dccp/dccp.ko 2023-03-29T10:41:28.372 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dccp/dccp_diag.ko 2023-03-29T10:41:28.372 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dccp/dccp_ipv4.ko 2023-03-29T10:41:28.372 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dccp/dccp_ipv6.ko 2023-03-29T10:41:28.372 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dns_resolver 2023-03-29T10:41:28.372 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dns_resolver/dns_resolver.ko 2023-03-29T10:41:28.372 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dsa 2023-03-29T10:41:28.372 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dsa/dsa_core.ko 2023-03-29T10:41:28.372 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dsa/tag_brcm.ko 2023-03-29T10:41:28.372 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dsa/tag_dsa.ko 2023-03-29T10:41:28.372 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dsa/tag_none.ko 2023-03-29T10:41:28.372 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/dsa/tag_trailer.ko 2023-03-29T10:41:28.372 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ieee802154 2023-03-29T10:41:28.372 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ieee802154/6lowpan 2023-03-29T10:41:28.373 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-03-29T10:41:28.373 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ieee802154/ieee802154.ko 2023-03-29T10:41:28.373 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ieee802154/ieee802154_socket.ko 2023-03-29T10:41:28.373 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4 2023-03-29T10:41:28.373 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/ah4.ko 2023-03-29T10:41:28.373 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/esp4.ko 2023-03-29T10:41:28.373 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/fou.ko 2023-03-29T10:41:28.373 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/gre.ko 2023-03-29T10:41:28.373 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/inet_diag.ko 2023-03-29T10:41:28.373 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/ip_gre.ko 2023-03-29T10:41:28.373 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/ip_tunnel.ko 2023-03-29T10:41:28.373 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/ip_vti.ko 2023-03-29T10:41:28.373 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/ipcomp.ko 2023-03-29T10:41:28.373 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/ipip.ko 2023-03-29T10:41:28.373 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter 2023-03-29T10:41:28.373 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/arp_tables.ko 2023-03-29T10:41:28.373 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-03-29T10:41:28.374 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-03-29T10:41:28.374 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-03-29T10:41:28.374 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-03-29T10:41:28.374 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-03-29T10:41:28.374 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-03-29T10:41:28.374 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-03-29T10:41:28.374 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-03-29T10:41:28.374 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-03-29T10:41:28.374 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/iptable_security.ko 2023-03-29T10:41:28.374 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-03-29T10:41:28.374 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-03-29T10:41:28.374 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-03-29T10:41:28.374 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-03-29T10:41:28.374 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-03-29T10:41:28.374 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-03-29T10:41:28.374 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-03-29T10:41:28.374 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-03-29T10:41:28.374 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-03-29T10:41:28.375 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-03-29T10:41:28.375 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_bic.ko 2023-03-29T10:41:28.375 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_dctcp.ko 2023-03-29T10:41:28.375 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_diag.ko 2023-03-29T10:41:28.375 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_highspeed.ko 2023-03-29T10:41:28.375 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_htcp.ko 2023-03-29T10:41:28.375 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_hybla.ko 2023-03-29T10:41:28.375 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_illinois.ko 2023-03-29T10:41:28.375 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_lp.ko 2023-03-29T10:41:28.375 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_scalable.ko 2023-03-29T10:41:28.375 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_vegas.ko 2023-03-29T10:41:28.375 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_veno.ko 2023-03-29T10:41:28.375 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_westwood.ko 2023-03-29T10:41:28.375 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tcp_yeah.ko 2023-03-29T10:41:28.375 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/tunnel4.ko 2023-03-29T10:41:28.375 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/udp_diag.ko 2023-03-29T10:41:28.376 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/udp_tunnel.ko 2023-03-29T10:41:28.376 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv4/xfrm4_tunnel.ko 2023-03-29T10:41:28.376 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6 2023-03-29T10:41:28.376 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/ah6.ko 2023-03-29T10:41:28.376 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/esp6.ko 2023-03-29T10:41:28.376 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/fou6.ko 2023-03-29T10:41:28.376 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/ip6_tunnel.ko 2023-03-29T10:41:28.376 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-03-29T10:41:28.376 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/ip6_vti.ko 2023-03-29T10:41:28.376 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/ipcomp6.ko 2023-03-29T10:41:28.376 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter 2023-03-29T10:41:28.376 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-03-29T10:41:28.376 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-03-29T10:41:28.376 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-03-29T10:41:28.376 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-03-29T10:41:28.376 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-03-29T10:41:28.376 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-03-29T10:41:28.377 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-03-29T10:41:28.377 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-03-29T10:41:28.377 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-03-29T10:41:28.377 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-03-29T10:41:28.377 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-03-29T10:41:28.377 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-03-29T10:41:28.377 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-03-29T10:41:28.377 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-03-29T10:41:28.377 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-03-29T10:41:28.377 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-03-29T10:41:28.377 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-03-29T10:41:28.377 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-03-29T10:41:28.377 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-03-29T10:41:28.378 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-03-29T10:41:28.378 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-03-29T10:41:28.378 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-03-29T10:41:28.378 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-03-29T10:41:28.378 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-03-29T10:41:28.378 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/sit.ko 2023-03-29T10:41:28.378 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/tunnel6.ko 2023-03-29T10:41:28.378 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/ipv6/xfrm6_tunnel.ko 2023-03-29T10:41:28.378 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/key 2023-03-29T10:41:28.378 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/key/af_key.ko 2023-03-29T10:41:28.378 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp 2023-03-29T10:41:28.378 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_core.ko 2023-03-29T10:41:28.378 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_debugfs.ko 2023-03-29T10:41:28.378 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_eth.ko 2023-03-29T10:41:28.378 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_ip.ko 2023-03-29T10:41:28.378 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_ip6.ko 2023-03-29T10:41:28.378 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_netlink.ko 2023-03-29T10:41:28.378 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/l2tp/l2tp_ppp.ko 2023-03-29T10:41:28.379 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/llc 2023-03-29T10:41:28.379 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/llc/llc.ko 2023-03-29T10:41:28.379 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/mac802154 2023-03-29T10:41:28.379 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/mac802154/mac802154.ko 2023-03-29T10:41:28.379 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/mpls 2023-03-29T10:41:28.379 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/mpls/mpls_gso.ko 2023-03-29T10:41:28.379 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter 2023-03-29T10:41:28.379 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset 2023-03-29T10:41:28.379 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set.ko 2023-03-29T10:41:28.379 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-03-29T10:41:28.379 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-03-29T10:41:28.379 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-03-29T10:41:28.379 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-03-29T10:41:28.379 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-03-29T10:41:28.379 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-03-29T10:41:28.379 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-03-29T10:41:28.380 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-03-29T10:41:28.380 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-03-29T10:41:28.380 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-03-29T10:41:28.380 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-03-29T10:41:28.380 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-03-29T10:41:28.380 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-03-29T10:41:28.380 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-03-29T10:41:28.380 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-03-29T10:41:28.380 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs 2023-03-29T10:41:28.380 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs.ko 2023-03-29T10:41:28.380 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-03-29T10:41:28.380 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-03-29T10:41:28.380 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-03-29T10:41:28.380 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-03-29T10:41:28.380 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-03-29T10:41:28.380 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-03-29T10:41:28.380 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-03-29T10:41:28.380 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-03-29T10:41:28.381 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-03-29T10:41:28.381 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-03-29T10:41:28.381 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-03-29T10:41:28.381 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-03-29T10:41:28.381 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-03-29T10:41:28.381 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conncount.ko 2023-03-29T10:41:28.381 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack.ko 2023-03-29T10:41:28.381 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-03-29T10:41:28.381 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-03-29T10:41:28.381 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-03-29T10:41:28.381 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_h323.ko 2023-03-29T10:41:28.381 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_irc.ko 2023-03-29T10:41:28.381 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-03-29T10:41:28.381 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-03-29T10:41:28.381 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-03-29T10:41:28.381 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_sane.ko 2023-03-29T10:41:28.382 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_sip.ko 2023-03-29T10:41:28.382 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-03-29T10:41:28.382 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-03-29T10:41:28.382 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_dup_netdev.ko 2023-03-29T10:41:28.382 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_flow_table.ko 2023-03-29T10:41:28.382 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_flow_table_inet.ko 2023-03-29T10:41:28.382 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_log_syslog.ko 2023-03-29T10:41:28.382 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_nat.ko 2023-03-29T10:41:28.382 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_nat_amanda.ko 2023-03-29T10:41:28.382 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_nat_ftp.ko 2023-03-29T10:41:28.382 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_nat_irc.ko 2023-03-29T10:41:28.382 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_nat_sip.ko 2023-03-29T10:41:28.382 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_nat_tftp.ko 2023-03-29T10:41:28.382 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_synproxy_core.ko 2023-03-29T10:41:28.382 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nf_tables.ko 2023-03-29T10:41:28.382 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nfnetlink.ko 2023-03-29T10:41:28.382 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nfnetlink_acct.ko 2023-03-29T10:41:28.383 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nfnetlink_log.ko 2023-03-29T10:41:28.383 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nfnetlink_osf.ko 2023-03-29T10:41:28.383 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nfnetlink_queue.ko 2023-03-29T10:41:28.383 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_chain_nat.ko 2023-03-29T10:41:28.383 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_compat.ko 2023-03-29T10:41:28.383 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_ct.ko 2023-03-29T10:41:28.383 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_dup_netdev.ko 2023-03-29T10:41:28.383 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_fib.ko 2023-03-29T10:41:28.383 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_fwd_netdev.ko 2023-03-29T10:41:28.383 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_hash.ko 2023-03-29T10:41:28.383 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_limit.ko 2023-03-29T10:41:28.383 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_log.ko 2023-03-29T10:41:28.383 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_masq.ko 2023-03-29T10:41:28.383 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_nat.ko 2023-03-29T10:41:28.383 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_numgen.ko 2023-03-29T10:41:28.384 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_queue.ko 2023-03-29T10:41:28.384 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_quota.ko 2023-03-29T10:41:28.384 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_redir.ko 2023-03-29T10:41:28.384 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_reject.ko 2023-03-29T10:41:28.384 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_reject_inet.ko 2023-03-29T10:41:28.384 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_socket.ko 2023-03-29T10:41:28.384 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/nft_tunnel.ko 2023-03-29T10:41:28.384 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_AUDIT.ko 2023-03-29T10:41:28.384 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_CHECKSUM.ko 2023-03-29T10:41:28.384 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_CLASSIFY.ko 2023-03-29T10:41:28.384 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-03-29T10:41:28.384 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_CT.ko 2023-03-29T10:41:28.384 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_DSCP.ko 2023-03-29T10:41:28.384 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_HL.ko 2023-03-29T10:41:28.384 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_HMARK.ko 2023-03-29T10:41:28.384 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_IDLETIMER.ko 2023-03-29T10:41:28.384 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_LED.ko 2023-03-29T10:41:28.385 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_LOG.ko 2023-03-29T10:41:28.385 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_MASQUERADE.ko 2023-03-29T10:41:28.385 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_NETMAP.ko 2023-03-29T10:41:28.385 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_NFLOG.ko 2023-03-29T10:41:28.385 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_NFQUEUE.ko 2023-03-29T10:41:28.385 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_RATEEST.ko 2023-03-29T10:41:28.385 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_REDIRECT.ko 2023-03-29T10:41:28.385 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_SECMARK.ko 2023-03-29T10:41:28.385 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_TCPMSS.ko 2023-03-29T10:41:28.385 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-03-29T10:41:28.385 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_TEE.ko 2023-03-29T10:41:28.385 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_TPROXY.ko 2023-03-29T10:41:28.385 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_TRACE.ko 2023-03-29T10:41:28.385 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_addrtype.ko 2023-03-29T10:41:28.385 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_bpf.ko 2023-03-29T10:41:28.385 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_cgroup.ko 2023-03-29T10:41:28.386 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_cluster.ko 2023-03-29T10:41:28.386 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_comment.ko 2023-03-29T10:41:28.386 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_connbytes.ko 2023-03-29T10:41:28.386 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_connlabel.ko 2023-03-29T10:41:28.386 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_connlimit.ko 2023-03-29T10:41:28.386 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_connmark.ko 2023-03-29T10:41:28.386 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_conntrack.ko 2023-03-29T10:41:28.386 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_cpu.ko 2023-03-29T10:41:28.386 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_dccp.ko 2023-03-29T10:41:28.386 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_devgroup.ko 2023-03-29T10:41:28.386 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_dscp.ko 2023-03-29T10:41:28.386 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_ecn.ko 2023-03-29T10:41:28.386 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_esp.ko 2023-03-29T10:41:28.386 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_hashlimit.ko 2023-03-29T10:41:28.386 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_helper.ko 2023-03-29T10:41:28.386 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_hl.ko 2023-03-29T10:41:28.386 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_ipcomp.ko 2023-03-29T10:41:28.387 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_iprange.ko 2023-03-29T10:41:28.387 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_ipvs.ko 2023-03-29T10:41:28.387 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_l2tp.ko 2023-03-29T10:41:28.387 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_length.ko 2023-03-29T10:41:28.387 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_limit.ko 2023-03-29T10:41:28.387 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_mac.ko 2023-03-29T10:41:28.387 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_mark.ko 2023-03-29T10:41:28.387 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_multiport.ko 2023-03-29T10:41:28.387 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_nat.ko 2023-03-29T10:41:28.387 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_nfacct.ko 2023-03-29T10:41:28.387 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_osf.ko 2023-03-29T10:41:28.387 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_owner.ko 2023-03-29T10:41:28.387 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_physdev.ko 2023-03-29T10:41:28.387 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_pkttype.ko 2023-03-29T10:41:28.387 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_policy.ko 2023-03-29T10:41:28.387 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_quota.ko 2023-03-29T10:41:28.387 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_rateest.ko 2023-03-29T10:41:28.388 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_realm.ko 2023-03-29T10:41:28.388 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_recent.ko 2023-03-29T10:41:28.388 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_sctp.ko 2023-03-29T10:41:28.388 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_set.ko 2023-03-29T10:41:28.388 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_socket.ko 2023-03-29T10:41:28.388 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_state.ko 2023-03-29T10:41:28.388 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_statistic.ko 2023-03-29T10:41:28.388 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_string.ko 2023-03-29T10:41:28.388 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_tcpmss.ko 2023-03-29T10:41:28.388 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_time.ko 2023-03-29T10:41:28.388 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netfilter/xt_u32.ko 2023-03-29T10:41:28.388 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netlink 2023-03-29T10:41:28.388 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/netlink/netlink_diag.ko 2023-03-29T10:41:28.388 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/nsh 2023-03-29T10:41:28.388 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/nsh/nsh.ko 2023-03-29T10:41:28.388 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/openvswitch 2023-03-29T10:41:28.388 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/openvswitch/openvswitch.ko 2023-03-29T10:41:28.388 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/openvswitch/vport-geneve.ko 2023-03-29T10:41:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/openvswitch/vport-gre.ko 2023-03-29T10:41:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/openvswitch/vport-vxlan.ko 2023-03-29T10:41:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/packet 2023-03-29T10:41:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/packet/af_packet_diag.ko 2023-03-29T10:41:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/rds 2023-03-29T10:41:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/rds/rds.ko 2023-03-29T10:41:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/rds/rds_rdma.ko 2023-03-29T10:41:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/rds/rds_tcp.ko 2023-03-29T10:41:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched 2023-03-29T10:41:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_csum.ko 2023-03-29T10:41:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_gact.ko 2023-03-29T10:41:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_ipt.ko 2023-03-29T10:41:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_mirred.ko 2023-03-29T10:41:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_nat.ko 2023-03-29T10:41:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_pedit.ko 2023-03-29T10:41:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_police.ko 2023-03-29T10:41:28.389 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_simple.ko 2023-03-29T10:41:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_skbedit.ko 2023-03-29T10:41:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/act_vlan.ko 2023-03-29T10:41:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/cls_basic.ko 2023-03-29T10:41:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/cls_bpf.ko 2023-03-29T10:41:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/cls_flow.ko 2023-03-29T10:41:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/cls_fw.ko 2023-03-29T10:41:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/cls_route.ko 2023-03-29T10:41:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/cls_u32.ko 2023-03-29T10:41:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/em_cmp.ko 2023-03-29T10:41:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/em_ipset.ko 2023-03-29T10:41:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/em_meta.ko 2023-03-29T10:41:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/em_nbyte.ko 2023-03-29T10:41:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/em_text.ko 2023-03-29T10:41:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/em_u32.ko 2023-03-29T10:41:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_choke.ko 2023-03-29T10:41:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_codel.ko 2023-03-29T10:41:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_drr.ko 2023-03-29T10:41:28.390 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_fq.ko 2023-03-29T10:41:28.391 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_gred.ko 2023-03-29T10:41:28.391 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_hfsc.ko 2023-03-29T10:41:28.391 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_hhf.ko 2023-03-29T10:41:28.391 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_htb.ko 2023-03-29T10:41:28.391 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_ingress.ko 2023-03-29T10:41:28.391 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_mqprio.ko 2023-03-29T10:41:28.391 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_mqprio_lib.ko 2023-03-29T10:41:28.391 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_multiq.ko 2023-03-29T10:41:28.391 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_netem.ko 2023-03-29T10:41:28.391 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_pie.ko 2023-03-29T10:41:28.391 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_plug.ko 2023-03-29T10:41:28.391 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_prio.ko 2023-03-29T10:41:28.391 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_qfq.ko 2023-03-29T10:41:28.391 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_red.ko 2023-03-29T10:41:28.391 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_sfb.ko 2023-03-29T10:41:28.391 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_sfq.ko 2023-03-29T10:41:28.391 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_tbf.ko 2023-03-29T10:41:28.391 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sched/sch_teql.ko 2023-03-29T10:41:28.392 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sctp 2023-03-29T10:41:28.392 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sctp/sctp.ko 2023-03-29T10:41:28.392 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sctp/sctp_diag.ko 2023-03-29T10:41:28.392 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc 2023-03-29T10:41:28.392 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc/auth_gss 2023-03-29T10:41:28.392 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-03-29T10:41:28.392 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-03-29T10:41:28.392 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc/sunrpc.ko 2023-03-29T10:41:28.392 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc/xprtrdma 2023-03-29T10:41:28.392 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-03-29T10:41:28.392 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/tipc 2023-03-29T10:41:28.392 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/tipc/diag.ko 2023-03-29T10:41:28.392 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/tipc/tipc.ko 2023-03-29T10:41:28.392 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/unix 2023-03-29T10:41:28.392 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/unix/unix_diag.ko 2023-03-29T10:41:28.392 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/vmw_vsock 2023-03-29T10:41:28.392 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-03-29T10:41:28.393 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-03-29T10:41:28.393 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/vmw_vsock/vsock.ko 2023-03-29T10:41:28.393 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/vmw_vsock/vsock_diag.ko 2023-03-29T10:41:28.393 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/vmw_vsock/vsock_loopback.ko 2023-03-29T10:41:28.393 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/xfrm 2023-03-29T10:41:28.393 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/net/xfrm/xfrm_ipcomp.ko 2023-03-29T10:41:28.393 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/security 2023-03-29T10:41:28.393 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/security/keys 2023-03-29T10:41:28.393 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/security/keys/encrypted-keys 2023-03-29T10:41:28.393 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-03-29T10:41:28.393 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/security/keys/trusted-keys 2023-03-29T10:41:28.393 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/security/keys/trusted-keys/trusted.ko 2023-03-29T10:41:28.393 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/virt 2023-03-29T10:41:28.393 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/virt/lib 2023-03-29T10:41:28.393 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/kernel/virt/lib/irqbypass.ko 2023-03-29T10:41:28.393 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.alias 2023-03-29T10:41:28.393 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.alias.bin 2023-03-29T10:41:28.394 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.builtin 2023-03-29T10:41:28.394 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.builtin.bin 2023-03-29T10:41:28.394 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.builtin.modinfo 2023-03-29T10:41:28.394 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.dep 2023-03-29T10:41:28.394 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.dep.bin 2023-03-29T10:41:28.394 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.devname 2023-03-29T10:41:28.394 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.order 2023-03-29T10:41:28.394 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.softdep 2023-03-29T10:41:28.394 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.symbols 2023-03-29T10:41:28.394 INFO:teuthology.orchestra.run.smithi111.stdout:/lib/modules/6.3.0-rc3-g1776e8f494e9/modules.symbols.bin 2023-03-29T10:41:28.394 DEBUG:teuthology.task.kernel:get_image_version: 6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:28.395 DEBUG:teuthology.orchestra.run.smithi111:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-03-29T10:41:28.419 INFO:teuthology.orchestra.run.smithi111.stdout:postinstall scriptlet (using /bin/sh): 2023-03-29T10:41:28.419 INFO:teuthology.orchestra.run.smithi111.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.3.0-rc3-g1776e8f494e9 -a -r /boot/System.map-6.3.0-rc3-g1776e8f494e9 ]; then 2023-03-29T10:41:28.419 INFO:teuthology.orchestra.run.smithi111.stdout:cp /boot/vmlinuz-6.3.0-rc3-g1776e8f494e9 /boot/.vmlinuz-6.3.0-rc3-g1776e8f494e9-rpm 2023-03-29T10:41:28.419 INFO:teuthology.orchestra.run.smithi111.stdout:cp /boot/System.map-6.3.0-rc3-g1776e8f494e9 /boot/.System.map-6.3.0-rc3-g1776e8f494e9-rpm 2023-03-29T10:41:28.419 INFO:teuthology.orchestra.run.smithi111.stdout:rm -f /boot/vmlinuz-6.3.0-rc3-g1776e8f494e9 /boot/System.map-6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:28.419 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/installkernel 6.3.0-rc3-g1776e8f494e9 /boot/.vmlinuz-6.3.0-rc3-g1776e8f494e9-rpm /boot/.System.map-6.3.0-rc3-g1776e8f494e9-rpm 2023-03-29T10:41:28.419 INFO:teuthology.orchestra.run.smithi111.stdout:rm -f /boot/.vmlinuz-6.3.0-rc3-g1776e8f494e9-rpm /boot/.System.map-6.3.0-rc3-g1776e8f494e9-rpm 2023-03-29T10:41:28.419 INFO:teuthology.orchestra.run.smithi111.stdout:fi 2023-03-29T10:41:28.419 INFO:teuthology.orchestra.run.smithi111.stdout:preuninstall scriptlet (using /bin/sh): 2023-03-29T10:41:28.419 INFO:teuthology.orchestra.run.smithi111.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-03-29T10:41:28.419 INFO:teuthology.orchestra.run.smithi111.stdout:new-kernel-pkg --remove 6.3.0-rc3-g1776e8f494e9 --rminitrd --initrdfile=/boot/initramfs-6.3.0-rc3-g1776e8f494e9.img 2023-03-29T10:41:28.419 INFO:teuthology.orchestra.run.smithi111.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-03-29T10:41:28.419 INFO:teuthology.orchestra.run.smithi111.stdout:kernel-install remove 6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:28.420 INFO:teuthology.orchestra.run.smithi111.stdout:fi 2023-03-29T10:41:28.420 INFO:teuthology.orchestra.run.smithi111.stdout:postuninstall scriptlet (using /bin/sh): 2023-03-29T10:41:28.420 INFO:teuthology.orchestra.run.smithi111.stdout:if [ -x /sbin/update-bootloader ]; then 2023-03-29T10:41:28.420 INFO:teuthology.orchestra.run.smithi111.stdout:/sbin/update-bootloader --remove 6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:28.420 INFO:teuthology.orchestra.run.smithi111.stdout:fi 2023-03-29T10:41:28.420 DEBUG:teuthology.orchestra.run.smithi111:> sudo rpm -qi grub2-tools 2023-03-29T10:41:28.495 INFO:teuthology.orchestra.run.smithi111.stdout:Name : grub2-tools 2023-03-29T10:41:28.495 INFO:teuthology.orchestra.run.smithi111.stdout:Epoch : 1 2023-03-29T10:41:28.496 INFO:teuthology.orchestra.run.smithi111.stdout:Version : 2.02 2023-03-29T10:41:28.496 INFO:teuthology.orchestra.run.smithi111.stdout:Release : 129.el8 2023-03-29T10:41:28.496 INFO:teuthology.orchestra.run.smithi111.stdout:Architecture: x86_64 2023-03-29T10:41:28.496 INFO:teuthology.orchestra.run.smithi111.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-03-29T10:41:28.496 INFO:teuthology.orchestra.run.smithi111.stdout:Group : System Environment/Base 2023-03-29T10:41:28.496 INFO:teuthology.orchestra.run.smithi111.stdout:Size : 9265586 2023-03-29T10:41:28.496 INFO:teuthology.orchestra.run.smithi111.stdout:License : GPLv3+ 2023-03-29T10:41:28.496 INFO:teuthology.orchestra.run.smithi111.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-03-29T10:41:28.496 INFO:teuthology.orchestra.run.smithi111.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-03-29T10:41:28.496 INFO:teuthology.orchestra.run.smithi111.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-03-29T10:41:28.497 INFO:teuthology.orchestra.run.smithi111.stdout:Build Host : kbuilder.bsys.centos.org 2023-03-29T10:41:28.497 INFO:teuthology.orchestra.run.smithi111.stdout:Relocations : (not relocatable) 2023-03-29T10:41:28.497 INFO:teuthology.orchestra.run.smithi111.stdout:Packager : CentOS BuildSystem 2023-03-29T10:41:28.497 INFO:teuthology.orchestra.run.smithi111.stdout:Vendor : CentOS 2023-03-29T10:41:28.497 INFO:teuthology.orchestra.run.smithi111.stdout:URL : http://www.gnu.org/software/grub/ 2023-03-29T10:41:28.497 INFO:teuthology.orchestra.run.smithi111.stdout:Summary : Support tools for GRUB. 2023-03-29T10:41:28.497 INFO:teuthology.orchestra.run.smithi111.stdout:Description : 2023-03-29T10:41:28.497 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:41:28.498 INFO:teuthology.orchestra.run.smithi111.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-03-29T10:41:28.498 INFO:teuthology.orchestra.run.smithi111.stdout:customizable bootloader with modular architecture. It supports a rich 2023-03-29T10:41:28.498 INFO:teuthology.orchestra.run.smithi111.stdout:variety of kernel formats, file systems, computer architectures and 2023-03-29T10:41:28.498 INFO:teuthology.orchestra.run.smithi111.stdout:hardware devices. 2023-03-29T10:41:28.498 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:41:28.498 INFO:teuthology.orchestra.run.smithi111.stdout:This subpackage provides tools for support of all platforms. 2023-03-29T10:41:28.499 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-03-29T10:41:28.499 INFO:teuthology.task.kernel:Updating grub on smithi111 to boot 6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:28.500 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-03-29T10:41:28.730 INFO:teuthology.orchestra.run.smithi119.stderr:done 2023-03-29T10:41:28.732 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2023-03-29T10:41:28.754 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.VQMKWXTsLY 2023-03-29T10:41:28.755 DEBUG:teuthology.orchestra.run.smithi119:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.VQMKWXTsLY 2023-03-29T10:41:28.828 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.VQMKWXTsLY 2023-03-29T10:41:28.990 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.VQMKWXTsLY is 6KB 2023-03-29T10:41:29.011 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.VQMKWXTsLY 2023-03-29T10:41:29.027 DEBUG:teuthology.orchestra.run.smithi119:> sudo /bin/ls /boot/loader/entries || true 2023-03-29T10:41:29.092 INFO:teuthology.orchestra.run.smithi119.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-03-29T10:41:29.092 INFO:teuthology.orchestra.run.smithi119.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-03-29T10:41:29.092 INFO:teuthology.orchestra.run.smithi119.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-03-29T10:41:29.092 INFO:teuthology.orchestra.run.smithi119.stdout:2ff282740cf84d4fa02ff60ed95fbbff-6.3.0-rc3-g1776e8f494e9.conf 2023-03-29T10:41:29.093 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:29.250 DEBUG:teuthology.orchestra.run.smithi119:> sudo shutdown -r now 2023-03-29T10:41:32.701 INFO:teuthology.orchestra.run.smithi111.stderr:Generating grub configuration file ... 2023-03-29T10:41:34.787 INFO:teuthology.orchestra.run.smithi111.stderr:done 2023-03-29T10:41:34.789 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2023-03-29T10:41:34.811 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.18rqpP8p2r 2023-03-29T10:41:34.811 DEBUG:teuthology.orchestra.run.smithi111:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.18rqpP8p2r 2023-03-29T10:41:34.885 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0666 /tmp/tmp.18rqpP8p2r 2023-03-29T10:41:35.051 DEBUG:teuthology.orchestra.remote:smithi111:/tmp/tmp.18rqpP8p2r is 6KB 2023-03-29T10:41:35.072 DEBUG:teuthology.orchestra.run.smithi111:> rm -fr /tmp/tmp.18rqpP8p2r 2023-03-29T10:41:35.089 DEBUG:teuthology.orchestra.run.smithi111:> sudo /bin/ls /boot/loader/entries || true 2023-03-29T10:41:35.165 INFO:teuthology.orchestra.run.smithi111.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-03-29T10:41:35.165 INFO:teuthology.orchestra.run.smithi111.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-03-29T10:41:35.165 INFO:teuthology.orchestra.run.smithi111.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-03-29T10:41:35.165 INFO:teuthology.orchestra.run.smithi111.stdout:2ff282740cf84d4fa02ff60ed95fbbff-6.3.0-rc3-g1776e8f494e9.conf 2023-03-29T10:41:35.166 DEBUG:teuthology.orchestra.run.smithi111:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-6.3.0-rc3-g1776e8f494e9 2023-03-29T10:41:35.323 DEBUG:teuthology.orchestra.run.smithi111:> sudo shutdown -r now 2023-03-29T10:41:59.273 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-03-29T10:41:59.273 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-03-29T10:41:59.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-29T10:42:05.329 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-03-29T10:42:05.329 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-03-29T10:42:05.329 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-29T10:42:33.865 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-03-29T10:42:40.005 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-03-29T10:43:03.889 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-03-29T10:43:03.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-29T10:43:10.009 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-03-29T10:43:10.010 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-29T10:43:19.552 DEBUG:teuthology.orchestra.run.smithi119:> true 2023-03-29T10:43:19.945 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-03-29T10:43:19.945 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.3.0-rc3-g1776e8f494e9"... 2023-03-29T10:43:19.945 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2023-03-29T10:43:19.985 INFO:teuthology.orchestra.run.smithi119.stdout:6.3.0-rc3-g1776e8f494e9 2023-03-29T10:43:19.986 DEBUG:teuthology.task.kernel:current kernel version is 6.3.0-rc3-g1776e8f494e9 vs 6.3.0-rc3-g1776e8f494e9 2023-03-29T10:43:19.986 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-03-29T10:43:19.986 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-03-29T10:43:20.987 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-03-29T10:43:20.987 DEBUG:teuthology.orchestra.run.smithi119:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-03-29T10:43:21.113 INFO:teuthology.orchestra.run.smithi119.stdout:ttyS1 2023-03-29T10:43:21.136 DEBUG:teuthology.parallel:result is None 2023-03-29T10:43:28.389 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.111 2023-03-29T10:43:58.390 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-03-29T10:43:58.391 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-29T10:43:58.846 DEBUG:teuthology.orchestra.run.smithi111:> true 2023-03-29T10:43:59.247 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-03-29T10:43:59.248 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.3.0-rc3-g1776e8f494e9"... 2023-03-29T10:43:59.248 DEBUG:teuthology.orchestra.run.smithi111:> uname -r 2023-03-29T10:43:59.287 INFO:teuthology.orchestra.run.smithi111.stdout:6.3.0-rc3-g1776e8f494e9 2023-03-29T10:43:59.287 DEBUG:teuthology.task.kernel:current kernel version is 6.3.0-rc3-g1776e8f494e9 vs 6.3.0-rc3-g1776e8f494e9 2023-03-29T10:43:59.288 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-03-29T10:43:59.288 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-03-29T10:44:00.289 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-03-29T10:44:00.289 DEBUG:teuthology.orchestra.run.smithi111:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-03-29T10:44:00.416 INFO:teuthology.orchestra.run.smithi111.stdout:ttyS1 2023-03-29T10:44:00.442 DEBUG:teuthology.parallel:result is None 2023-03-29T10:44:00.443 INFO:teuthology.run_tasks:Running task internal.base... 2023-03-29T10:44:00.450 INFO:teuthology.task.internal:Creating test directory... 2023-03-29T10:44:00.450 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-03-29T10:44:00.454 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-03-29T10:44:00.500 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-03-29T10:44:00.508 INFO:teuthology.run_tasks:Running task internal.archive... 2023-03-29T10:44:00.516 INFO:teuthology.task.internal:Creating archive directory... 2023-03-29T10:44:00.516 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-03-29T10:44:00.537 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-03-29T10:44:00.640 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-03-29T10:44:00.649 INFO:teuthology.task.internal:Enabling coredump saving... 2023-03-29T10:44:00.654 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-03-29T10:44:00.658 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-03-29T10:44:00.724 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-29T10:44:00.740 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-29T10:44:00.748 INFO:teuthology.orchestra.run.smithi111.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-29T10:44:00.764 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-03-29T10:44:00.767 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-03-29T10:44:00.777 INFO:teuthology.task.internal:Configuring sudo... 2023-03-29T10:44:00.777 DEBUG:teuthology.orchestra.run.smithi111:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-03-29T10:44:00.781 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-03-29T10:44:00.842 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-03-29T10:44:00.852 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-03-29T10:44:00.852 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-03-29T10:44:00.856 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-03-29T10:44:00.896 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-29T10:44:01.006 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-29T10:44:01.080 DEBUG:teuthology.orchestra.run.smithi111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-29T10:44:01.153 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-29T10:44:01.225 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:44:01.226 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-03-29T10:44:01.288 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-29T10:44:01.363 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-03-29T10:44:01.436 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-29T10:44:01.508 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-03-29T10:44:01.580 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:44:01.580 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-03-29T10:44:01.641 DEBUG:teuthology.orchestra.run.smithi111:> sudo service rsyslog restart 2023-03-29T10:44:01.644 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2023-03-29T10:44:01.751 INFO:teuthology.orchestra.run.smithi111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-03-29T10:44:01.785 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-03-29T10:44:02.225 INFO:teuthology.run_tasks:Running task internal.timer... 2023-03-29T10:44:02.235 INFO:teuthology.task.internal:Starting timer... 2023-03-29T10:44:02.235 INFO:teuthology.run_tasks:Running task pcp... 2023-03-29T10:44:02.258 INFO:teuthology.run_tasks:Running task selinux... 2023-03-29T10:44:02.267 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-03-29T10:44:02.267 DEBUG:teuthology.orchestra.run.smithi111:> sudo service auditd rotate 2023-03-29T10:44:02.381 INFO:teuthology.orchestra.run.smithi111.stdout:Rotating logs: 2023-03-29T10:44:02.384 DEBUG:teuthology.orchestra.run.smithi119:> sudo service auditd rotate 2023-03-29T10:44:02.484 INFO:teuthology.orchestra.run.smithi119.stdout:Rotating logs: 2023-03-29T10:44:02.488 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-03-29T10:44:02.488 DEBUG:teuthology.orchestra.run.smithi111:> /usr/sbin/getenforce 2023-03-29T10:44:02.535 INFO:teuthology.orchestra.run.smithi111.stdout:Permissive 2023-03-29T10:44:02.536 DEBUG:teuthology.orchestra.run.smithi119:> /usr/sbin/getenforce 2023-03-29T10:44:02.588 INFO:teuthology.orchestra.run.smithi119.stdout:Permissive 2023-03-29T10:44:02.588 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi111.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi119.front.sepia.ceph.com': 'permissive'} 2023-03-29T10:44:02.589 DEBUG:teuthology.orchestra.run.smithi111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-03-29T10:44:02.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T10:44:02.652 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-03-29T10:44:02.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T10:44:02.714 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-03-29T10:44:02.715 DEBUG:teuthology.orchestra.run.smithi111:> sudo /usr/sbin/setenforce permissive 2023-03-29T10:44:02.783 DEBUG:teuthology.orchestra.run.smithi119:> sudo /usr/sbin/setenforce permissive 2023-03-29T10:44:02.845 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-03-29T10:44:02.855 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-03-29T10:44:02.862 INFO:teuthology.repo_utils:Fetching main from origin 2023-03-29T10:44:03.000 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-03-29T10:44:03.022 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-03-29T10:44:03.024 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi111.front.sepia.ceph.com,smithi119.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-03-29T10:53:50.185 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi111.front.sepia.ceph.com'), Remote(name='ubuntu@smithi119.front.sepia.ceph.com')] 2023-03-29T10:53:50.186 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi111.front.sepia.ceph.com' 2023-03-29T10:53:50.187 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-29T10:53:50.274 DEBUG:teuthology.orchestra.run.smithi111:> true 2023-03-29T10:53:50.381 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-03-29T10:53:50.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-03-29T10:53:50.466 DEBUG:teuthology.orchestra.run.smithi119:> true 2023-03-29T10:53:50.573 INFO:teuthology.run_tasks:Running task clock... 2023-03-29T10:53:50.584 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-03-29T10:53:50.585 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-03-29T10:53:50.585 DEBUG:teuthology.orchestra.run.smithi111:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-03-29T10:53:50.589 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-03-29T10:53:50.589 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-03-29T10:53:50.653 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-03-29T10:53:50.681 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-03-29T10:53:50.683 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-03-29T10:53:50.710 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-03-29T10:53:50.765 INFO:teuthology.orchestra.run.smithi111.stderr:sudo: ntpd: command not found 2023-03-29T10:53:50.790 INFO:teuthology.orchestra.run.smithi111.stdout:506 Cannot talk to daemon 2023-03-29T10:53:50.790 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: ntpd: command not found 2023-03-29T10:53:50.815 INFO:teuthology.orchestra.run.smithi119.stdout:506 Cannot talk to daemon 2023-03-29T10:53:50.820 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-03-29T10:53:50.844 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-03-29T10:53:50.850 INFO:teuthology.orchestra.run.smithi111.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-03-29T10:53:50.873 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-03-29T10:53:50.944 INFO:teuthology.orchestra.run.smithi111.stderr:bash: ntpq: command not found 2023-03-29T10:53:50.950 INFO:teuthology.orchestra.run.smithi111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-03-29T10:53:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:=============================================================================== 2023-03-29T10:53:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-29T10:53:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-29T10:53:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-29T10:53:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-29T10:53:50.954 INFO:teuthology.orchestra.run.smithi119.stderr:bash: ntpq: command not found 2023-03-29T10:53:50.961 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-03-29T10:53:50.961 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2023-03-29T10:53:50.961 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-29T10:53:50.961 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-29T10:53:50.961 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-29T10:53:50.961 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-03-29T10:53:50.962 INFO:teuthology.run_tasks:Running task install... 2023-03-29T10:53:50.970 DEBUG:teuthology.task.install:project ceph 2023-03-29T10:53:50.970 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f03d69a8741c605ae6711afe2c9991ac46095f7d'}} 2023-03-29T10:53:50.970 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f03d69a8741c605ae6711afe2c9991ac46095f7d'} 2023-03-29T10:53:50.970 INFO:teuthology.task.install:Using flavor: default 2023-03-29T10:53:50.976 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-03-29T10:53:50.976 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-03-29T10:53:50.977 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f03d69a8741c605ae6711afe2c9991ac46095f7d', 'tag': None, 'wait_for_package': False} 2023-03-29T10:53:50.977 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f03d69a8741c605ae6711afe2c9991ac46095f7d 2023-03-29T10:53:50.979 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f03d69a8741c605ae6711afe2c9991ac46095f7d', 'tag': None, 'wait_for_package': False} 2023-03-29T10:53:50.979 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f03d69a8741c605ae6711afe2c9991ac46095f7d 2023-03-29T10:53:51.118 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230317.095222/f03d69a8741c605ae6711afe2c9991ac46095f7d/centos/8/flavors/default/ 2023-03-29T10:53:51.119 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3066.gf03d69a8 2023-03-29T10:53:51.123 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230317.095222/f03d69a8741c605ae6711afe2c9991ac46095f7d/centos/8/flavors/default/ 2023-03-29T10:53:51.123 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3066.gf03d69a8 2023-03-29T10:53:51.390 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230317.095222/f03d69a8741c605ae6711afe2c9991ac46095f7d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230317.095222/f03d69a8741c605ae6711afe2c9991ac46095f7d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230317.095222/f03d69a8741c605ae6711afe2c9991ac46095f7d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-03-29T10:53:51.390 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:53:51.390 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-03-29T10:53:51.400 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230317.095222/f03d69a8741c605ae6711afe2c9991ac46095f7d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230317.095222/f03d69a8741c605ae6711afe2c9991ac46095f7d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20230317.095222/f03d69a8741c605ae6711afe2c9991ac46095f7d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-03-29T10:53:51.400 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:53:51.400 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-03-29T10:53:51.452 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-03-29T10:53:51.453 DEBUG:teuthology.orchestra.run.smithi111:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f03d69a8741c605ae6711afe2c9991ac46095f7d/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-03-29T10:53:51.462 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-03-29T10:53:51.462 DEBUG:teuthology.orchestra.run.smithi119:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f03d69a8741c605ae6711afe2c9991ac46095f7d/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-03-29T10:53:51.555 DEBUG:teuthology.orchestra.run.smithi111:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-03-29T10:53:51.565 DEBUG:teuthology.orchestra.run.smithi119:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-03-29T10:53:51.640 DEBUG:teuthology.orchestra.run.smithi111:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-03-29T10:53:51.649 DEBUG:teuthology.orchestra.run.smithi119:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-03-29T10:53:51.701 INFO:teuthology.orchestra.run.smithi111.stdout:check_obsoletes = 1 2023-03-29T10:53:51.704 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum clean all 2023-03-29T10:53:51.709 INFO:teuthology.orchestra.run.smithi119.stdout:check_obsoletes = 1 2023-03-29T10:53:51.712 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum clean all 2023-03-29T10:53:52.018 INFO:teuthology.orchestra.run.smithi111.stdout:71 files removed 2023-03-29T10:53:52.021 INFO:teuthology.orchestra.run.smithi119.stdout:71 files removed 2023-03-29T10:53:52.055 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-03-29T10:53:52.056 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-03-29T10:53:53.213 INFO:teuthology.orchestra.run.smithi119.stdout:ceph packages for x86_64 129 kB/s | 80 kB 00:00 2023-03-29T10:53:53.240 INFO:teuthology.orchestra.run.smithi111.stdout:ceph packages for x86_64 124 kB/s | 80 kB 00:00 2023-03-29T10:53:53.780 INFO:teuthology.orchestra.run.smithi119.stdout:ceph noarch packages 18 kB/s | 9.4 kB 00:00 2023-03-29T10:53:53.799 INFO:teuthology.orchestra.run.smithi111.stdout:ceph noarch packages 18 kB/s | 9.4 kB 00:00 2023-03-29T10:53:54.348 INFO:teuthology.orchestra.run.smithi119.stdout:ceph source packages 2.8 kB/s | 1.5 kB 00:00 2023-03-29T10:53:54.366 INFO:teuthology.orchestra.run.smithi111.stdout:ceph source packages 2.8 kB/s | 1.5 kB 00:00 2023-03-29T10:53:55.100 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-8 - AppStream 39 MB/s | 29 MB 00:00 2023-03-29T10:53:55.236 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS-8 - AppStream 34 MB/s | 29 MB 00:00 2023-03-29T10:54:01.342 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-8 - BaseOS 37 MB/s | 28 MB 00:00 2023-03-29T10:54:01.461 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS-8 - BaseOS 36 MB/s | 28 MB 00:00 2023-03-29T10:54:06.299 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-8 - Extras 40 kB/s | 18 kB 00:00 2023-03-29T10:54:07.560 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS-8 - Extras 11 kB/s | 18 kB 00:01 2023-03-29T10:54:09.102 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-8 - PowerTools 2.2 MB/s | 5.8 MB 00:02 2023-03-29T10:54:09.579 INFO:teuthology.orchestra.run.smithi111.stdout:CentOS-8 - PowerTools 3.1 MB/s | 5.8 MB 00:01 2023-03-29T10:54:10.458 INFO:teuthology.orchestra.run.smithi119.stdout:Copr repo for python3-asyncssh owned by ceph 21 kB/s | 3.5 kB 00:00 2023-03-29T10:54:11.144 INFO:teuthology.orchestra.run.smithi111.stdout:Copr repo for python3-asyncssh owned by ceph 9.5 kB/s | 3.5 kB 00:00 2023-03-29T10:54:11.843 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 11 MB/s | 14 MB 00:01 2023-03-29T10:54:13.486 INFO:teuthology.orchestra.run.smithi111.stdout:Extra Packages for Enterprise Linux 6.3 MB/s | 14 MB 00:02 2023-03-29T10:54:15.851 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 188 kB/s | 24 kB 00:00 2023-03-29T10:54:17.461 INFO:teuthology.orchestra.run.smithi111.stdout:lab-extras 176 kB/s | 24 kB 00:00 2023-03-29T10:54:19.474 INFO:teuthology.orchestra.run.smithi119.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-03-29T10:54:19.475 INFO:teuthology.orchestra.run.smithi119.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-03-29T10:54:19.487 INFO:teuthology.orchestra.run.smithi119.stdout:Package NetworkManager-1:1.39.12-1.el8.x86_64 is already installed. 2023-03-29T10:54:19.487 INFO:teuthology.orchestra.run.smithi119.stdout:Package NetworkManager-initscripts-updown-1:1.39.12-1.el8.noarch is already installed. 2023-03-29T10:54:19.488 INFO:teuthology.orchestra.run.smithi119.stdout:Package iproute-5.18.0-1.el8.x86_64 is already installed. 2023-03-29T10:54:19.489 INFO:teuthology.orchestra.run.smithi119.stdout:Package util-linux-2.32.1-38.el8.x86_64 is already installed. 2023-03-29T10:54:19.490 INFO:teuthology.orchestra.run.smithi119.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-03-29T10:54:19.492 INFO:teuthology.orchestra.run.smithi119.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-03-29T10:54:19.495 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-03-29T10:54:19.496 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-03-29T10:54:19.635 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-03-29T10:54:19.649 INFO:teuthology.orchestra.run.smithi119.stdout:========================================================================================================= 2023-03-29T10:54:19.649 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-03-29T10:54:19.649 INFO:teuthology.orchestra.run.smithi119.stdout:========================================================================================================= 2023-03-29T10:54:19.649 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-03-29T10:54:19.649 INFO:teuthology.orchestra.run.smithi119.stdout: bison x86_64 3.0.4-10.el8 CentOS-AppStream 688 k 2023-03-29T10:54:19.649 INFO:teuthology.orchestra.run.smithi119.stdout: ceph x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 6.4 k 2023-03-29T10:54:19.649 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 5.4 M 2023-03-29T10:54:19.649 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 848 k 2023-03-29T10:54:19.649 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 165 k 2023-03-29T10:54:19.649 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 1.5 M 2023-03-29T10:54:19.649 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 123 k 2023-03-29T10:54:19.650 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 1.7 M 2023-03-29T10:54:19.650 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 7.4 M 2023-03-29T10:54:19.650 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 47 k 2023-03-29T10:54:19.650 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 13 M 2023-03-29T10:54:19.651 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 41 M 2023-03-29T10:54:19.651 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 263 k 2023-03-29T10:54:19.651 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 83 k 2023-03-29T10:54:19.651 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 258 k 2023-03-29T10:54:19.651 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 18 k 2023-03-29T10:54:19.651 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel x86_64 0.188-3.el8 CentOS-BaseOS 61 k 2023-03-29T10:54:19.651 INFO:teuthology.orchestra.run.smithi119.stdout: flex x86_64 2.6.1-9.el8 CentOS-AppStream 320 k 2023-03-29T10:54:19.651 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 19 k 2023-03-29T10:54:19.651 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 30 k 2023-03-29T10:54:19.651 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2 x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 769 k 2023-03-29T10:54:19.651 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 139 k 2023-03-29T10:54:19.651 INFO:teuthology.orchestra.run.smithi119.stdout: libtool x86_64 2.4.6-25.el8 CentOS-AppStream 709 k 2023-03-29T10:54:19.651 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel x86_64 2.32.1-41.el8 CentOS-BaseOS 99 k 2023-03-29T10:54:19.651 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 CentOS-BaseOS 3.2 M 2023-03-29T10:54:19.652 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql x86_64 10.23-1.module_el8+300+2c53004c CentOS-AppStream 1.6 M 2023-03-29T10:54:19.652 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib x86_64 10.23-1.module_el8+300+2c53004c CentOS-AppStream 903 k 2023-03-29T10:54:19.652 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server x86_64 10.23-1.module_el8+300+2c53004c CentOS-AppStream 5.1 M 2023-03-29T10:54:19.652 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 190 k 2023-03-29T10:54:19.652 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 369 k 2023-03-29T10:54:19.652 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 365 k 2023-03-29T10:54:19.652 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 107 k 2023-03-29T10:54:19.652 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 95 k 2023-03-29T10:54:19.652 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 3.5 M 2023-03-29T10:54:19.652 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 180 k 2023-03-29T10:54:19.652 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 CentOS-BaseOS 72 k 2023-03-29T10:54:19.652 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2023-03-29T10:54:19.652 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-1.el8 CentOS-BaseOS 142 k 2023-03-29T10:54:19.652 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-debuginfod-client x86_64 0.188-3.el8 CentOS-BaseOS 75 k 2023-03-29T10:54:19.652 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf x86_64 0.188-3.el8 CentOS-BaseOS 231 k 2023-03-29T10:54:19.653 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libs x86_64 0.188-3.el8 CentOS-BaseOS 299 k 2023-03-29T10:54:19.653 INFO:teuthology.orchestra.run.smithi119.stdout: libblkid x86_64 2.32.1-41.el8 CentOS-BaseOS 220 k 2023-03-29T10:54:19.653 INFO:teuthology.orchestra.run.smithi119.stdout: libfdisk x86_64 2.32.1-41.el8 CentOS-BaseOS 253 k 2023-03-29T10:54:19.653 INFO:teuthology.orchestra.run.smithi119.stdout: libmount x86_64 2.32.1-41.el8 CentOS-BaseOS 236 k 2023-03-29T10:54:19.653 INFO:teuthology.orchestra.run.smithi119.stdout: librados2 x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 3.7 M 2023-03-29T10:54:19.653 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1 x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 3.6 M 2023-03-29T10:54:19.653 INFO:teuthology.orchestra.run.smithi119.stdout: libsmartcols x86_64 2.32.1-41.el8 CentOS-BaseOS 179 k 2023-03-29T10:54:19.653 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid x86_64 2.32.1-41.el8 CentOS-BaseOS 98 k 2023-03-29T10:54:19.653 INFO:teuthology.orchestra.run.smithi119.stdout: openssl x86_64 1:1.1.1k-9.el8 CentOS-BaseOS 737 k 2023-03-29T10:54:19.653 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs x86_64 1:1.1.1k-9.el8 CentOS-BaseOS 1.5 M 2023-03-29T10:54:19.653 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux x86_64 2.32.1-41.el8 CentOS-BaseOS 2.6 M 2023-03-29T10:54:19.653 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-user x86_64 2.32.1-41.el8 CentOS-BaseOS 102 k 2023-03-29T10:54:19.653 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2023-03-29T10:54:19.653 INFO:teuthology.orchestra.run.smithi119.stdout: automake noarch 1.16.1-7.el8 CentOS-AppStream 713 k 2023-03-29T10:54:19.653 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 22 M 2023-03-29T10:54:19.654 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 23 k 2023-03-29T10:54:19.654 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 2.3 M 2023-03-29T10:54:19.654 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 239 k 2023-03-29T10:54:19.654 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 4.6 M 2023-03-29T10:54:19.654 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 18 M 2023-03-29T10:54:19.654 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 15 k 2023-03-29T10:54:19.654 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 24 k 2023-03-29T10:54:19.654 INFO:teuthology.orchestra.run.smithi119.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-03-29T10:54:19.654 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-03-29T10:54:19.654 INFO:teuthology.orchestra.run.smithi119.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-03-29T10:54:19.654 INFO:teuthology.orchestra.run.smithi119.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 CentOS-BaseOS 48 k 2023-03-29T10:54:19.654 INFO:teuthology.orchestra.run.smithi119.stdout: krb5-devel x86_64 1.18.2-22.el8 CentOS-BaseOS 561 k 2023-03-29T10:54:19.654 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 137 k 2023-03-29T10:54:19.654 INFO:teuthology.orchestra.run.smithi119.stdout: libcom_err-devel x86_64 1.45.6-5.el8 CentOS-BaseOS 39 k 2023-03-29T10:54:19.654 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-03-29T10:54:19.655 INFO:teuthology.orchestra.run.smithi119.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-03-29T10:54:19.655 INFO:teuthology.orchestra.run.smithi119.stdout: libpq x86_64 13.5-1.el8 CentOS-AppStream 198 k 2023-03-29T10:54:19.655 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-03-29T10:54:19.655 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-03-29T10:54:19.655 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 474 k 2023-03-29T10:54:19.655 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-03-29T10:54:19.655 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2 x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 6.5 M 2023-03-29T10:54:19.655 INFO:teuthology.orchestra.run.smithi119.stdout: libselinux-devel x86_64 2.9-6.el8 CentOS-BaseOS 200 k 2023-03-29T10:54:19.655 INFO:teuthology.orchestra.run.smithi119.stdout: libsepol-devel x86_64 2.9-3.el8 CentOS-BaseOS 87 k 2023-03-29T10:54:19.655 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-03-29T10:54:19.655 INFO:teuthology.orchestra.run.smithi119.stdout: libverto-devel x86_64 0.3.2-2.el8 CentOS-BaseOS 18 k 2023-03-29T10:54:19.655 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-03-29T10:54:19.655 INFO:teuthology.orchestra.run.smithi119.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-03-29T10:54:19.655 INFO:teuthology.orchestra.run.smithi119.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-03-29T10:54:19.655 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-03-29T10:54:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: pcre2-devel x86_64 10.32-3.el8 CentOS-BaseOS 605 k 2023-03-29T10:54:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: pcre2-utf16 x86_64 10.32-3.el8 CentOS-BaseOS 229 k 2023-03-29T10:54:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: pcre2-utf32 x86_64 10.32-3.el8 CentOS-BaseOS 220 k 2023-03-29T10:54:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-03-29T10:54:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-03-29T10:54:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-03-29T10:54:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-03-29T10:54:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-03-29T10:54:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 45 k 2023-03-29T10:54:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 117 k 2023-03-29T10:54:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-03-29T10:54:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-03-29T10:54:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-03-29T10:54:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-03-29T10:54:19.656 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-03-29T10:54:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-03-29T10:54:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-03-29T10:54:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-03-29T10:54:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-03-29T10:54:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-03-29T10:54:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-03-29T10:54:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-03-29T10:54:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-03-29T10:54:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-03-29T10:54:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako noarch 1.0.6-14.el8 CentOS-AppStream 157 k 2023-03-29T10:54:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-03-29T10:54:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-03-29T10:54:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-03-29T10:54:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-03-29T10:54:19.657 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-03-29T10:54:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-03-29T10:54:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-03-29T10:54:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-03-29T10:54:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-03-29T10:54:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-03-29T10:54:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-03-29T10:54:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-03-29T10:54:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-03-29T10:54:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-03-29T10:54:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-03-29T10:54:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-03-29T10:54:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-03-29T10:54:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-03-29T10:54:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-03-29T10:54:19.658 INFO:teuthology.orchestra.run.smithi119.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-03-29T10:54:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-03-29T10:54:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-03-29T10:54:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-03-29T10:54:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-03-29T10:54:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-03-29T10:54:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-03-29T10:54:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-03-29T10:54:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-03-29T10:54:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-03-29T10:54:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-03-29T10:54:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-03-29T10:54:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: uuid x86_64 1.6.2-43.el8 CentOS-AppStream 64 k 2023-03-29T10:54:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-03-29T10:54:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel x86_64 1.2.11-19.el8 CentOS-BaseOS 58 k 2023-03-29T10:54:19.659 INFO:teuthology.orchestra.run.smithi119.stdout:Enabling module streams: 2023-03-29T10:54:19.659 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql 10 2023-03-29T10:54:19.660 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:54:19.660 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-03-29T10:54:19.660 INFO:teuthology.orchestra.run.smithi119.stdout:========================================================================================================= 2023-03-29T10:54:19.660 INFO:teuthology.orchestra.run.smithi119.stdout:Install 127 Packages 2023-03-29T10:54:19.660 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 15 Packages 2023-03-29T10:54:19.660 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:54:19.660 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 204 M 2023-03-29T10:54:19.661 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-03-29T10:54:19.794 INFO:teuthology.orchestra.run.smithi119.stdout:(1/142): ceph-18.0.0-3066.gf03d69a8.el8.x86_64. 52 kB/s | 6.4 kB 00:00 2023-03-29T10:54:20.011 INFO:teuthology.orchestra.run.smithi119.stdout:(2/142): ceph-fuse-18.0.0-3066.gf03d69a8.el8.x8 3.8 MB/s | 848 kB 00:00 2023-03-29T10:54:20.086 INFO:teuthology.orchestra.run.smithi119.stdout:(3/142): ceph-immutable-object-cache-18.0.0-306 2.2 MB/s | 165 kB 00:00 2023-03-29T10:54:20.411 INFO:teuthology.orchestra.run.smithi119.stdout:(4/142): ceph-mds-18.0.0-3066.gf03d69a8.el8.x86 7.0 MB/s | 2.3 MB 00:00 2023-03-29T10:54:20.520 INFO:teuthology.orchestra.run.smithi119.stdout:(5/142): ceph-base-18.0.0-3066.gf03d69a8.el8.x8 6.3 MB/s | 5.4 MB 00:00 2023-03-29T10:54:20.679 INFO:teuthology.orchestra.run.smithi119.stdout:(6/142): ceph-mgr-18.0.0-3066.gf03d69a8.el8.x86 5.8 MB/s | 1.5 MB 00:00 2023-03-29T10:54:21.070 INFO:teuthology.orchestra.run.smithi119.stdout:(7/142): ceph-mon-18.0.0-3066.gf03d69a8.el8.x86 8.4 MB/s | 4.6 MB 00:00 2023-03-29T10:54:21.076 INFO:teuthology.orchestra.run.smithi111.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-03-29T10:54:21.077 INFO:teuthology.orchestra.run.smithi111.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-03-29T10:54:21.089 INFO:teuthology.orchestra.run.smithi111.stdout:Package NetworkManager-1:1.39.12-1.el8.x86_64 is already installed. 2023-03-29T10:54:21.089 INFO:teuthology.orchestra.run.smithi111.stdout:Package NetworkManager-initscripts-updown-1:1.39.12-1.el8.noarch is already installed. 2023-03-29T10:54:21.090 INFO:teuthology.orchestra.run.smithi111.stdout:Package iproute-5.18.0-1.el8.x86_64 is already installed. 2023-03-29T10:54:21.091 INFO:teuthology.orchestra.run.smithi111.stdout:Package util-linux-2.32.1-38.el8.x86_64 is already installed. 2023-03-29T10:54:21.092 INFO:teuthology.orchestra.run.smithi111.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-03-29T10:54:21.094 INFO:teuthology.orchestra.run.smithi111.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-03-29T10:54:21.097 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-03-29T10:54:21.098 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-03-29T10:54:21.237 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2023-03-29T10:54:21.250 INFO:teuthology.orchestra.run.smithi111.stdout:========================================================================================================= 2023-03-29T10:54:21.251 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2023-03-29T10:54:21.251 INFO:teuthology.orchestra.run.smithi111.stdout:========================================================================================================= 2023-03-29T10:54:21.251 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2023-03-29T10:54:21.251 INFO:teuthology.orchestra.run.smithi111.stdout: bison x86_64 3.0.4-10.el8 CentOS-AppStream 688 k 2023-03-29T10:54:21.251 INFO:teuthology.orchestra.run.smithi111.stdout: ceph x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 6.4 k 2023-03-29T10:54:21.251 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 5.4 M 2023-03-29T10:54:21.251 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 848 k 2023-03-29T10:54:21.251 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 165 k 2023-03-29T10:54:21.251 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 1.5 M 2023-03-29T10:54:21.251 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 123 k 2023-03-29T10:54:21.251 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 1.7 M 2023-03-29T10:54:21.251 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 7.4 M 2023-03-29T10:54:21.251 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 47 k 2023-03-29T10:54:21.251 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 13 M 2023-03-29T10:54:21.251 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 41 M 2023-03-29T10:54:21.252 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 263 k 2023-03-29T10:54:21.252 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 83 k 2023-03-29T10:54:21.252 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 258 k 2023-03-29T10:54:21.252 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 18 k 2023-03-29T10:54:21.252 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel x86_64 0.188-3.el8 CentOS-BaseOS 61 k 2023-03-29T10:54:21.252 INFO:teuthology.orchestra.run.smithi111.stdout: flex x86_64 2.6.1-9.el8 CentOS-AppStream 320 k 2023-03-29T10:54:21.252 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 19 k 2023-03-29T10:54:21.252 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 30 k 2023-03-29T10:54:21.252 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2 x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 769 k 2023-03-29T10:54:21.252 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 139 k 2023-03-29T10:54:21.252 INFO:teuthology.orchestra.run.smithi111.stdout: libtool x86_64 2.4.6-25.el8 CentOS-AppStream 709 k 2023-03-29T10:54:21.252 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel x86_64 2.32.1-41.el8 CentOS-BaseOS 99 k 2023-03-29T10:54:21.252 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 CentOS-BaseOS 3.2 M 2023-03-29T10:54:21.252 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql x86_64 10.23-1.module_el8+300+2c53004c CentOS-AppStream 1.6 M 2023-03-29T10:54:21.252 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib x86_64 10.23-1.module_el8+300+2c53004c CentOS-AppStream 903 k 2023-03-29T10:54:21.253 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server x86_64 10.23-1.module_el8+300+2c53004c CentOS-AppStream 5.1 M 2023-03-29T10:54:21.253 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 190 k 2023-03-29T10:54:21.253 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 369 k 2023-03-29T10:54:21.253 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 365 k 2023-03-29T10:54:21.253 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 107 k 2023-03-29T10:54:21.253 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 95 k 2023-03-29T10:54:21.253 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 3.5 M 2023-03-29T10:54:21.253 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 180 k 2023-03-29T10:54:21.253 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 CentOS-BaseOS 72 k 2023-03-29T10:54:21.253 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrading: 2023-03-29T10:54:21.253 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-1.el8 CentOS-BaseOS 142 k 2023-03-29T10:54:21.253 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-debuginfod-client x86_64 0.188-3.el8 CentOS-BaseOS 75 k 2023-03-29T10:54:21.253 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf x86_64 0.188-3.el8 CentOS-BaseOS 231 k 2023-03-29T10:54:21.254 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libs x86_64 0.188-3.el8 CentOS-BaseOS 299 k 2023-03-29T10:54:21.254 INFO:teuthology.orchestra.run.smithi111.stdout: libblkid x86_64 2.32.1-41.el8 CentOS-BaseOS 220 k 2023-03-29T10:54:21.254 INFO:teuthology.orchestra.run.smithi111.stdout: libfdisk x86_64 2.32.1-41.el8 CentOS-BaseOS 253 k 2023-03-29T10:54:21.254 INFO:teuthology.orchestra.run.smithi111.stdout: libmount x86_64 2.32.1-41.el8 CentOS-BaseOS 236 k 2023-03-29T10:54:21.254 INFO:teuthology.orchestra.run.smithi111.stdout: librados2 x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 3.7 M 2023-03-29T10:54:21.254 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1 x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 3.6 M 2023-03-29T10:54:21.254 INFO:teuthology.orchestra.run.smithi111.stdout: libsmartcols x86_64 2.32.1-41.el8 CentOS-BaseOS 179 k 2023-03-29T10:54:21.254 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid x86_64 2.32.1-41.el8 CentOS-BaseOS 98 k 2023-03-29T10:54:21.254 INFO:teuthology.orchestra.run.smithi111.stdout: openssl x86_64 1:1.1.1k-9.el8 CentOS-BaseOS 737 k 2023-03-29T10:54:21.254 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs x86_64 1:1.1.1k-9.el8 CentOS-BaseOS 1.5 M 2023-03-29T10:54:21.254 INFO:teuthology.orchestra.run.smithi111.stdout: util-linux x86_64 2.32.1-41.el8 CentOS-BaseOS 2.6 M 2023-03-29T10:54:21.254 INFO:teuthology.orchestra.run.smithi111.stdout: util-linux-user x86_64 2.32.1-41.el8 CentOS-BaseOS 102 k 2023-03-29T10:54:21.254 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2023-03-29T10:54:21.254 INFO:teuthology.orchestra.run.smithi111.stdout: automake noarch 1.16.1-7.el8 CentOS-AppStream 713 k 2023-03-29T10:54:21.254 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 22 M 2023-03-29T10:54:21.254 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 23 k 2023-03-29T10:54:21.254 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 2.3 M 2023-03-29T10:54:21.255 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 239 k 2023-03-29T10:54:21.255 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 4.6 M 2023-03-29T10:54:21.255 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 18 M 2023-03-29T10:54:21.255 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3066.gf03d69a8.el8 ceph-noarch 15 k 2023-03-29T10:54:21.255 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 24 k 2023-03-29T10:54:21.255 INFO:teuthology.orchestra.run.smithi111.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-03-29T10:54:21.255 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-03-29T10:54:21.255 INFO:teuthology.orchestra.run.smithi111.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-03-29T10:54:21.255 INFO:teuthology.orchestra.run.smithi111.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 CentOS-BaseOS 48 k 2023-03-29T10:54:21.255 INFO:teuthology.orchestra.run.smithi111.stdout: krb5-devel x86_64 1.18.2-22.el8 CentOS-BaseOS 561 k 2023-03-29T10:54:21.255 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 137 k 2023-03-29T10:54:21.255 INFO:teuthology.orchestra.run.smithi111.stdout: libcom_err-devel x86_64 1.45.6-5.el8 CentOS-BaseOS 39 k 2023-03-29T10:54:21.255 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-03-29T10:54:21.255 INFO:teuthology.orchestra.run.smithi111.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-03-29T10:54:21.255 INFO:teuthology.orchestra.run.smithi111.stdout: libpq x86_64 13.5-1.el8 CentOS-AppStream 198 k 2023-03-29T10:54:21.255 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-03-29T10:54:21.255 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-03-29T10:54:21.255 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1 x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 474 k 2023-03-29T10:54:21.256 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-03-29T10:54:21.256 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2 x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 6.5 M 2023-03-29T10:54:21.256 INFO:teuthology.orchestra.run.smithi111.stdout: libselinux-devel x86_64 2.9-6.el8 CentOS-BaseOS 200 k 2023-03-29T10:54:21.256 INFO:teuthology.orchestra.run.smithi111.stdout: libsepol-devel x86_64 2.9-3.el8 CentOS-BaseOS 87 k 2023-03-29T10:54:21.256 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-03-29T10:54:21.256 INFO:teuthology.orchestra.run.smithi111.stdout: libverto-devel x86_64 0.3.2-2.el8 CentOS-BaseOS 18 k 2023-03-29T10:54:21.256 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-03-29T10:54:21.256 INFO:teuthology.orchestra.run.smithi111.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-03-29T10:54:21.256 INFO:teuthology.orchestra.run.smithi111.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-03-29T10:54:21.256 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-03-29T10:54:21.256 INFO:teuthology.orchestra.run.smithi111.stdout: pcre2-devel x86_64 10.32-3.el8 CentOS-BaseOS 605 k 2023-03-29T10:54:21.256 INFO:teuthology.orchestra.run.smithi111.stdout: pcre2-utf16 x86_64 10.32-3.el8 CentOS-BaseOS 229 k 2023-03-29T10:54:21.256 INFO:teuthology.orchestra.run.smithi111.stdout: pcre2-utf32 x86_64 10.32-3.el8 CentOS-BaseOS 220 k 2023-03-29T10:54:21.256 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-03-29T10:54:21.256 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-03-29T10:54:21.256 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-03-29T10:54:21.257 INFO:teuthology.orchestra.run.smithi111.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-03-29T10:54:21.257 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-03-29T10:54:21.257 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 45 k 2023-03-29T10:54:21.257 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common x86_64 2:18.0.0-3066.gf03d69a8.el8 ceph 117 k 2023-03-29T10:54:21.257 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-03-29T10:54:21.257 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-03-29T10:54:21.257 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-03-29T10:54:21.257 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-03-29T10:54:21.257 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-03-29T10:54:21.257 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-03-29T10:54:21.257 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-03-29T10:54:21.257 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-03-29T10:54:21.257 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-03-29T10:54:21.257 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-03-29T10:54:21.257 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-03-29T10:54:21.257 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-03-29T10:54:21.257 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-03-29T10:54:21.258 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-03-29T10:54:21.258 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako noarch 1.0.6-14.el8 CentOS-AppStream 157 k 2023-03-29T10:54:21.258 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-03-29T10:54:21.258 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-03-29T10:54:21.258 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-03-29T10:54:21.258 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-03-29T10:54:21.258 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-03-29T10:54:21.258 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-03-29T10:54:21.258 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-03-29T10:54:21.258 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-03-29T10:54:21.258 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-03-29T10:54:21.258 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-03-29T10:54:21.258 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-03-29T10:54:21.258 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-03-29T10:54:21.258 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-03-29T10:54:21.258 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-03-29T10:54:21.258 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-03-29T10:54:21.259 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-03-29T10:54:21.259 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-03-29T10:54:21.259 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-03-29T10:54:21.259 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-03-29T10:54:21.259 INFO:teuthology.orchestra.run.smithi111.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-03-29T10:54:21.259 INFO:teuthology.orchestra.run.smithi111.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-03-29T10:54:21.259 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-03-29T10:54:21.259 INFO:teuthology.orchestra.run.smithi111.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-03-29T10:54:21.259 INFO:teuthology.orchestra.run.smithi111.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-03-29T10:54:21.259 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-03-29T10:54:21.259 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-03-29T10:54:21.259 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-03-29T10:54:21.259 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-03-29T10:54:21.259 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-03-29T10:54:21.259 INFO:teuthology.orchestra.run.smithi111.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-03-29T10:54:21.259 INFO:teuthology.orchestra.run.smithi111.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-03-29T10:54:21.259 INFO:teuthology.orchestra.run.smithi111.stdout: uuid x86_64 1.6.2-43.el8 CentOS-AppStream 64 k 2023-03-29T10:54:21.259 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-03-29T10:54:21.260 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel x86_64 1.2.11-19.el8 CentOS-BaseOS 58 k 2023-03-29T10:54:21.260 INFO:teuthology.orchestra.run.smithi111.stdout:Enabling module streams: 2023-03-29T10:54:21.260 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql 10 2023-03-29T10:54:21.260 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:54:21.260 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2023-03-29T10:54:21.260 INFO:teuthology.orchestra.run.smithi111.stdout:========================================================================================================= 2023-03-29T10:54:21.260 INFO:teuthology.orchestra.run.smithi111.stdout:Install 127 Packages 2023-03-29T10:54:21.260 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrade 15 Packages 2023-03-29T10:54:21.260 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:54:21.260 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 204 M 2023-03-29T10:54:21.261 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2023-03-29T10:54:21.621 INFO:teuthology.orchestra.run.smithi111.stdout:(1/142): ceph-18.0.0-3066.gf03d69a8.el8.x86_64. 18 kB/s | 6.4 kB 00:00 2023-03-29T10:54:22.046 INFO:teuthology.orchestra.run.smithi119.stdout:(8/142): ceph-common-18.0.0-3066.gf03d69a8.el8. 9.1 MB/s | 22 MB 00:02 2023-03-29T10:54:22.063 INFO:teuthology.orchestra.run.smithi111.stdout:(2/142): ceph-fuse-18.0.0-3066.gf03d69a8.el8.x8 1.9 MB/s | 848 kB 00:00 2023-03-29T10:54:22.113 INFO:teuthology.orchestra.run.smithi119.stdout:(9/142): ceph-selinux-18.0.0-3066.gf03d69a8.el8 367 kB/s | 24 kB 00:00 2023-03-29T10:54:22.180 INFO:teuthology.orchestra.run.smithi111.stdout:(3/142): ceph-immutable-object-cache-18.0.0-306 1.4 MB/s | 165 kB 00:00 2023-03-29T10:54:22.821 INFO:teuthology.orchestra.run.smithi119.stdout:(10/142): ceph-radosgw-18.0.0-3066.gf03d69a8.el 7.5 MB/s | 13 MB 00:01 2023-03-29T10:54:22.905 INFO:teuthology.orchestra.run.smithi119.stdout:(11/142): cephfs-mirror-18.0.0-3066.gf03d69a8.e 3.0 MB/s | 258 kB 00:00 2023-03-29T10:54:22.972 INFO:teuthology.orchestra.run.smithi119.stdout:(12/142): libcephfs-devel-18.0.0-3066.gf03d69a8 458 kB/s | 30 kB 00:00 2023-03-29T10:54:23.138 INFO:teuthology.orchestra.run.smithi111.stdout:(4/142): ceph-mds-18.0.0-3066.gf03d69a8.el8.x86 2.4 MB/s | 2.3 MB 00:00 2023-03-29T10:54:23.247 INFO:teuthology.orchestra.run.smithi119.stdout:(13/142): ceph-osd-18.0.0-3066.gf03d69a8.el8.x8 6.9 MB/s | 18 MB 00:02 2023-03-29T10:54:23.280 INFO:teuthology.orchestra.run.smithi119.stdout:(14/142): libcephfs2-18.0.0-3066.gf03d69a8.el8. 2.4 MB/s | 769 kB 00:00 2023-03-29T10:54:23.305 INFO:teuthology.orchestra.run.smithi111.stdout:(5/142): ceph-base-18.0.0-3066.gf03d69a8.el8.x8 2.6 MB/s | 5.4 MB 00:02 2023-03-29T10:54:23.306 INFO:teuthology.orchestra.run.smithi119.stdout:(15/142): libcephsqlite-18.0.0-3066.gf03d69a8.e 2.3 MB/s | 137 kB 00:00 2023-03-29T10:54:23.340 INFO:teuthology.orchestra.run.smithi119.stdout:(16/142): librados-devel-18.0.0-3066.gf03d69a8. 2.3 MB/s | 139 kB 00:00 2023-03-29T10:54:23.440 INFO:teuthology.orchestra.run.smithi119.stdout:(17/142): libradosstriper1-18.0.0-3066.gf03d69a 3.5 MB/s | 474 kB 00:00 2023-03-29T10:54:23.515 INFO:teuthology.orchestra.run.smithi119.stdout:(18/142): python3-ceph-argparse-18.0.0-3066.gf0 595 kB/s | 45 kB 00:00 2023-03-29T10:54:23.599 INFO:teuthology.orchestra.run.smithi119.stdout:(19/142): python3-ceph-common-18.0.0-3066.gf03d 1.4 MB/s | 117 kB 00:00 2023-03-29T10:54:23.622 INFO:teuthology.orchestra.run.smithi111.stdout:(6/142): ceph-mgr-18.0.0-3066.gf03d69a8.el8.x86 3.2 MB/s | 1.5 MB 00:00 2023-03-29T10:54:23.674 INFO:teuthology.orchestra.run.smithi119.stdout:(20/142): python3-cephfs-18.0.0-3066.gf03d69a8. 2.5 MB/s | 190 kB 00:00 2023-03-29T10:54:23.766 INFO:teuthology.orchestra.run.smithi119.stdout:(21/142): python3-rados-18.0.0-3066.gf03d69a8.e 3.9 MB/s | 369 kB 00:00 2023-03-29T10:54:23.858 INFO:teuthology.orchestra.run.smithi119.stdout:(22/142): python3-rbd-18.0.0-3066.gf03d69a8.el8 3.9 MB/s | 365 kB 00:00 2023-03-29T10:54:23.950 INFO:teuthology.orchestra.run.smithi119.stdout:(23/142): python3-rgw-18.0.0-3066.gf03d69a8.el8 1.1 MB/s | 107 kB 00:00 2023-03-29T10:54:24.034 INFO:teuthology.orchestra.run.smithi119.stdout:(24/142): rbd-fuse-18.0.0-3066.gf03d69a8.el8.x8 1.1 MB/s | 95 kB 00:00 2023-03-29T10:54:24.367 INFO:teuthology.orchestra.run.smithi119.stdout:(25/142): librgw2-18.0.0-3066.gf03d69a8.el8.x86 6.3 MB/s | 6.5 MB 00:01 2023-03-29T10:54:24.459 INFO:teuthology.orchestra.run.smithi119.stdout:(26/142): rbd-nbd-18.0.0-3066.gf03d69a8.el8.x86 1.9 MB/s | 180 kB 00:00 2023-03-29T10:54:24.535 INFO:teuthology.orchestra.run.smithi119.stdout:(27/142): ceph-grafana-dashboards-18.0.0-3066.g 308 kB/s | 23 kB 00:00 2023-03-29T10:54:24.635 INFO:teuthology.orchestra.run.smithi119.stdout:(28/142): rbd-mirror-18.0.0-3066.gf03d69a8.el8. 5.7 MB/s | 3.5 MB 00:00 2023-03-29T10:54:24.660 INFO:teuthology.orchestra.run.smithi119.stdout:(29/142): ceph-mgr-cephadm-18.0.0-3066.gf03d69a 986 kB/s | 123 kB 00:00 2023-03-29T10:54:24.664 INFO:teuthology.orchestra.run.smithi111.stdout:(7/142): ceph-mon-18.0.0-3066.gf03d69a8.el8.x86 3.4 MB/s | 4.6 MB 00:01 2023-03-29T10:54:24.985 INFO:teuthology.orchestra.run.smithi119.stdout:(30/142): ceph-mgr-dashboard-18.0.0-3066.gf03d6 4.8 MB/s | 1.7 MB 00:00 2023-03-29T10:54:25.077 INFO:teuthology.orchestra.run.smithi119.stdout:(31/142): ceph-mgr-modules-core-18.0.0-3066.gf0 2.6 MB/s | 239 kB 00:00 2023-03-29T10:54:25.161 INFO:teuthology.orchestra.run.smithi119.stdout:(32/142): ceph-mgr-rook-18.0.0-3066.gf03d69a8.e 567 kB/s | 47 kB 00:00 2023-03-29T10:54:25.245 INFO:teuthology.orchestra.run.smithi119.stdout:(33/142): ceph-prometheus-alerts-18.0.0-3066.gf 175 kB/s | 15 kB 00:00 2023-03-29T10:54:25.345 INFO:teuthology.orchestra.run.smithi119.stdout:(34/142): ceph-volume-18.0.0-3066.gf03d69a8.el8 2.6 MB/s | 263 kB 00:00 2023-03-29T10:54:25.429 INFO:teuthology.orchestra.run.smithi119.stdout:(35/142): cephadm-18.0.0-3066.gf03d69a8.el8.noa 995 kB/s | 83 kB 00:00 2023-03-29T10:54:25.512 INFO:teuthology.orchestra.run.smithi119.stdout:(36/142): cephfs-top-18.0.0-3066.gf03d69a8.el8. 217 kB/s | 18 kB 00:00 2023-03-29T10:54:25.587 INFO:teuthology.orchestra.run.smithi119.stdout:(37/142): automake-1.16.1-7.el8.noarch.rpm 9.3 MB/s | 713 kB 00:00 2023-03-29T10:54:25.630 INFO:teuthology.orchestra.run.smithi119.stdout:(38/142): bison-3.0.4-10.el8.x86_64.rpm 16 MB/s | 688 kB 00:00 2023-03-29T10:54:25.663 INFO:teuthology.orchestra.run.smithi119.stdout:(39/142): flex-2.6.1-9.el8.x86_64.rpm 9.4 MB/s | 320 kB 00:00 2023-03-29T10:54:25.688 INFO:teuthology.orchestra.run.smithi119.stdout:(40/142): jq-1.6-6.el8.x86_64.rpm 8.0 MB/s | 202 kB 00:00 2023-03-29T10:54:25.780 INFO:teuthology.orchestra.run.smithi119.stdout:(41/142): ceph-mgr-diskprediction-local-18.0.0- 6.6 MB/s | 7.4 MB 00:01 2023-03-29T10:54:25.806 INFO:teuthology.orchestra.run.smithi119.stdout:(42/142): libpq-13.5-1.el8.x86_64.rpm 1.6 MB/s | 198 kB 00:00 2023-03-29T10:54:25.839 INFO:teuthology.orchestra.run.smithi119.stdout:(43/142): librdkafka-0.11.4-3.el8.x86_64.rpm 11 MB/s | 354 kB 00:00 2023-03-29T10:54:25.873 INFO:teuthology.orchestra.run.smithi119.stdout:(44/142): lttng-ust-2.8.1-11.el8.x86_64.rpm 8.1 MB/s | 259 kB 00:00 2023-03-29T10:54:25.907 INFO:teuthology.orchestra.run.smithi119.stdout:(45/142): oniguruma-6.8.2-2.el8.x86_64.rpm 5.5 MB/s | 187 kB 00:00 2023-03-29T10:54:25.940 INFO:teuthology.orchestra.run.smithi119.stdout:(46/142): libtool-2.4.6-25.el8.x86_64.rpm 5.2 MB/s | 709 kB 00:00 2023-03-29T10:54:26.074 INFO:teuthology.orchestra.run.smithi119.stdout:(47/142): openblas-0.3.15-4.el8.x86_64.rpm 28 MB/s | 4.7 MB 00:00 2023-03-29T10:54:26.141 INFO:teuthology.orchestra.run.smithi119.stdout:(48/142): postgresql-10.23-1.module_el8+300+2c5 23 MB/s | 1.6 MB 00:00 2023-03-29T10:54:26.517 INFO:teuthology.orchestra.run.smithi119.stdout:(49/142): ceph-test-18.0.0-3066.gf03d69a8.el8.x 9.2 MB/s | 41 MB 00:04 2023-03-29T10:54:26.592 INFO:teuthology.orchestra.run.smithi119.stdout:(50/142): openblas-threads-0.3.15-4.el8.x86_64. 7.5 MB/s | 4.9 MB 00:00 2023-03-29T10:54:26.634 INFO:teuthology.orchestra.run.smithi119.stdout:(51/142): postgresql-contrib-10.23-1.module_el8 1.8 MB/s | 903 kB 00:00 2023-03-29T10:54:26.676 INFO:teuthology.orchestra.run.smithi119.stdout:(52/142): python3-jinja2-2.10.1-3.el8.noarch.rp 13 MB/s | 538 kB 00:00 2023-03-29T10:54:26.701 INFO:teuthology.orchestra.run.smithi119.stdout:(53/142): python3-jsonpatch-1.21-2.el8.noarch.r 1.0 MB/s | 27 kB 00:00 2023-03-29T10:54:26.726 INFO:teuthology.orchestra.run.smithi119.stdout:(54/142): python3-jsonpointer-1.10-11.el8.noarc 810 kB/s | 20 kB 00:00 2023-03-29T10:54:26.760 INFO:teuthology.orchestra.run.smithi119.stdout:(55/142): python3-mako-1.0.6-14.el8.noarch.rpm 4.6 MB/s | 157 kB 00:00 2023-03-29T10:54:26.785 INFO:teuthology.orchestra.run.smithi119.stdout:(56/142): python3-markupsafe-0.23-19.el8.x86_64 1.5 MB/s | 39 kB 00:00 2023-03-29T10:54:26.927 INFO:teuthology.orchestra.run.smithi119.stdout:(57/142): python3-numpy-1.14.3-10.el8.x86_64.rp 26 MB/s | 3.7 MB 00:00 2023-03-29T10:54:26.947 INFO:teuthology.orchestra.run.smithi111.stdout:(8/142): ceph-common-18.0.0-3066.gf03d69a8.el8. 3.8 MB/s | 22 MB 00:05 2023-03-29T10:54:26.989 INFO:teuthology.orchestra.run.smithi111.stdout:(9/142): ceph-selinux-18.0.0-3066.gf03d69a8.el8 592 kB/s | 24 kB 00:00 2023-03-29T10:54:26.994 INFO:teuthology.orchestra.run.smithi119.stdout:(58/142): python3-babel-2.5.1-7.el8.noarch.rpm 12 MB/s | 4.8 MB 00:00 2023-03-29T10:54:27.020 INFO:teuthology.orchestra.run.smithi119.stdout:(59/142): python3-prettytable-0.7.2-14.el8.noar 1.7 MB/s | 44 kB 00:00 2023-03-29T10:54:27.044 INFO:teuthology.orchestra.run.smithi119.stdout:(60/142): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.9 MB/s | 225 kB 00:00 2023-03-29T10:54:27.070 INFO:teuthology.orchestra.run.smithi119.stdout:(61/142): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-03-29T10:54:27.104 INFO:teuthology.orchestra.run.smithi119.stdout:(62/142): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.1 MB/s | 126 kB 00:00 2023-03-29T10:54:27.129 INFO:teuthology.orchestra.run.smithi119.stdout:(63/142): python3-pyasn1-modules-0.3.7-6.el8.no 1.8 MB/s | 110 kB 00:00 2023-03-29T10:54:27.154 INFO:teuthology.orchestra.run.smithi119.stdout:(64/142): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-03-29T10:54:27.165 INFO:teuthology.orchestra.run.smithi111.stdout:(10/142): ceph-radosgw-18.0.0-3066.gf03d69a8.el 5.2 MB/s | 13 MB 00:02 2023-03-29T10:54:27.230 INFO:teuthology.orchestra.run.smithi119.stdout:(65/142): postgresql-server-10.23-1.module_el8+ 7.2 MB/s | 5.1 MB 00:00 2023-03-29T10:54:27.240 INFO:teuthology.orchestra.run.smithi111.stdout:(11/142): cephfs-mirror-18.0.0-3066.gf03d69a8.e 3.4 MB/s | 258 kB 00:00 2023-03-29T10:54:27.255 INFO:teuthology.orchestra.run.smithi119.stdout:(66/142): python3-werkzeug-0.12.2-4.el8.noarch. 4.4 MB/s | 457 kB 00:00 2023-03-29T10:54:27.282 INFO:teuthology.orchestra.run.smithi111.stdout:(12/142): libcephfs-devel-18.0.0-3066.gf03d69a8 738 kB/s | 30 kB 00:00 2023-03-29T10:54:27.288 INFO:teuthology.orchestra.run.smithi119.stdout:(67/142): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-03-29T10:54:27.305 INFO:teuthology.orchestra.run.smithi119.stdout:(68/142): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-03-29T10:54:27.331 INFO:teuthology.orchestra.run.smithi119.stdout:(69/142): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.6 MB/s | 71 kB 00:00 2023-03-29T10:54:27.347 INFO:teuthology.orchestra.run.smithi119.stdout:(70/142): elfutils-libelf-devel-0.188-3.el8.x86 1.4 MB/s | 61 kB 00:00 2023-03-29T10:54:27.364 INFO:teuthology.orchestra.run.smithi119.stdout:(71/142): keyutils-libs-devel-1.5.10-9.el8.x86_ 2.8 MB/s | 48 kB 00:00 2023-03-29T10:54:27.390 INFO:teuthology.orchestra.run.smithi119.stdout:(72/142): fuse-2.9.7-16.el8.x86_64.rpm 1.4 MB/s | 83 kB 00:00 2023-03-29T10:54:27.399 INFO:teuthology.orchestra.run.smithi111.stdout:(13/142): libcephfs2-18.0.0-3066.gf03d69a8.el8. 6.5 MB/s | 769 kB 00:00 2023-03-29T10:54:27.407 INFO:teuthology.orchestra.run.smithi119.stdout:(73/142): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.1 MB/s | 19 kB 00:00 2023-03-29T10:54:27.424 INFO:teuthology.orchestra.run.smithi119.stdout:(74/142): libcom_err-devel-1.45.6-5.el8.x86_64. 2.3 MB/s | 39 kB 00:00 2023-03-29T10:54:27.449 INFO:teuthology.orchestra.run.smithi119.stdout:(75/142): krb5-devel-1.18.2-22.el8.x86_64.rpm 6.5 MB/s | 561 kB 00:00 2023-03-29T10:54:27.457 INFO:teuthology.orchestra.run.smithi111.stdout:(14/142): libcephsqlite-18.0.0-3066.gf03d69a8.e 2.3 MB/s | 137 kB 00:00 2023-03-29T10:54:27.483 INFO:teuthology.orchestra.run.smithi119.stdout:(76/142): libquadmath-8.5.0-18.el8.x86_64.rpm 5.1 MB/s | 172 kB 00:00 2023-03-29T10:54:27.500 INFO:teuthology.orchestra.run.smithi119.stdout:(77/142): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.8 MB/s | 47 kB 00:00 2023-03-29T10:54:27.507 INFO:teuthology.orchestra.run.smithi111.stdout:(15/142): librados-devel-18.0.0-3066.gf03d69a8. 2.7 MB/s | 139 kB 00:00 2023-03-29T10:54:27.534 INFO:teuthology.orchestra.run.smithi119.stdout:(78/142): libselinux-devel-2.9-6.el8.x86_64.rpm 5.9 MB/s | 200 kB 00:00 2023-03-29T10:54:27.559 INFO:teuthology.orchestra.run.smithi119.stdout:(79/142): libsepol-devel-2.9-3.el8.x86_64.rpm 3.4 MB/s | 87 kB 00:00 2023-03-29T10:54:27.584 INFO:teuthology.orchestra.run.smithi119.stdout:(80/142): libuuid-devel-2.32.1-41.el8.x86_64.rp 3.9 MB/s | 99 kB 00:00 2023-03-29T10:54:27.601 INFO:teuthology.orchestra.run.smithi119.stdout:(81/142): libverto-devel-0.3.2-2.el8.x86_64.rpm 1.1 MB/s | 18 kB 00:00 2023-03-29T10:54:27.651 INFO:teuthology.orchestra.run.smithi119.stdout:(82/142): libgfortran-8.5.0-18.el8.x86_64.rpm 2.8 MB/s | 645 kB 00:00 2023-03-29T10:54:27.708 INFO:teuthology.orchestra.run.smithi111.stdout:(16/142): ceph-osd-18.0.0-3066.gf03d69a8.el8.x8 4.3 MB/s | 18 MB 00:04 2023-03-29T10:54:27.741 INFO:teuthology.orchestra.run.smithi111.stdout:(17/142): libradosstriper1-18.0.0-3066.gf03d69a 2.0 MB/s | 474 kB 00:00 2023-03-29T10:54:27.742 INFO:teuthology.orchestra.run.smithi119.stdout:(83/142): pcre2-devel-10.32-3.el8.x86_64.rpm 6.6 MB/s | 605 kB 00:00 2023-03-29T10:54:27.783 INFO:teuthology.orchestra.run.smithi111.stdout:(18/142): python3-ceph-argparse-18.0.0-3066.gf0 1.0 MB/s | 45 kB 00:00 2023-03-29T10:54:27.810 INFO:teuthology.orchestra.run.smithi119.stdout:(84/142): openssl-devel-1.1.1k-9.el8.x86_64.rpm 15 MB/s | 3.2 MB 00:00 2023-03-29T10:54:27.833 INFO:teuthology.orchestra.run.smithi111.stdout:(19/142): python3-ceph-common-18.0.0-3066.gf03d 2.3 MB/s | 117 kB 00:00 2023-03-29T10:54:27.836 INFO:teuthology.orchestra.run.smithi119.stdout:(85/142): pcre2-utf16-10.32-3.el8.x86_64.rpm 2.4 MB/s | 229 kB 00:00 2023-03-29T10:54:27.861 INFO:teuthology.orchestra.run.smithi119.stdout:(86/142): pcre2-utf32-10.32-3.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-03-29T10:54:27.892 INFO:teuthology.orchestra.run.smithi111.stdout:(20/142): python3-cephfs-18.0.0-3066.gf03d69a8. 3.2 MB/s | 190 kB 00:00 2023-03-29T10:54:27.895 INFO:teuthology.orchestra.run.smithi119.stdout:(87/142): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 237 kB 00:00 2023-03-29T10:54:27.937 INFO:teuthology.orchestra.run.smithi119.stdout:(88/142): python3-cryptography-3.2.1-5.el8.x86_ 7.3 MB/s | 559 kB 00:00 2023-03-29T10:54:27.961 INFO:teuthology.orchestra.run.smithi119.stdout:(89/142): python3-jwt-1.6.1-2.el8.noarch.rpm 637 kB/s | 43 kB 00:00 2023-03-29T10:54:27.976 INFO:teuthology.orchestra.run.smithi111.stdout:(21/142): python3-rados-18.0.0-3066.gf03d69a8.e 4.3 MB/s | 369 kB 00:00 2023-03-29T10:54:27.987 INFO:teuthology.orchestra.run.smithi119.stdout:(90/142): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-03-29T10:54:28.012 INFO:teuthology.orchestra.run.smithi119.stdout:(91/142): python3-requests-oauthlib-1.0.0-1.el8 1.7 MB/s | 43 kB 00:00 2023-03-29T10:54:28.037 INFO:teuthology.orchestra.run.smithi119.stdout:(92/142): python3-pycparser-2.14-14.el8.noarch. 1.4 MB/s | 109 kB 00:00 2023-03-29T10:54:28.060 INFO:teuthology.orchestra.run.smithi111.stdout:(22/142): python3-rbd-18.0.0-3066.gf03d69a8.el8 4.3 MB/s | 365 kB 00:00 2023-03-29T10:54:28.063 INFO:teuthology.orchestra.run.smithi119.stdout:(93/142): zlib-devel-1.2.11-19.el8.x86_64.rpm 2.3 MB/s | 58 kB 00:00 2023-03-29T10:54:28.088 INFO:teuthology.orchestra.run.smithi119.stdout:(94/142): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 958 kB/s | 72 kB 00:00 2023-03-29T10:54:28.118 INFO:teuthology.orchestra.run.smithi111.stdout:(23/142): python3-rgw-18.0.0-3066.gf03d69a8.el8 1.8 MB/s | 107 kB 00:00 2023-03-29T10:54:28.193 INFO:teuthology.orchestra.run.smithi111.stdout:(24/142): rbd-fuse-18.0.0-3066.gf03d69a8.el8.x8 1.2 MB/s | 95 kB 00:00 2023-03-29T10:54:28.306 INFO:teuthology.orchestra.run.smithi119.stdout:(95/142): python3-scipy-1.0.0-21.module_el8.5.0 12 MB/s | 14 MB 00:01 2023-03-29T10:54:28.331 INFO:teuthology.orchestra.run.smithi119.stdout:(96/142): liboath-2.6.2-3.el8.x86_64.rpm 243 kB/s | 59 kB 00:00 2023-03-29T10:54:28.389 INFO:teuthology.orchestra.run.smithi119.stdout:(97/142): gperftools-libs-2.7-9.el8.x86_64.rpm 940 kB/s | 306 kB 00:00 2023-03-29T10:54:28.439 INFO:teuthology.orchestra.run.smithi119.stdout:(98/142): libunwind-1.3.1-3.el8.x86_64.rpm 563 kB/s | 75 kB 00:00 2023-03-29T10:54:28.464 INFO:teuthology.orchestra.run.smithi119.stdout:(99/142): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 580 kB/s | 44 kB 00:00 2023-03-29T10:54:28.490 INFO:teuthology.orchestra.run.smithi119.stdout:(100/142): python3-asyncssh-2.7.0-2.el8.noarch. 2.9 MB/s | 476 kB 00:00 2023-03-29T10:54:28.515 INFO:teuthology.orchestra.run.smithi119.stdout:(101/142): python3-cachetools-3.1.1-4.el8.noarc 654 kB/s | 33 kB 00:00 2023-03-29T10:54:28.540 INFO:teuthology.orchestra.run.smithi119.stdout:(102/142): python3-beautifulsoup4-4.6.3-2.el8.1 1.8 MB/s | 185 kB 00:00 2023-03-29T10:54:28.566 INFO:teuthology.orchestra.run.smithi119.stdout:(103/142): python3-certifi-2018.10.15-7.el8.noa 215 kB/s | 16 kB 00:00 2023-03-29T10:54:28.591 INFO:teuthology.orchestra.run.smithi119.stdout:(104/142): python3-cheroot-8.5.2-1.el8.noarch.r 2.2 MB/s | 173 kB 00:00 2023-03-29T10:54:28.611 INFO:teuthology.orchestra.run.smithi111.stdout:(25/142): librgw2-18.0.0-3066.gf03d69a8.el8.x86 7.2 MB/s | 6.5 MB 00:00 2023-03-29T10:54:28.667 INFO:teuthology.orchestra.run.smithi119.stdout:(105/142): python3-google-auth-1.1.1-10.el8.noa 809 kB/s | 81 kB 00:00 2023-03-29T10:54:28.669 INFO:teuthology.orchestra.run.smithi111.stdout:(26/142): rbd-mirror-18.0.0-3066.gf03d69a8.el8. 7.3 MB/s | 3.5 MB 00:00 2023-03-29T10:54:28.684 INFO:teuthology.orchestra.run.smithi119.stdout:(106/142): python3-jaraco-6.2-6.el8.noarch.rpm 120 kB/s | 11 kB 00:00 2023-03-29T10:54:28.694 INFO:teuthology.orchestra.run.smithi111.stdout:(27/142): rbd-nbd-18.0.0-3066.gf03d69a8.el8.x86 2.1 MB/s | 180 kB 00:00 2023-03-29T10:54:28.709 INFO:teuthology.orchestra.run.smithi119.stdout:(107/142): python3-cherrypy-18.4.0-1.el8.noarch 2.2 MB/s | 384 kB 00:00 2023-03-29T10:54:28.719 INFO:teuthology.orchestra.run.smithi111.stdout:(28/142): ceph-grafana-dashboards-18.0.0-3066.g 463 kB/s | 23 kB 00:00 2023-03-29T10:54:28.726 INFO:teuthology.orchestra.run.smithi119.stdout:(108/142): python3-jaraco-functools-2.0-4.el8.n 298 kB/s | 18 kB 00:00 2023-03-29T10:54:28.745 INFO:teuthology.orchestra.run.smithi111.stdout:(29/142): ceph-mgr-cephadm-18.0.0-3066.gf03d69a 2.4 MB/s | 123 kB 00:00 2023-03-29T10:54:28.751 INFO:teuthology.orchestra.run.smithi119.stdout:(109/142): python3-logutils-0.3.5-11.el8.noarch 1.1 MB/s | 49 kB 00:00 2023-03-29T10:54:28.776 INFO:teuthology.orchestra.run.smithi119.stdout:(110/142): python3-more-itertools-7.2.0-3.el8.n 1.1 MB/s | 59 kB 00:00 2023-03-29T10:54:28.793 INFO:teuthology.orchestra.run.smithi119.stdout:(111/142): python3-natsort-7.1.1-2.el8.noarch.r 1.4 MB/s | 60 kB 00:00 2023-03-29T10:54:28.835 INFO:teuthology.orchestra.run.smithi119.stdout:(112/142): python3-portend-2.6-1.el8.noarch.rpm 372 kB/s | 16 kB 00:00 2023-03-29T10:54:28.861 INFO:teuthology.orchestra.run.smithi119.stdout:(113/142): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-03-29T10:54:28.878 INFO:teuthology.orchestra.run.smithi119.stdout:(114/142): python3-repoze-lru-0.7-6.el8.noarch. 807 kB/s | 34 kB 00:00 2023-03-29T10:54:28.920 INFO:teuthology.orchestra.run.smithi119.stdout:(115/142): python3-kubernetes-11.0.0-6.el8.noar 4.9 MB/s | 1.2 MB 00:00 2023-03-29T10:54:28.937 INFO:teuthology.orchestra.run.smithi119.stdout:(116/142): python3-rsa-4.8-1.el8.noarch.rpm 1.0 MB/s | 61 kB 00:00 2023-03-29T10:54:28.953 INFO:teuthology.orchestra.run.smithi111.stdout:(30/142): ceph-mgr-dashboard-18.0.0-3066.gf03d6 7.3 MB/s | 1.7 MB 00:00 2023-03-29T10:54:28.962 INFO:teuthology.orchestra.run.smithi119.stdout:(117/142): python3-routes-2.4.1-12.el8.noarch.r 1.9 MB/s | 196 kB 00:00 2023-03-29T10:54:28.979 INFO:teuthology.orchestra.run.smithi119.stdout:(118/142): python3-simplegeneric-0.8.1-17.el8.n 319 kB/s | 19 kB 00:00 2023-03-29T10:54:28.996 INFO:teuthology.orchestra.run.smithi119.stdout:(119/142): python3-singledispatch-3.4.0.3-18.el 405 kB/s | 24 kB 00:00 2023-03-29T10:54:29.012 INFO:teuthology.orchestra.run.smithi111.stdout:(31/142): ceph-mgr-modules-core-18.0.0-3066.gf0 4.0 MB/s | 239 kB 00:00 2023-03-29T10:54:29.012 INFO:teuthology.orchestra.run.smithi119.stdout:(120/142): python3-tempora-1.14.1-5.el8.noarch. 580 kB/s | 29 kB 00:00 2023-03-29T10:54:29.030 INFO:teuthology.orchestra.run.smithi119.stdout:(121/142): python3-trustme-0.6.0-4.el8.noarch.r 543 kB/s | 27 kB 00:00 2023-03-29T10:54:29.062 INFO:teuthology.orchestra.run.smithi111.stdout:(32/142): ceph-mgr-rook-18.0.0-3066.gf03d69a8.e 952 kB/s | 47 kB 00:00 2023-03-29T10:54:29.072 INFO:teuthology.orchestra.run.smithi119.stdout:(122/142): python3-waitress-1.2.1-2.el8.1.noarc 3.1 MB/s | 241 kB 00:00 2023-03-29T10:54:29.097 INFO:teuthology.orchestra.run.smithi119.stdout:(123/142): python3-webob-1.8.5-1.el8.1.noarch.r 2.9 MB/s | 251 kB 00:00 2023-03-29T10:54:29.113 INFO:teuthology.orchestra.run.smithi111.stdout:(33/142): ceph-prometheus-alerts-18.0.0-3066.gf 290 kB/s | 15 kB 00:00 2023-03-29T10:54:29.122 INFO:teuthology.orchestra.run.smithi119.stdout:(124/142): python3-websocket-client-0.56.0-5.el 664 kB/s | 61 kB 00:00 2023-03-29T10:54:29.147 INFO:teuthology.orchestra.run.smithi119.stdout:(125/142): python3-webtest-2.0.33-1.el8.noarch. 1.1 MB/s | 85 kB 00:00 2023-03-29T10:54:29.173 INFO:teuthology.orchestra.run.smithi119.stdout:(126/142): python3-zc-lockfile-2.0-2.el8.noarch 304 kB/s | 23 kB 00:00 2023-03-29T10:54:29.296 INFO:teuthology.orchestra.run.smithi111.stdout:(34/142): ceph-volume-18.0.0-3066.gf03d69a8.el8 1.4 MB/s | 263 kB 00:00 2023-03-29T10:54:29.363 INFO:teuthology.orchestra.run.smithi111.stdout:(35/142): cephadm-18.0.0-3066.gf03d69a8.el8.noa 1.2 MB/s | 83 kB 00:00 2023-03-29T10:54:29.423 INFO:teuthology.orchestra.run.smithi119.stdout:(127/142): thrift-0.13.0-2.el8.x86_64.rpm 5.8 MB/s | 1.7 MB 00:00 2023-03-29T10:54:29.438 INFO:teuthology.orchestra.run.smithi111.stdout:(36/142): cephfs-top-18.0.0-3066.gf03d69a8.el8. 241 kB/s | 18 kB 00:00 2023-03-29T10:54:29.457 INFO:teuthology.orchestra.run.smithi119.stdout:(128/142): NetworkManager-initscripts-updown-1. 4.2 MB/s | 142 kB 00:00 2023-03-29T10:54:29.482 INFO:teuthology.orchestra.run.smithi119.stdout:(129/142): elfutils-debuginfod-client-0.188-3.e 3.0 MB/s | 75 kB 00:00 2023-03-29T10:54:29.516 INFO:teuthology.orchestra.run.smithi119.stdout:(130/142): elfutils-libelf-0.188-3.el8.x86_64.r 6.9 MB/s | 231 kB 00:00 2023-03-29T10:54:29.530 INFO:teuthology.orchestra.run.smithi111.stdout:(37/142): automake-1.16.1-7.el8.noarch.rpm 7.6 MB/s | 713 kB 00:00 2023-03-29T10:54:29.549 INFO:teuthology.orchestra.run.smithi119.stdout:(131/142): elfutils-libs-0.188-3.el8.x86_64.rpm 8.7 MB/s | 299 kB 00:00 2023-03-29T10:54:29.564 INFO:teuthology.orchestra.run.smithi111.stdout:(38/142): bison-3.0.4-10.el8.x86_64.rpm 20 MB/s | 688 kB 00:00 2023-03-29T10:54:29.583 INFO:teuthology.orchestra.run.smithi119.stdout:(132/142): libblkid-2.32.1-41.el8.x86_64.rpm 6.5 MB/s | 220 kB 00:00 2023-03-29T10:54:29.598 INFO:teuthology.orchestra.run.smithi111.stdout:(39/142): flex-2.6.1-9.el8.x86_64.rpm 9.3 MB/s | 320 kB 00:00 2023-03-29T10:54:29.617 INFO:teuthology.orchestra.run.smithi119.stdout:(133/142): libfdisk-2.32.1-41.el8.x86_64.rpm 7.4 MB/s | 253 kB 00:00 2023-03-29T10:54:29.623 INFO:teuthology.orchestra.run.smithi111.stdout:(40/142): jq-1.6-6.el8.x86_64.rpm 8.1 MB/s | 202 kB 00:00 2023-03-29T10:54:29.649 INFO:teuthology.orchestra.run.smithi111.stdout:(41/142): libpq-13.5-1.el8.x86_64.rpm 7.7 MB/s | 198 kB 00:00 2023-03-29T10:54:29.675 INFO:teuthology.orchestra.run.smithi119.stdout:(134/142): librados2-18.0.0-3066.gf03d69a8.el8. 6.9 MB/s | 3.7 MB 00:00 2023-03-29T10:54:29.682 INFO:teuthology.orchestra.run.smithi111.stdout:(42/142): librdkafka-0.11.4-3.el8.x86_64.rpm 10 MB/s | 354 kB 00:00 2023-03-29T10:54:29.700 INFO:teuthology.orchestra.run.smithi119.stdout:(135/142): libmount-2.32.1-41.el8.x86_64.rpm 2.8 MB/s | 236 kB 00:00 2023-03-29T10:54:29.716 INFO:teuthology.orchestra.run.smithi111.stdout:(43/142): libtool-2.4.6-25.el8.x86_64.rpm 21 MB/s | 709 kB 00:00 2023-03-29T10:54:29.726 INFO:teuthology.orchestra.run.smithi119.stdout:(136/142): libsmartcols-2.32.1-41.el8.x86_64.rp 3.5 MB/s | 179 kB 00:00 2023-03-29T10:54:29.750 INFO:teuthology.orchestra.run.smithi111.stdout:(44/142): lttng-ust-2.8.1-11.el8.x86_64.rpm 7.6 MB/s | 259 kB 00:00 2023-03-29T10:54:29.751 INFO:teuthology.orchestra.run.smithi119.stdout:(137/142): libuuid-2.32.1-41.el8.x86_64.rpm 1.9 MB/s | 98 kB 00:00 2023-03-29T10:54:29.775 INFO:teuthology.orchestra.run.smithi111.stdout:(45/142): oniguruma-6.8.2-2.el8.x86_64.rpm 7.5 MB/s | 187 kB 00:00 2023-03-29T10:54:29.818 INFO:teuthology.orchestra.run.smithi119.stdout:(138/142): librbd1-18.0.0-3066.gf03d69a8.el8.x8 5.5 MB/s | 3.6 MB 00:00 2023-03-29T10:54:29.852 INFO:teuthology.orchestra.run.smithi119.stdout:(139/142): openssl-1.1.1k-9.el8.x86_64.rpm 5.7 MB/s | 737 kB 00:00 2023-03-29T10:54:29.885 INFO:teuthology.orchestra.run.smithi119.stdout:(140/142): util-linux-user-2.32.1-41.el8.x86_64 3.0 MB/s | 102 kB 00:00 2023-03-29T10:54:29.917 INFO:teuthology.orchestra.run.smithi111.stdout:(46/142): ceph-mgr-diskprediction-local-18.0.0- 6.3 MB/s | 7.4 MB 00:01 2023-03-29T10:54:29.961 INFO:teuthology.orchestra.run.smithi119.stdout:(141/142): util-linux-2.32.1-41.el8.x86_64.rpm 18 MB/s | 2.6 MB 00:00 2023-03-29T10:54:29.992 INFO:teuthology.orchestra.run.smithi111.stdout:(47/142): openblas-0.3.15-4.el8.x86_64.rpm 22 MB/s | 4.7 MB 00:00 2023-03-29T10:54:29.994 INFO:teuthology.orchestra.run.smithi119.stdout:(142/142): openssl-libs-1.1.1k-9.el8.x86_64.rpm 6.1 MB/s | 1.5 MB 00:00 2023-03-29T10:54:29.998 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-03-29T10:54:29.999 INFO:teuthology.orchestra.run.smithi119.stdout:Total 20 MB/s | 204 MB 00:10 2023-03-29T10:54:30.051 INFO:teuthology.orchestra.run.smithi111.stdout:(48/142): postgresql-10.23-1.module_el8+300+2c5 28 MB/s | 1.6 MB 00:00 2023-03-29T10:54:30.101 INFO:teuthology.orchestra.run.smithi111.stdout:(49/142): postgresql-contrib-10.23-1.module_el8 18 MB/s | 903 kB 00:00 2023-03-29T10:54:30.227 INFO:teuthology.orchestra.run.smithi111.stdout:(50/142): postgresql-server-10.23-1.module_el8+ 41 MB/s | 5.1 MB 00:00 2023-03-29T10:54:30.351 INFO:teuthology.orchestra.run.smithi111.stdout:(51/142): python3-babel-2.5.1-7.el8.noarch.rpm 39 MB/s | 4.8 MB 00:00 2023-03-29T10:54:30.393 INFO:teuthology.orchestra.run.smithi111.stdout:(52/142): python3-jinja2-2.10.1-3.el8.noarch.rp 13 MB/s | 538 kB 00:00 2023-03-29T10:54:30.419 INFO:teuthology.orchestra.run.smithi111.stdout:(53/142): python3-jsonpatch-1.21-2.el8.noarch.r 1.0 MB/s | 27 kB 00:00 2023-03-29T10:54:30.444 INFO:teuthology.orchestra.run.smithi111.stdout:(54/142): python3-jsonpointer-1.10-11.el8.noarc 801 kB/s | 20 kB 00:00 2023-03-29T10:54:30.469 INFO:teuthology.orchestra.run.smithi111.stdout:(55/142): python3-mako-1.0.6-14.el8.noarch.rpm 6.1 MB/s | 157 kB 00:00 2023-03-29T10:54:30.536 INFO:teuthology.orchestra.run.smithi111.stdout:(56/142): openblas-threads-0.3.15-4.el8.x86_64. 7.8 MB/s | 4.9 MB 00:00 2023-03-29T10:54:30.562 INFO:teuthology.orchestra.run.smithi111.stdout:(57/142): python3-markupsafe-0.23-19.el8.x86_64 423 kB/s | 39 kB 00:00 2023-03-29T10:54:30.595 INFO:teuthology.orchestra.run.smithi111.stdout:(58/142): python3-numpy-f2py-1.14.3-10.el8.x86_ 6.6 MB/s | 225 kB 00:00 2023-03-29T10:54:30.620 INFO:teuthology.orchestra.run.smithi111.stdout:(59/142): python3-prettytable-0.7.2-14.el8.noar 1.7 MB/s | 44 kB 00:00 2023-03-29T10:54:30.646 INFO:teuthology.orchestra.run.smithi111.stdout:(60/142): python3-pyOpenSSL-19.0.0-1.el8.noarch 4.0 MB/s | 103 kB 00:00 2023-03-29T10:54:30.671 INFO:teuthology.orchestra.run.smithi111.stdout:(61/142): python3-pyasn1-0.3.7-6.el8.noarch.rpm 5.0 MB/s | 126 kB 00:00 2023-03-29T10:54:30.696 INFO:teuthology.orchestra.run.smithi111.stdout:(62/142): python3-pyasn1-modules-0.3.7-6.el8.no 4.3 MB/s | 110 kB 00:00 2023-03-29T10:54:30.713 INFO:teuthology.orchestra.run.smithi111.stdout:(63/142): python3-pytz-2017.2-9.el8.noarch.rpm 3.2 MB/s | 54 kB 00:00 2023-03-29T10:54:30.831 INFO:teuthology.orchestra.run.smithi111.stdout:(64/142): python3-numpy-1.14.3-10.el8.x86_64.rp 13 MB/s | 3.7 MB 00:00 2023-03-29T10:54:30.881 INFO:teuthology.orchestra.run.smithi111.stdout:(65/142): python3-werkzeug-0.12.2-4.el8.noarch. 9.0 MB/s | 457 kB 00:00 2023-03-29T10:54:30.923 INFO:teuthology.orchestra.run.smithi111.stdout:(66/142): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-03-29T10:54:30.956 INFO:teuthology.orchestra.run.smithi111.stdout:(67/142): uuid-1.6.2-43.el8.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2023-03-29T10:54:30.990 INFO:teuthology.orchestra.run.smithi111.stdout:(68/142): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.1 MB/s | 71 kB 00:00 2023-03-29T10:54:31.024 INFO:teuthology.orchestra.run.smithi111.stdout:(69/142): elfutils-libelf-devel-0.188-3.el8.x86 1.8 MB/s | 61 kB 00:00 2023-03-29T10:54:31.057 INFO:teuthology.orchestra.run.smithi111.stdout:(70/142): fuse-2.9.7-16.el8.x86_64.rpm 2.4 MB/s | 83 kB 00:00 2023-03-29T10:54:31.091 INFO:teuthology.orchestra.run.smithi111.stdout:(71/142): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.4 MB/s | 48 kB 00:00 2023-03-29T10:54:31.124 INFO:teuthology.orchestra.run.smithi111.stdout:(72/142): krb5-devel-1.18.2-22.el8.x86_64.rpm 16 MB/s | 561 kB 00:00 2023-03-29T10:54:31.150 INFO:teuthology.orchestra.run.smithi111.stdout:(73/142): libaio-devel-0.3.112-1.el8.x86_64.rpm 745 kB/s | 19 kB 00:00 2023-03-29T10:54:31.175 INFO:teuthology.orchestra.run.smithi111.stdout:(74/142): libcom_err-devel-1.45.6-5.el8.x86_64. 1.5 MB/s | 39 kB 00:00 2023-03-29T10:54:31.217 INFO:teuthology.orchestra.run.smithi111.stdout:(75/142): libgfortran-8.5.0-18.el8.x86_64.rpm 15 MB/s | 645 kB 00:00 2023-03-29T10:54:31.359 INFO:teuthology.orchestra.run.smithi111.stdout:(76/142): python3-scipy-1.0.0-21.module_el8.5.0 21 MB/s | 14 MB 00:00 2023-03-29T10:54:31.385 INFO:teuthology.orchestra.run.smithi111.stdout:(77/142): libquadmath-8.5.0-18.el8.x86_64.rpm 1.0 MB/s | 172 kB 00:00 2023-03-29T10:54:31.410 INFO:teuthology.orchestra.run.smithi111.stdout:(78/142): librabbitmq-0.9.0-3.el8.x86_64.rpm 940 kB/s | 47 kB 00:00 2023-03-29T10:54:31.452 INFO:teuthology.orchestra.run.smithi111.stdout:(79/142): libsepol-devel-2.9-3.el8.x86_64.rpm 2.1 MB/s | 87 kB 00:00 2023-03-29T10:54:31.477 INFO:teuthology.orchestra.run.smithi111.stdout:(80/142): libselinux-devel-2.9-6.el8.x86_64.rpm 2.9 MB/s | 200 kB 00:00 2023-03-29T10:54:31.503 INFO:teuthology.orchestra.run.smithi111.stdout:(81/142): libuuid-devel-2.32.1-41.el8.x86_64.rp 1.9 MB/s | 99 kB 00:00 2023-03-29T10:54:31.528 INFO:teuthology.orchestra.run.smithi111.stdout:(82/142): libverto-devel-0.3.2-2.el8.x86_64.rpm 359 kB/s | 18 kB 00:00 2023-03-29T10:54:31.561 INFO:teuthology.orchestra.run.smithi111.stdout:(83/142): pcre2-devel-10.32-3.el8.x86_64.rpm 18 MB/s | 605 kB 00:00 2023-03-29T10:54:31.583 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-03-29T10:54:31.595 INFO:teuthology.orchestra.run.smithi111.stdout:(84/142): pcre2-utf16-10.32-3.el8.x86_64.rpm 6.7 MB/s | 229 kB 00:00 2023-03-29T10:54:31.629 INFO:teuthology.orchestra.run.smithi111.stdout:(85/142): pcre2-utf32-10.32-3.el8.x86_64.rpm 6.5 MB/s | 220 kB 00:00 2023-03-29T10:54:31.663 INFO:teuthology.orchestra.run.smithi111.stdout:(86/142): python3-cffi-1.11.5-5.el8.x86_64.rpm 6.9 MB/s | 237 kB 00:00 2023-03-29T10:54:31.696 INFO:teuthology.orchestra.run.smithi111.stdout:(87/142): python3-cryptography-3.2.1-5.el8.x86_ 17 MB/s | 559 kB 00:00 2023-03-29T10:54:31.721 INFO:teuthology.orchestra.run.smithi111.stdout:(88/142): python3-jwt-1.6.1-2.el8.noarch.rpm 1.7 MB/s | 43 kB 00:00 2023-03-29T10:54:31.755 INFO:teuthology.orchestra.run.smithi111.stdout:(89/142): python3-oauthlib-2.1.0-1.el8.noarch.r 4.6 MB/s | 155 kB 00:00 2023-03-29T10:54:31.780 INFO:teuthology.orchestra.run.smithi111.stdout:(90/142): python3-pycparser-2.14-14.el8.noarch. 4.3 MB/s | 109 kB 00:00 2023-03-29T10:54:31.805 INFO:teuthology.orchestra.run.smithi111.stdout:(91/142): python3-requests-oauthlib-1.0.0-1.el8 1.7 MB/s | 43 kB 00:00 2023-03-29T10:54:31.864 INFO:teuthology.orchestra.run.smithi111.stdout:(92/142): openssl-devel-1.1.1k-9.el8.x86_64.rpm 8.8 MB/s | 3.2 MB 00:00 2023-03-29T10:54:31.887 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-03-29T10:54:31.887 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-03-29T10:54:31.889 INFO:teuthology.orchestra.run.smithi111.stdout:(93/142): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 864 kB/s | 72 kB 00:00 2023-03-29T10:54:31.915 INFO:teuthology.orchestra.run.smithi111.stdout:(94/142): zlib-devel-1.2.11-19.el8.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-03-29T10:54:32.307 INFO:teuthology.orchestra.run.smithi111.stdout:(95/142): ceph-test-18.0.0-3066.gf03d69a8.el8.x 7.6 MB/s | 41 MB 00:05 2023-03-29T10:54:32.432 INFO:teuthology.orchestra.run.smithi111.stdout:(96/142): liboath-2.6.2-3.el8.x86_64.rpm 114 kB/s | 59 kB 00:00 2023-03-29T10:54:32.457 INFO:teuthology.orchestra.run.smithi111.stdout:(97/142): libunwind-1.3.1-3.el8.x86_64.rpm 502 kB/s | 75 kB 00:00 2023-03-29T10:54:32.491 INFO:teuthology.orchestra.run.smithi111.stdout:(98/142): gperftools-libs-2.7-9.el8.x86_64.rpm 509 kB/s | 306 kB 00:00 2023-03-29T10:54:32.516 INFO:teuthology.orchestra.run.smithi111.stdout:(99/142): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 747 kB/s | 44 kB 00:00 2023-03-29T10:54:32.550 INFO:teuthology.orchestra.run.smithi111.stdout:(100/142): python3-beautifulsoup4-4.6.3-2.el8.1 3.1 MB/s | 185 kB 00:00 2023-03-29T10:54:32.567 INFO:teuthology.orchestra.run.smithi111.stdout:(101/142): python3-cachetools-3.1.1-4.el8.noarc 658 kB/s | 33 kB 00:00 2023-03-29T10:54:32.592 INFO:teuthology.orchestra.run.smithi111.stdout:(102/142): python3-asyncssh-2.7.0-2.el8.noarch. 2.9 MB/s | 476 kB 00:00 2023-03-29T10:54:32.618 INFO:teuthology.orchestra.run.smithi111.stdout:(103/142): python3-certifi-2018.10.15-7.el8.noa 242 kB/s | 16 kB 00:00 2023-03-29T10:54:32.651 INFO:teuthology.orchestra.run.smithi111.stdout:(104/142): python3-cheroot-8.5.2-1.el8.noarch.r 2.0 MB/s | 173 kB 00:00 2023-03-29T10:54:32.685 INFO:teuthology.orchestra.run.smithi111.stdout:(105/142): python3-google-auth-1.1.1-10.el8.noa 1.2 MB/s | 81 kB 00:00 2023-03-29T10:54:32.710 INFO:teuthology.orchestra.run.smithi111.stdout:(106/142): python3-jaraco-6.2-6.el8.noarch.rpm 189 kB/s | 11 kB 00:00 2023-03-29T10:54:32.735 INFO:teuthology.orchestra.run.smithi111.stdout:(107/142): python3-cherrypy-18.4.0-1.el8.noarch 2.6 MB/s | 384 kB 00:00 2023-03-29T10:54:32.782 INFO:teuthology.orchestra.run.smithi111.stdout:(108/142): python3-jaraco-functools-2.0-4.el8.n 182 kB/s | 18 kB 00:00 2023-03-29T10:54:32.802 INFO:teuthology.orchestra.run.smithi111.stdout:(109/142): python3-logutils-0.3.5-11.el8.noarch 732 kB/s | 49 kB 00:00 2023-03-29T10:54:32.828 INFO:teuthology.orchestra.run.smithi111.stdout:(110/142): python3-more-itertools-7.2.0-3.el8.n 1.3 MB/s | 59 kB 00:00 2023-03-29T10:54:32.845 INFO:teuthology.orchestra.run.smithi111.stdout:(111/142): python3-natsort-7.1.1-2.el8.noarch.r 1.4 MB/s | 60 kB 00:00 2023-03-29T10:54:32.887 INFO:teuthology.orchestra.run.smithi111.stdout:(112/142): python3-portend-2.6-1.el8.noarch.rpm 375 kB/s | 16 kB 00:00 2023-03-29T10:54:32.912 INFO:teuthology.orchestra.run.smithi111.stdout:(113/142): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-03-29T10:54:32.946 INFO:teuthology.orchestra.run.smithi111.stdout:(114/142): python3-kubernetes-11.0.0-6.el8.noar 5.0 MB/s | 1.2 MB 00:00 2023-03-29T10:54:32.963 INFO:teuthology.orchestra.run.smithi111.stdout:(115/142): python3-repoze-lru-0.7-6.el8.noarch. 447 kB/s | 34 kB 00:00 2023-03-29T10:54:32.988 INFO:teuthology.orchestra.run.smithi111.stdout:(116/142): python3-routes-2.4.1-12.el8.noarch.r 2.5 MB/s | 196 kB 00:00 2023-03-29T10:54:33.013 INFO:teuthology.orchestra.run.smithi111.stdout:(117/142): python3-rsa-4.8-1.el8.noarch.rpm 918 kB/s | 61 kB 00:00 2023-03-29T10:54:33.030 INFO:teuthology.orchestra.run.smithi111.stdout:(118/142): python3-simplegeneric-0.8.1-17.el8.n 279 kB/s | 19 kB 00:00 2023-03-29T10:54:33.048 INFO:teuthology.orchestra.run.smithi111.stdout:(119/142): python3-singledispatch-3.4.0.3-18.el 406 kB/s | 24 kB 00:00 2023-03-29T10:54:33.064 INFO:teuthology.orchestra.run.smithi111.stdout:(120/142): python3-tempora-1.14.1-5.el8.noarch. 580 kB/s | 29 kB 00:00 2023-03-29T10:54:33.081 INFO:teuthology.orchestra.run.smithi111.stdout:(121/142): python3-trustme-0.6.0-4.el8.noarch.r 545 kB/s | 27 kB 00:00 2023-03-29T10:54:33.123 INFO:teuthology.orchestra.run.smithi111.stdout:(122/142): python3-webob-1.8.5-1.el8.1.noarch.r 4.2 MB/s | 251 kB 00:00 2023-03-29T10:54:33.148 INFO:teuthology.orchestra.run.smithi111.stdout:(123/142): python3-waitress-1.2.1-2.el8.1.noarc 2.3 MB/s | 241 kB 00:00 2023-03-29T10:54:33.174 INFO:teuthology.orchestra.run.smithi111.stdout:(124/142): python3-websocket-client-0.56.0-5.el 665 kB/s | 61 kB 00:00 2023-03-29T10:54:33.199 INFO:teuthology.orchestra.run.smithi111.stdout:(125/142): python3-webtest-2.0.33-1.el8.noarch. 1.1 MB/s | 85 kB 00:00 2023-03-29T10:54:33.224 INFO:teuthology.orchestra.run.smithi111.stdout:(126/142): python3-zc-lockfile-2.0-2.el8.noarch 304 kB/s | 23 kB 00:00 2023-03-29T10:54:33.441 INFO:teuthology.orchestra.run.smithi111.stdout:(127/142): thrift-0.13.0-2.el8.x86_64.rpm 6.5 MB/s | 1.7 MB 00:00 2023-03-29T10:54:33.466 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-03-29T10:54:33.471 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-03-29T10:54:33.475 INFO:teuthology.orchestra.run.smithi111.stdout:(128/142): NetworkManager-initscripts-updown-1. 4.2 MB/s | 142 kB 00:00 2023-03-29T10:54:33.500 INFO:teuthology.orchestra.run.smithi111.stdout:(129/142): elfutils-debuginfod-client-0.188-3.e 3.0 MB/s | 75 kB 00:00 2023-03-29T10:54:33.534 INFO:teuthology.orchestra.run.smithi111.stdout:(130/142): elfutils-libelf-0.188-3.el8.x86_64.r 6.9 MB/s | 231 kB 00:00 2023-03-29T10:54:33.568 INFO:teuthology.orchestra.run.smithi111.stdout:(131/142): elfutils-libs-0.188-3.el8.x86_64.rpm 8.7 MB/s | 299 kB 00:00 2023-03-29T10:54:33.601 INFO:teuthology.orchestra.run.smithi111.stdout:(132/142): libblkid-2.32.1-41.el8.x86_64.rpm 6.5 MB/s | 220 kB 00:00 2023-03-29T10:54:33.635 INFO:teuthology.orchestra.run.smithi111.stdout:(133/142): libfdisk-2.32.1-41.el8.x86_64.rpm 7.4 MB/s | 253 kB 00:00 2023-03-29T10:54:33.669 INFO:teuthology.orchestra.run.smithi111.stdout:(134/142): libmount-2.32.1-41.el8.x86_64.rpm 7.0 MB/s | 236 kB 00:00 2023-03-29T10:54:33.727 INFO:teuthology.orchestra.run.smithi111.stdout:(135/142): librados2-18.0.0-3066.gf03d69a8.el8. 6.9 MB/s | 3.7 MB 00:00 2023-03-29T10:54:33.752 INFO:teuthology.orchestra.run.smithi111.stdout:(136/142): libsmartcols-2.32.1-41.el8.x86_64.rp 2.1 MB/s | 179 kB 00:00 2023-03-29T10:54:33.778 INFO:teuthology.orchestra.run.smithi111.stdout:(137/142): libuuid-2.32.1-41.el8.x86_64.rpm 1.9 MB/s | 98 kB 00:00 2023-03-29T10:54:33.845 INFO:teuthology.orchestra.run.smithi111.stdout:(138/142): librbd1-18.0.0-3066.gf03d69a8.el8.x8 5.7 MB/s | 3.6 MB 00:00 2023-03-29T10:54:33.895 INFO:teuthology.orchestra.run.smithi111.stdout:(139/142): openssl-1.1.1k-9.el8.x86_64.rpm 5.1 MB/s | 737 kB 00:00 2023-03-29T10:54:33.928 INFO:teuthology.orchestra.run.smithi111.stdout:(140/142): util-linux-user-2.32.1-41.el8.x86_64 3.2 MB/s | 102 kB 00:00 2023-03-29T10:54:33.996 INFO:teuthology.orchestra.run.smithi111.stdout:(141/142): openssl-libs-1.1.1k-9.el8.x86_64.rpm 6.8 MB/s | 1.5 MB 00:00 2023-03-29T10:54:34.037 INFO:teuthology.orchestra.run.smithi111.stdout:(142/142): util-linux-2.32.1-41.el8.x86_64.rpm 14 MB/s | 2.6 MB 00:00 2023-03-29T10:54:34.043 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2023-03-29T10:54:34.043 INFO:teuthology.orchestra.run.smithi111.stdout:Total 16 MB/s | 204 MB 00:12 2023-03-29T10:54:35.651 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2023-03-29T10:54:35.957 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2023-03-29T10:54:35.957 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2023-03-29T10:54:37.544 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2023-03-29T10:54:37.548 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2023-03-29T10:54:38.978 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-03-29T10:54:39.159 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8.x86_64 1/1 2023-03-29T10:54:39.392 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-libs-1:1.1.1k-9.el8.x86_64 1/157 2023-03-29T10:54:39.614 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8.x86_64 1/157 2023-03-29T10:54:39.786 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/157 2023-03-29T10:54:39.939 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libuuid-2.32.1-41.el8.x86_64 3/157 2023-03-29T10:54:40.018 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 3/157 2023-03-29T10:54:40.156 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libblkid-2.32.1-41.el8.x86_64 4/157 2023-03-29T10:54:40.244 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 4/157 2023-03-29T10:54:40.439 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/157 2023-03-29T10:54:40.597 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/157 2023-03-29T10:54:40.674 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/157 2023-03-29T10:54:40.844 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 7/157 2023-03-29T10:54:41.030 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/157 2023-03-29T10:54:41.478 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/157 2023-03-29T10:54:41.638 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librados2-2:18.0.0-3066.gf03d69a8.el8.x86_64 9/157 2023-03-29T10:54:42.064 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:18.0.0-3066.gf03d69a8.el8.x86_64 9/157 2023-03-29T10:54:42.205 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librbd1-2:18.0.0-3066.gf03d69a8.el8.x86_64 10/157 2023-03-29T10:54:42.337 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:18.0.0-3066.gf03d69a8.el8.x86_64 10/157 2023-03-29T10:54:42.513 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs2-2:18.0.0-3066.gf03d69a8.el8.x86_64 11/157 2023-03-29T10:54:42.605 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs2-2:18.0.0-3066.gf03d69a8.el8.x86_64 11/157 2023-03-29T10:54:42.814 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rados-2:18.0.0-3066.gf03d69a8.el8.x86_64 12/157 2023-03-29T10:54:42.915 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2023-03-29T10:54:43.015 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libpq-13.5-1.el8.x86_64 13/157 2023-03-29T10:54:43.098 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8.x86_64 1/1 2023-03-29T10:54:43.285 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : elfutils-libelf-0.188-3.el8.x86_64 14/157 2023-03-29T10:54:43.313 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-libs-1:1.1.1k-9.el8.x86_64 1/157 2023-03-29T10:54:43.478 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 15/157 2023-03-29T10:54:43.561 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8.x86_64 1/157 2023-03-29T10:54:43.774 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/157 2023-03-29T10:54:43.910 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : libuuid-2.32.1-41.el8.x86_64 3/157 2023-03-29T10:54:43.915 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/157 2023-03-29T10:54:43.984 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 3/157 2023-03-29T10:54:44.144 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : libblkid-2.32.1-41.el8.x86_64 4/157 2023-03-29T10:54:44.229 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 4/157 2023-03-29T10:54:44.302 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 17/157 2023-03-29T10:54:44.438 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephsqlite-2:18.0.0-3066.gf03d69a8.el8.x86_64 18/157 2023-03-29T10:54:44.461 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/157 2023-03-29T10:54:44.546 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3066.gf03d69a8.el8.x86_64 18/157 2023-03-29T10:54:44.710 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/157 2023-03-29T10:54:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libradosstriper1-2:18.0.0-3066.gf03d69a8.el8.x86 19/157 2023-03-29T10:54:44.785 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/157 2023-03-29T10:54:44.863 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3066.gf03d69a8.el8.x86 19/157 2023-03-29T10:54:44.983 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 7/157 2023-03-29T10:54:45.095 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 20/157 2023-03-29T10:54:45.152 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/157 2023-03-29T10:54:45.249 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : zlib-devel-1.2.11-19.el8.x86_64 21/157 2023-03-29T10:54:45.404 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 22/157 2023-03-29T10:54:45.558 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 22/157 2023-03-29T10:54:45.601 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/157 2023-03-29T10:54:45.696 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 23/157 2023-03-29T10:54:45.776 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librados2-2:18.0.0-3066.gf03d69a8.el8.x86_64 9/157 2023-03-29T10:54:46.221 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-2:18.0.0-3066.gf03d69a8.el8.x86_64 9/157 2023-03-29T10:54:46.356 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 23/157 2023-03-29T10:54:46.393 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : librbd1-2:18.0.0-3066.gf03d69a8.el8.x86_64 10/157 2023-03-29T10:54:46.523 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-2:18.0.0-3066.gf03d69a8.el8.x86_64 10/157 2023-03-29T10:54:46.529 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-0.3.15-4.el8.x86_64 24/157 2023-03-29T10:54:46.718 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs2-2:18.0.0-3066.gf03d69a8.el8.x86_64 11/157 2023-03-29T10:54:46.834 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephfs2-2:18.0.0-3066.gf03d69a8.el8.x86_64 11/157 2023-03-29T10:54:47.052 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rados-2:18.0.0-3066.gf03d69a8.el8.x86_64 12/157 2023-03-29T10:54:47.205 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 24/157 2023-03-29T10:54:47.245 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libpq-13.5-1.el8.x86_64 13/157 2023-03-29T10:54:47.371 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 25/157 2023-03-29T10:54:47.490 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : elfutils-libelf-0.188-3.el8.x86_64 14/157 2023-03-29T10:54:47.726 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 15/157 2023-03-29T10:54:48.160 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/157 2023-03-29T10:54:48.184 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 25/157 2023-03-29T10:54:48.398 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 17/157 2023-03-29T10:54:48.543 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephsqlite-2:18.0.0-3066.gf03d69a8.el8.x86_64 18/157 2023-03-29T10:54:48.562 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 26/157 2023-03-29T10:54:48.657 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3066.gf03d69a8.el8.x86_64 18/157 2023-03-29T10:54:48.788 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/157 2023-03-29T10:54:48.826 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libradosstriper1-2:18.0.0-3066.gf03d69a8.el8.x86 19/157 2023-03-29T10:54:48.952 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/157 2023-03-29T10:54:48.971 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3066.gf03d69a8.el8.x86 19/157 2023-03-29T10:54:49.208 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 20/157 2023-03-29T10:54:49.232 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/157 2023-03-29T10:54:49.380 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : zlib-devel-1.2.11-19.el8.x86_64 21/157 2023-03-29T10:54:49.453 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-common-2:18.0.0-3066.gf03d69a8.el8. 30/157 2023-03-29T10:54:49.542 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 22/157 2023-03-29T10:54:49.616 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-argparse-2:18.0.0-3066.gf03d69a8.el 31/157 2023-03-29T10:54:49.706 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 22/157 2023-03-29T10:54:49.863 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cephfs-2:18.0.0-3066.gf03d69a8.el8.x86_6 32/157 2023-03-29T10:54:49.943 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 23/157 2023-03-29T10:54:50.622 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 23/157 2023-03-29T10:54:50.817 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-0.3.15-4.el8.x86_64 24/157 2023-03-29T10:54:50.885 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/157 2023-03-29T10:54:51.289 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/157 2023-03-29T10:54:51.520 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 24/157 2023-03-29T10:54:51.604 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/157 2023-03-29T10:54:51.701 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 25/157 2023-03-29T10:54:51.853 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/157 2023-03-29T10:54:52.493 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 25/157 2023-03-29T10:54:52.891 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 26/157 2023-03-29T10:54:53.109 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 27/157 2023-03-29T10:54:53.290 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 28/157 2023-03-29T10:54:53.518 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 29/157 2023-03-29T10:54:53.741 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-common-2:18.0.0-3066.gf03d69a8.el8. 30/157 2023-03-29T10:54:53.887 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-ceph-argparse-2:18.0.0-3066.gf03d69a8.el 31/157 2023-03-29T10:54:53.888 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/157 2023-03-29T10:54:54.132 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cephfs-2:18.0.0-3066.gf03d69a8.el8.x86_6 32/157 2023-03-29T10:54:54.231 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 37/157 2023-03-29T10:54:54.435 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 38/157 2023-03-29T10:54:54.687 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rsa-4.8-1.el8.noarch 39/157 2023-03-29T10:54:54.897 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : elfutils-libs-0.188-3.el8.x86_64 40/157 2023-03-29T10:54:55.091 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : elfutils-debuginfod-client-0.188-3.el8.x86_64 41/157 2023-03-29T10:54:55.166 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 33/157 2023-03-29T10:54:55.268 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rbd-2:18.0.0-3066.gf03d69a8.el8.x86_64 42/157 2023-03-29T10:54:55.448 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librados-devel-2:18.0.0-3066.gf03d69a8.el8.x86_6 43/157 2023-03-29T10:54:55.585 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 34/157 2023-03-29T10:54:55.636 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libfdisk-2.32.1-41.el8.x86_64 44/157 2023-03-29T10:54:55.716 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 44/157 2023-03-29T10:54:55.886 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libmount-2.32.1-41.el8.x86_64 45/157 2023-03-29T10:54:55.949 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 45/157 2023-03-29T10:54:55.960 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 35/157 2023-03-29T10:54:56.137 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libuuid-devel-2.32.1-41.el8.x86_64 46/157 2023-03-29T10:54:56.200 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/157 2023-03-29T10:54:56.315 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 47/157 2023-03-29T10:54:56.478 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libsmartcols-2.32.1-41.el8.x86_64 48/157 2023-03-29T10:54:57.018 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 48/157 2023-03-29T10:54:57.312 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : util-linux-2.32.1-41.el8.x86_64 49/157 2023-03-29T10:54:57.337 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: util-linux-2.32.1-41.el8.x86_64 49/157 2023-03-29T10:54:57.543 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 50/157 2023-03-29T10:54:57.812 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 51/157 2023-03-29T10:54:58.052 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 52/157 2023-03-29T10:54:58.224 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 53/157 2023-03-29T10:54:58.248 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 36/157 2023-03-29T10:54:58.415 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 54/157 2023-03-29T10:54:58.585 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 37/157 2023-03-29T10:54:58.681 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 55/157 2023-03-29T10:54:58.790 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 38/157 2023-03-29T10:54:58.908 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 56/157 2023-03-29T10:54:59.030 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rsa-4.8-1.el8.noarch 39/157 2023-03-29T10:54:59.107 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 57/157 2023-03-29T10:54:59.217 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : elfutils-libs-0.188-3.el8.x86_64 40/157 2023-03-29T10:54:59.339 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 58/157 2023-03-29T10:54:59.447 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : elfutils-debuginfod-client-0.188-3.el8.x86_64 41/157 2023-03-29T10:54:59.517 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 59/157 2023-03-29T10:54:59.632 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rbd-2:18.0.0-3066.gf03d69a8.el8.x86_64 42/157 2023-03-29T10:54:59.716 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 60/157 2023-03-29T10:54:59.819 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librados-devel-2:18.0.0-3066.gf03d69a8.el8.x86_6 43/157 2023-03-29T10:54:59.934 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 61/157 2023-03-29T10:54:59.999 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : libfdisk-2.32.1-41.el8.x86_64 44/157 2023-03-29T10:55:00.089 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 44/157 2023-03-29T10:55:00.129 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-portend-2.6-1.el8.noarch 62/157 2023-03-29T10:55:00.274 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : libmount-2.32.1-41.el8.x86_64 45/157 2023-03-29T10:55:00.348 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 45/157 2023-03-29T10:55:00.356 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 63/157 2023-03-29T10:55:00.568 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libuuid-devel-2.32.1-41.el8.x86_64 46/157 2023-03-29T10:55:00.621 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 64/157 2023-03-29T10:55:00.778 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 47/157 2023-03-29T10:55:00.882 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 65/157 2023-03-29T10:55:00.941 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : libsmartcols-2.32.1-41.el8.x86_64 48/157 2023-03-29T10:55:01.135 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 66/157 2023-03-29T10:55:01.487 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 67/157 2023-03-29T10:55:01.492 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 48/157 2023-03-29T10:55:01.800 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : util-linux-2.32.1-41.el8.x86_64 49/157 2023-03-29T10:55:01.804 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 68/157 2023-03-29T10:55:01.826 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: util-linux-2.32.1-41.el8.x86_64 49/157 2023-03-29T10:55:02.024 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 69/157 2023-03-29T10:55:02.048 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 50/157 2023-03-29T10:55:02.318 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 51/157 2023-03-29T10:55:02.540 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 52/157 2023-03-29T10:55:02.721 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 53/157 2023-03-29T10:55:02.903 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 54/157 2023-03-29T10:55:02.921 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 70/157 2023-03-29T10:55:03.126 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librgw2-2:18.0.0-3066.gf03d69a8.el8.x86_64 71/157 2023-03-29T10:55:03.170 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 55/157 2023-03-29T10:55:03.214 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librgw2-2:18.0.0-3066.gf03d69a8.el8.x86_64 71/157 2023-03-29T10:55:03.414 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 56/157 2023-03-29T10:55:03.478 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rgw-2:18.0.0-3066.gf03d69a8.el8.x86_64 72/157 2023-03-29T10:55:03.637 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 57/157 2023-03-29T10:55:03.703 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 73/157 2023-03-29T10:55:03.860 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 58/157 2023-03-29T10:55:04.031 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 59/157 2023-03-29T10:55:04.114 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 74/157 2023-03-29T10:55:04.238 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 60/157 2023-03-29T10:55:04.408 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 75/157 2023-03-29T10:55:04.456 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 61/157 2023-03-29T10:55:04.658 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-portend-2.6-1.el8.noarch 62/157 2023-03-29T10:55:04.674 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 76/157 2023-03-29T10:55:04.902 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 63/157 2023-03-29T10:55:04.936 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 77/157 2023-03-29T10:55:05.177 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 78/157 2023-03-29T10:55:05.242 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 64/157 2023-03-29T10:55:05.522 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 65/157 2023-03-29T10:55:05.765 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 66/157 2023-03-29T10:55:06.135 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 67/157 2023-03-29T10:55:06.434 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 68/157 2023-03-29T10:55:06.446 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 79/157 2023-03-29T10:55:06.687 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 69/157 2023-03-29T10:55:06.884 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 80/157 2023-03-29T10:55:07.186 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 81/157 2023-03-29T10:55:07.453 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 82/157 2023-03-29T10:55:07.574 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 70/157 2023-03-29T10:55:07.756 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : librgw2-2:18.0.0-3066.gf03d69a8.el8.x86_64 71/157 2023-03-29T10:55:07.833 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 83/157 2023-03-29T10:55:07.846 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librgw2-2:18.0.0-3066.gf03d69a8.el8.x86_64 71/157 2023-03-29T10:55:08.062 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 84/157 2023-03-29T10:55:08.117 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-rgw-2:18.0.0-3066.gf03d69a8.el8.x86_64 72/157 2023-03-29T10:55:08.289 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 85/157 2023-03-29T10:55:08.350 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 73/157 2023-03-29T10:55:08.506 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 86/157 2023-03-29T10:55:08.776 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 74/157 2023-03-29T10:55:08.812 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 87/157 2023-03-29T10:55:09.053 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 88/157 2023-03-29T10:55:09.055 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 75/157 2023-03-29T10:55:09.218 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 89/157 2023-03-29T10:55:09.296 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 76/157 2023-03-29T10:55:09.512 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 90/157 2023-03-29T10:55:09.592 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 77/157 2023-03-29T10:55:09.857 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 78/157 2023-03-29T10:55:10.147 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libselinux-devel-2.9-6.el8.x86_64 91/157 2023-03-29T10:55:10.300 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 92/157 2023-03-29T10:55:10.509 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 93/157 2023-03-29T10:55:10.712 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 94/157 2023-03-29T10:55:10.904 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fuse-2.9.7-16.el8.x86_64 95/157 2023-03-29T10:55:11.109 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 96/157 2023-03-29T10:55:11.152 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 79/157 2023-03-29T10:55:11.258 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : uuid-1.6.2-43.el8.x86_64 97/157 2023-03-29T10:55:11.333 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 97/157 2023-03-29T10:55:11.492 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 98/157 2023-03-29T10:55:11.630 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 80/157 2023-03-29T10:55:11.916 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 81/157 2023-03-29T10:55:12.191 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 82/157 2023-03-29T10:55:12.570 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 83/157 2023-03-29T10:55:12.850 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 84/157 2023-03-29T10:55:13.102 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 85/157 2023-03-29T10:55:13.311 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 86/157 2023-03-29T10:55:13.634 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 87/157 2023-03-29T10:55:13.841 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 88/157 2023-03-29T10:55:14.015 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 89/157 2023-03-29T10:55:14.356 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 90/157 2023-03-29T10:55:14.593 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libselinux-devel-2.9-6.el8.x86_64 91/157 2023-03-29T10:55:14.762 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 92/157 2023-03-29T10:55:14.996 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 93/157 2023-03-29T10:55:15.216 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 94/157 2023-03-29T10:55:15.416 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fuse-2.9.7-16.el8.x86_64 95/157 2023-03-29T10:55:15.629 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 96/157 2023-03-29T10:55:16.007 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:18.0.0-3066.gf03d69a8.el8.x86_64 99/157 2023-03-29T10:55:16.013 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : uuid-1.6.2-43.el8.x86_64 97/157 2023-03-29T10:55:16.101 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 97/157 2023-03-29T10:55:16.267 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-common-2:18.0.0-3066.gf03d69a8.el8.x86_64 99/157 2023-03-29T10:55:16.314 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 98/157 2023-03-29T10:55:16.933 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:18.0.0-3066.gf03d69a8.el8.x86_64 99/157 2023-03-29T10:55:17.270 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-base-2:18.0.0-3066.gf03d69a8.el8.x86_64 100/157 2023-03-29T10:55:17.470 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-base-2:18.0.0-3066.gf03d69a8.el8.x86_64 100/157 2023-03-29T10:55:17.633 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-selinux-2:18.0.0-3066.gf03d69a8.el8.x86_64 101/157 2023-03-29T10:55:20.874 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:18.0.0-3066.gf03d69a8.el8.x86_64 99/157 2023-03-29T10:55:21.963 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-common-2:18.0.0-3066.gf03d69a8.el8.x86_64 99/157 2023-03-29T10:55:22.635 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-common-2:18.0.0-3066.gf03d69a8.el8.x86_64 99/157 2023-03-29T10:55:22.982 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-base-2:18.0.0-3066.gf03d69a8.el8.x86_64 100/157 2023-03-29T10:55:23.161 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-base-2:18.0.0-3066.gf03d69a8.el8.x86_64 100/157 2023-03-29T10:55:23.337 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-selinux-2:18.0.0-3066.gf03d69a8.el8.x86_64 101/157 2023-03-29T10:55:38.251 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3066.gf03d69a8.el8.x86_64 101/157 2023-03-29T10:55:38.252 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /sys 2023-03-29T10:55:38.252 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /proc 2023-03-29T10:55:38.252 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /dev 2023-03-29T10:55:38.252 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /run 2023-03-29T10:55:38.252 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /mnt 2023-03-29T10:55:38.252 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /var/tmp 2023-03-29T10:55:38.253 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /home 2023-03-29T10:55:38.253 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /tmp 2023-03-29T10:55:38.253 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /dev 2023-03-29T10:55:38.253 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:55:38.663 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-volume-2:18.0.0-3066.gf03d69a8.el8.noarch 102/157 2023-03-29T10:55:40.369 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-volume-2:18.0.0-3066.gf03d69a8.el8.noarch 102/157 2023-03-29T10:55:40.566 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-osd-2:18.0.0-3066.gf03d69a8.el8.x86_64 103/157 2023-03-29T10:55:40.826 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-osd-2:18.0.0-3066.gf03d69a8.el8.x86_64 103/157 2023-03-29T10:55:41.013 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mds-2:18.0.0-3066.gf03d69a8.el8.x86_64 104/157 2023-03-29T10:55:41.527 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mds-2:18.0.0-3066.gf03d69a8.el8.x86_64 104/157 2023-03-29T10:55:41.729 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mon-2:18.0.0-3066.gf03d69a8.el8.x86_64 105/157 2023-03-29T10:55:41.774 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mon-2:18.0.0-3066.gf03d69a8.el8.x86_64 105/157 2023-03-29T10:55:41.996 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 106/157 2023-03-29T10:55:42.200 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 107/157 2023-03-29T10:55:42.362 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 108/157 2023-03-29T10:55:42.457 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 108/157 2023-03-29T10:55:42.727 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : jq-1.6-6.el8.x86_64 109/157 2023-03-29T10:55:42.979 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : automake-1.16.1-7.el8.noarch 110/157 2023-03-29T10:55:44.078 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3066.gf03d69a8.el8.x86_64 101/157 2023-03-29T10:55:44.078 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /sys 2023-03-29T10:55:44.078 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /proc 2023-03-29T10:55:44.078 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /dev 2023-03-29T10:55:44.078 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /run 2023-03-29T10:55:44.078 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /mnt 2023-03-29T10:55:44.078 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /var/tmp 2023-03-29T10:55:44.078 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /home 2023-03-29T10:55:44.078 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /tmp 2023-03-29T10:55:44.078 INFO:teuthology.orchestra.run.smithi111.stdout:skipping the directory /dev 2023-03-29T10:55:44.078 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:55:44.484 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-volume-2:18.0.0-3066.gf03d69a8.el8.noarch 102/157 2023-03-29T10:55:45.462 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:18.0.0-3066.gf03d69a8.el8.noarch 111/157 2023-03-29T10:55:45.618 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephadm-2:18.0.0-3066.gf03d69a8.el8.noarch 111/157 2023-03-29T10:55:45.728 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3066.gf03d69a8.e 112/157 2023-03-29T10:55:45.939 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3066.gf03d69a8. 113/157 2023-03-29T10:55:46.186 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3066.gf03d69a8.el8.noa 114/157 2023-03-29T10:55:46.208 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-volume-2:18.0.0-3066.gf03d69a8.el8.noarch 102/157 2023-03-29T10:55:46.437 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3066.gf03d69a8.el8.noa 114/157 2023-03-29T10:55:46.453 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-osd-2:18.0.0-3066.gf03d69a8.el8.x86_64 103/157 2023-03-29T10:55:46.689 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3066.gf03d69a8.el8.n 115/157 2023-03-29T10:55:46.716 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-osd-2:18.0.0-3066.gf03d69a8.el8.x86_64 103/157 2023-03-29T10:55:46.926 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mds-2:18.0.0-3066.gf03d69a8.el8.x86_64 104/157 2023-03-29T10:55:47.446 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mds-2:18.0.0-3066.gf03d69a8.el8.x86_64 104/157 2023-03-29T10:55:47.659 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mon-2:18.0.0-3066.gf03d69a8.el8.x86_64 105/157 2023-03-29T10:55:47.704 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mon-2:18.0.0-3066.gf03d69a8.el8.x86_64 105/157 2023-03-29T10:55:47.792 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3066.gf03d69a8.el8.n 115/157 2023-03-29T10:55:47.934 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 106/157 2023-03-29T10:55:47.986 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3066.gf03 116/157 2023-03-29T10:55:48.110 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3066.gf03 116/157 2023-03-29T10:55:48.138 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 107/157 2023-03-29T10:55:48.300 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 108/157 2023-03-29T10:55:48.400 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 108/157 2023-03-29T10:55:48.553 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3066.gf03d69a8.el 117/157 2023-03-29T10:55:48.692 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : jq-1.6-6.el8.x86_64 109/157 2023-03-29T10:55:48.797 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-2:18.0.0-3066.gf03d69a8.el8.x86_64 118/157 2023-03-29T10:55:48.857 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3066.gf03d69a8.el8.x86_64 118/157 2023-03-29T10:55:48.946 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : automake-1.16.1-7.el8.noarch 110/157 2023-03-29T10:55:49.044 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-rook-2:18.0.0-3066.gf03d69a8.el8.noarch 119/157 2023-03-29T10:55:49.068 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3066.gf03d69a8.el8.noarch 119/157 2023-03-29T10:55:49.268 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-2:18.0.0-3066.gf03d69a8.el8.x86_64 120/157 2023-03-29T10:55:49.452 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libtool-2.4.6-25.el8.x86_64 121/157 2023-03-29T10:55:51.360 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephadm-2:18.0.0-3066.gf03d69a8.el8.noarch 111/157 2023-03-29T10:55:51.547 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephadm-2:18.0.0-3066.gf03d69a8.el8.noarch 111/157 2023-03-29T10:55:51.666 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3066.gf03d69a8.e 112/157 2023-03-29T10:55:51.912 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3066.gf03d69a8. 113/157 2023-03-29T10:55:52.141 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3066.gf03d69a8.el8.noa 114/157 2023-03-29T10:55:52.395 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3066.gf03d69a8.el8.noa 114/157 2023-03-29T10:55:52.677 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3066.gf03d69a8.el8.n 115/157 2023-03-29T10:55:53.727 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 121/157 2023-03-29T10:55:53.793 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3066.gf03d69a8.el8.n 115/157 2023-03-29T10:55:53.991 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3066.gf03 116/157 2023-03-29T10:55:54.115 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3066.gf03 116/157 2023-03-29T10:55:54.539 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3066.gf03d69a8.el 117/157 2023-03-29T10:55:54.772 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-2:18.0.0-3066.gf03d69a8.el8.x86_64 118/157 2023-03-29T10:55:54.833 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3066.gf03d69a8.el8.x86_64 118/157 2023-03-29T10:55:54.999 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-mgr-rook-2:18.0.0-3066.gf03d69a8.el8.noarch 119/157 2023-03-29T10:55:55.024 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3066.gf03d69a8.el8.noarch 119/157 2023-03-29T10:55:55.224 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-2:18.0.0-3066.gf03d69a8.el8.x86_64 120/157 2023-03-29T10:55:55.235 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-test-2:18.0.0-3066.gf03d69a8.el8.x86_64 122/157 2023-03-29T10:55:55.399 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libtool-2.4.6-25.el8.x86_64 121/157 2023-03-29T10:55:55.479 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-radosgw-2:18.0.0-3066.gf03d69a8.el8.x86_64 123/157 2023-03-29T10:55:55.536 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3066.gf03d69a8.el8.x86_64 123/157 2023-03-29T10:55:55.752 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3066.gf03d6 124/157 2023-03-29T10:55:55.817 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3066.gf03d6 124/157 2023-03-29T10:55:56.035 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-mirror-2:18.0.0-3066.gf03d69a8.el8.x86_64 125/157 2023-03-29T10:55:56.436 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3066.gf03d69a8.el8.x86_64 125/157 2023-03-29T10:55:56.660 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-mirror-2:18.0.0-3066.gf03d69a8.el8.x86_64 126/157 2023-03-29T10:55:56.934 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3066.gf03d69a8.el8.x86_64 126/157 2023-03-29T10:55:57.280 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-contrib-10.23-1.module_el8+300+2c5300 127/157 2023-03-29T10:55:58.547 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-fuse-2:18.0.0-3066.gf03d69a8.el8.x86_64 128/157 2023-03-29T10:55:58.875 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 129/157 2023-03-29T10:55:59.064 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : util-linux-user-2.32.1-41.el8.x86_64 130/157 2023-03-29T10:55:59.231 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 131/157 2023-03-29T10:55:59.448 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs-devel-2:18.0.0-3066.gf03d69a8.el8.x86_ 132/157 2023-03-29T10:55:59.652 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : elfutils-libelf-devel-0.188-3.el8.x86_64 133/157 2023-03-29T10:55:59.786 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 121/157 2023-03-29T10:56:01.333 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-test-2:18.0.0-3066.gf03d69a8.el8.x86_64 122/157 2023-03-29T10:56:01.588 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-radosgw-2:18.0.0-3066.gf03d69a8.el8.x86_64 123/157 2023-03-29T10:56:01.645 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3066.gf03d69a8.el8.x86_64 123/157 2023-03-29T10:56:01.832 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3066.gf03d6 124/157 2023-03-29T10:56:01.898 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3066.gf03d6 124/157 2023-03-29T10:56:02.107 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-mirror-2:18.0.0-3066.gf03d69a8.el8.x86_64 125/157 2023-03-29T10:56:02.512 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3066.gf03d69a8.el8.x86_64 125/157 2023-03-29T10:56:02.690 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 134/157 2023-03-29T10:56:02.757 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-mirror-2:18.0.0-3066.gf03d69a8.el8.x86_64 126/157 2023-03-29T10:56:03.034 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3066.gf03d69a8.el8.x86_64 126/157 2023-03-29T10:56:03.034 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-server-10.23-1.module_el8+300+2c53004 134/157 2023-03-29T10:56:03.066 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 134/157 2023-03-29T10:56:03.318 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-top-2:18.0.0-3066.gf03d69a8.el8.noarch 135/157 2023-03-29T10:56:03.386 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-contrib-10.23-1.module_el8+300+2c5300 127/157 2023-03-29T10:56:03.518 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-fuse-2:18.0.0-3066.gf03d69a8.el8.x86_64 136/157 2023-03-29T10:56:03.813 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-nbd-2:18.0.0-3066.gf03d69a8.el8.x86_64 137/157 2023-03-29T10:56:04.069 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : openssl-1:1.1.1k-9.el8.x86_64 138/157 2023-03-29T10:56:04.216 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-1.el 139/157 2023-03-29T10:56:04.240 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-1.el 139/157 2023-03-29T10:56:04.457 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 140/157 2023-03-29T10:56:04.619 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ceph-fuse-2:18.0.0-3066.gf03d69a8.el8.x86_64 128/157 2023-03-29T10:56:04.650 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flex-2.6.1-9.el8.x86_64 141/157 2023-03-29T10:56:04.749 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 141/157 2023-03-29T10:56:04.958 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : bison-3.0.4-10.el8.x86_64 142/157 2023-03-29T10:56:04.972 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 129/157 2023-03-29T10:56:04.983 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 142/157 2023-03-29T10:56:05.185 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-1:1.1.1k-7.el8.x86_64 143/157 2023-03-29T10:56:05.202 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : util-linux-user-2.32.1-41.el8.x86_64 130/157 2023-03-29T10:56:05.343 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : elfutils-debuginfod-client-0.187-4.el8.x86_64 144/157 2023-03-29T10:56:05.403 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 131/157 2023-03-29T10:56:05.513 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : elfutils-libs-0.187-4.el8.x86_64 145/157 2023-03-29T10:56:05.514 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 146/157 2023-03-29T10:56:05.595 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libcephfs-devel-2:18.0.0-3066.gf03d69a8.el8.x86_ 132/157 2023-03-29T10:56:05.744 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 146/157 2023-03-29T10:56:05.757 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.39.12-1.el 147/157 2023-03-29T10:56:05.774 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : elfutils-libelf-devel-0.188-3.el8.x86_64 133/157 2023-03-29T10:56:05.856 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.39.12-1.el 147/157 2023-03-29T10:56:05.856 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 148/157 2023-03-29T10:56:06.094 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 148/157 2023-03-29T10:56:06.254 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : util-linux-user-2.32.1-38.el8.x86_64 149/157 2023-03-29T10:56:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : util-linux-2.32.1-38.el8.x86_64 150/157 2023-03-29T10:56:06.518 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libfdisk-2.32.1-38.el8.x86_64 151/157 2023-03-29T10:56:06.760 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libfdisk-2.32.1-38.el8.x86_64 151/157 2023-03-29T10:56:06.761 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libmount-2.32.1-38.el8.x86_64 152/157 2023-03-29T10:56:07.011 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libmount-2.32.1-38.el8.x86_64 152/157 2023-03-29T10:56:07.011 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libblkid-2.32.1-38.el8.x86_64 153/157 2023-03-29T10:56:07.219 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libblkid-2.32.1-38.el8.x86_64 153/157 2023-03-29T10:56:07.219 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libuuid-2.32.1-38.el8.x86_64 154/157 2023-03-29T10:56:07.436 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libuuid-2.32.1-38.el8.x86_64 154/157 2023-03-29T10:56:07.436 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libsmartcols-2.32.1-38.el8.x86_64 155/157 2023-03-29T10:56:07.619 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libsmartcols-2.32.1-38.el8.x86_64 155/157 2023-03-29T10:56:07.786 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : elfutils-libelf-0.187-4.el8.x86_64 156/157 2023-03-29T10:56:07.786 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : openssl-libs-1:1.1.1k-7.el8.x86_64 157/157 2023-03-29T10:56:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8.x86_64 157/157 2023-03-29T10:56:08.323 INFO:teuthology.orchestra.run.smithi119.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-03-29T10:56:08.323 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:08.801 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 134/157 2023-03-29T10:56:09.008 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-2:18.0.0-3066.gf03d69a8.el8.x86_64 1/157 2023-03-29T10:56:09.008 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-base-2:18.0.0-3066.gf03d69a8.el8.x86_64 2/157 2023-03-29T10:56:09.008 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-common-2:18.0.0-3066.gf03d69a8.el8.x86_64 3/157 2023-03-29T10:56:09.008 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-fuse-2:18.0.0-3066.gf03d69a8.el8.x86_64 4/157 2023-03-29T10:56:09.008 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3066.gf03d6 5/157 2023-03-29T10:56:09.009 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mds-2:18.0.0-3066.gf03d69a8.el8.x86_64 6/157 2023-03-29T10:56:09.009 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-2:18.0.0-3066.gf03d69a8.el8.x86_64 7/157 2023-03-29T10:56:09.009 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mon-2:18.0.0-3066.gf03d69a8.el8.x86_64 8/157 2023-03-29T10:56:09.009 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-osd-2:18.0.0-3066.gf03d69a8.el8.x86_64 9/157 2023-03-29T10:56:09.009 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-radosgw-2:18.0.0-3066.gf03d69a8.el8.x86_64 10/157 2023-03-29T10:56:09.009 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-selinux-2:18.0.0-3066.gf03d69a8.el8.x86_64 11/157 2023-03-29T10:56:09.009 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-test-2:18.0.0-3066.gf03d69a8.el8.x86_64 12/157 2023-03-29T10:56:09.009 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-mirror-2:18.0.0-3066.gf03d69a8.el8.x86_64 13/157 2023-03-29T10:56:09.009 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs-devel-2:18.0.0-3066.gf03d69a8.el8.x86_ 14/157 2023-03-29T10:56:09.009 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs2-2:18.0.0-3066.gf03d69a8.el8.x86_64 15/157 2023-03-29T10:56:09.009 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephsqlite-2:18.0.0-3066.gf03d69a8.el8.x86_64 16/157 2023-03-29T10:56:09.009 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados-devel-2:18.0.0-3066.gf03d69a8.el8.x86_6 17/157 2023-03-29T10:56:09.009 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libradosstriper1-2:18.0.0-3066.gf03d69a8.el8.x86 18/157 2023-03-29T10:56:09.010 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librgw2-2:18.0.0-3066.gf03d69a8.el8.x86_64 19/157 2023-03-29T10:56:09.010 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3066.gf03d69a8.el 20/157 2023-03-29T10:56:09.010 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-common-2:18.0.0-3066.gf03d69a8.el8. 21/157 2023-03-29T10:56:09.010 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cephfs-2:18.0.0-3066.gf03d69a8.el8.x86_6 22/157 2023-03-29T10:56:09.010 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rados-2:18.0.0-3066.gf03d69a8.el8.x86_64 23/157 2023-03-29T10:56:09.010 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rbd-2:18.0.0-3066.gf03d69a8.el8.x86_64 24/157 2023-03-29T10:56:09.010 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rgw-2:18.0.0-3066.gf03d69a8.el8.x86_64 25/157 2023-03-29T10:56:09.010 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-fuse-2:18.0.0-3066.gf03d69a8.el8.x86_64 26/157 2023-03-29T10:56:09.010 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-mirror-2:18.0.0-3066.gf03d69a8.el8.x86_64 27/157 2023-03-29T10:56:09.010 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-nbd-2:18.0.0-3066.gf03d69a8.el8.x86_64 28/157 2023-03-29T10:56:09.010 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3066.gf03d69a8. 29/157 2023-03-29T10:56:09.010 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3066.gf03d69a8.el8.noa 30/157 2023-03-29T10:56:09.010 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3066.gf03d69a8.el8.n 31/157 2023-03-29T10:56:09.010 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3066.gf03 32/157 2023-03-29T10:56:09.019 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3066.gf03d69a8.el 33/157 2023-03-29T10:56:09.019 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3066.gf03d69a8.el8.noarch 34/157 2023-03-29T10:56:09.019 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3066.gf03d69a8.e 35/157 2023-03-29T10:56:09.019 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-volume-2:18.0.0-3066.gf03d69a8.el8.noarch 36/157 2023-03-29T10:56:09.019 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephadm-2:18.0.0-3066.gf03d69a8.el8.noarch 37/157 2023-03-29T10:56:09.019 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-top-2:18.0.0-3066.gf03d69a8.el8.noarch 38/157 2023-03-29T10:56:09.019 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : automake-1.16.1-7.el8.noarch 39/157 2023-03-29T10:56:09.019 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/157 2023-03-29T10:56:09.019 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/157 2023-03-29T10:56:09.019 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : jq-1.6-6.el8.x86_64 42/157 2023-03-29T10:56:09.019 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libpq-13.5-1.el8.x86_64 43/157 2023-03-29T10:56:09.019 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 44/157 2023-03-29T10:56:09.019 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 45/157 2023-03-29T10:56:09.019 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 46/157 2023-03-29T10:56:09.019 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 47/157 2023-03-29T10:56:09.020 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 48/157 2023-03-29T10:56:09.020 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 49/157 2023-03-29T10:56:09.020 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 50/157 2023-03-29T10:56:09.020 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-contrib-10.23-1.module_el8+300+2c5300 51/157 2023-03-29T10:56:09.020 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-server-10.23-1.module_el8+300+2c53004 52/157 2023-03-29T10:56:09.020 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 53/157 2023-03-29T10:56:09.020 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 54/157 2023-03-29T10:56:09.020 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 55/157 2023-03-29T10:56:09.020 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 56/157 2023-03-29T10:56:09.020 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 57/157 2023-03-29T10:56:09.020 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 58/157 2023-03-29T10:56:09.020 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 59/157 2023-03-29T10:56:09.020 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 60/157 2023-03-29T10:56:09.020 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 61/157 2023-03-29T10:56:09.021 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/157 2023-03-29T10:56:09.021 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 63/157 2023-03-29T10:56:09.021 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 64/157 2023-03-29T10:56:09.021 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 65/157 2023-03-29T10:56:09.021 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 66/157 2023-03-29T10:56:09.021 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 67/157 2023-03-29T10:56:09.021 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 68/157 2023-03-29T10:56:09.021 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 69/157 2023-03-29T10:56:09.021 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 70/157 2023-03-29T10:56:09.022 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-devel-0.188-3.el8.x86_64 71/157 2023-03-29T10:56:09.022 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 72/157 2023-03-29T10:56:09.022 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 73/157 2023-03-29T10:56:09.022 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 74/157 2023-03-29T10:56:09.022 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 75/157 2023-03-29T10:56:09.022 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 76/157 2023-03-29T10:56:09.022 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 77/157 2023-03-29T10:56:09.022 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 78/157 2023-03-29T10:56:09.022 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 79/157 2023-03-29T10:56:09.022 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libselinux-devel-2.9-6.el8.x86_64 80/157 2023-03-29T10:56:09.022 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 81/157 2023-03-29T10:56:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-devel-2.32.1-41.el8.x86_64 82/157 2023-03-29T10:56:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 83/157 2023-03-29T10:56:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 84/157 2023-03-29T10:56:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 85/157 2023-03-29T10:56:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 86/157 2023-03-29T10:56:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 87/157 2023-03-29T10:56:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 88/157 2023-03-29T10:56:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 89/157 2023-03-29T10:56:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 90/157 2023-03-29T10:56:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 91/157 2023-03-29T10:56:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 92/157 2023-03-29T10:56:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 93/157 2023-03-29T10:56:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 94/157 2023-03-29T10:56:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : zlib-devel-1.2.11-19.el8.x86_64 95/157 2023-03-29T10:56:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 96/157 2023-03-29T10:56:09.023 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 97/157 2023-03-29T10:56:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 98/157 2023-03-29T10:56:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 99/157 2023-03-29T10:56:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 100/157 2023-03-29T10:56:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 101/157 2023-03-29T10:56:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 102/157 2023-03-29T10:56:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 103/157 2023-03-29T10:56:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 104/157 2023-03-29T10:56:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 105/157 2023-03-29T10:56:09.024 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 106/157 2023-03-29T10:56:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 107/157 2023-03-29T10:56:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 108/157 2023-03-29T10:56:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 109/157 2023-03-29T10:56:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 110/157 2023-03-29T10:56:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 111/157 2023-03-29T10:56:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 112/157 2023-03-29T10:56:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 113/157 2023-03-29T10:56:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-portend-2.6-1.el8.noarch 114/157 2023-03-29T10:56:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 115/157 2023-03-29T10:56:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 116/157 2023-03-29T10:56:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 117/157 2023-03-29T10:56:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 118/157 2023-03-29T10:56:09.025 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 119/157 2023-03-29T10:56:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 120/157 2023-03-29T10:56:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 121/157 2023-03-29T10:56:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 122/157 2023-03-29T10:56:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 123/157 2023-03-29T10:56:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 124/157 2023-03-29T10:56:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 125/157 2023-03-29T10:56:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 126/157 2023-03-29T10:56:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 127/157 2023-03-29T10:56:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:18.0.0-3066.gf03d69a8.el8.x86_64 128/157 2023-03-29T10:56:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 129/157 2023-03-29T10:56:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:18.0.0-3066.gf03d69a8.el8.x86_64 130/157 2023-03-29T10:56:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 131/157 2023-03-29T10:56:09.026 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-1.el 132/157 2023-03-29T10:56:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-initscripts-updown-1:1.39.12-1.el 133/157 2023-03-29T10:56:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-debuginfod-client-0.188-3.el8.x86_64 134/157 2023-03-29T10:56:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-debuginfod-client-0.187-4.el8.x86_64 135/157 2023-03-29T10:56:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-0.188-3.el8.x86_64 136/157 2023-03-29T10:56:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-0.187-4.el8.x86_64 137/157 2023-03-29T10:56:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libs-0.188-3.el8.x86_64 138/157 2023-03-29T10:56:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libs-0.187-4.el8.x86_64 139/157 2023-03-29T10:56:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 140/157 2023-03-29T10:56:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libblkid-2.32.1-38.el8.x86_64 141/157 2023-03-29T10:56:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 142/157 2023-03-29T10:56:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libfdisk-2.32.1-38.el8.x86_64 143/157 2023-03-29T10:56:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 144/157 2023-03-29T10:56:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libmount-2.32.1-38.el8.x86_64 145/157 2023-03-29T10:56:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 146/157 2023-03-29T10:56:09.027 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libsmartcols-2.32.1-38.el8.x86_64 147/157 2023-03-29T10:56:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 148/157 2023-03-29T10:56:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-2.32.1-38.el8.x86_64 149/157 2023-03-29T10:56:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:1.1.1k-9.el8.x86_64 150/157 2023-03-29T10:56:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-1:1.1.1k-7.el8.x86_64 151/157 2023-03-29T10:56:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8.x86_64 152/157 2023-03-29T10:56:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8.x86_64 153/157 2023-03-29T10:56:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 154/157 2023-03-29T10:56:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-2.32.1-38.el8.x86_64 155/157 2023-03-29T10:56:09.028 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 156/157 2023-03-29T10:56:09.114 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : postgresql-server-10.23-1.module_el8+300+2c53004 134/157 2023-03-29T10:56:09.147 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: postgresql-server-10.23-1.module_el8+300+2c53004 134/157 2023-03-29T10:56:09.398 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : cephfs-top-2:18.0.0-3066.gf03d69a8.el8.noarch 135/157 2023-03-29T10:56:09.640 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-fuse-2:18.0.0-3066.gf03d69a8.el8.x86_64 136/157 2023-03-29T10:56:09.950 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rbd-nbd-2:18.0.0-3066.gf03d69a8.el8.x86_64 137/157 2023-03-29T10:56:10.232 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : openssl-1:1.1.1k-9.el8.x86_64 138/157 2023-03-29T10:56:10.380 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-1.el 139/157 2023-03-29T10:56:10.402 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-1.el 139/157 2023-03-29T10:56:10.624 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 140/157 2023-03-29T10:56:10.838 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : flex-2.6.1-9.el8.x86_64 141/157 2023-03-29T10:56:10.941 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 141/157 2023-03-29T10:56:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : util-linux-user-2.32.1-38.el8.x86_64 157/157 2023-03-29T10:56:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:11.076 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2023-03-29T10:56:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-initscripts-updown-1:1.40.16-1.el8.noarch 2023-03-29T10:56:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-debuginfod-client-0.188-3.el8.x86_64 2023-03-29T10:56:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-0.188-3.el8.x86_64 2023-03-29T10:56:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libs-0.188-3.el8.x86_64 2023-03-29T10:56:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: libblkid-2.32.1-41.el8.x86_64 2023-03-29T10:56:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: libfdisk-2.32.1-41.el8.x86_64 2023-03-29T10:56:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: libmount-2.32.1-41.el8.x86_64 2023-03-29T10:56:11.076 INFO:teuthology.orchestra.run.smithi119.stdout: librados2-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: libsmartcols-2.32.1-41.el8.x86_64 2023-03-29T10:56:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-2.32.1-41.el8.x86_64 2023-03-29T10:56:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-1:1.1.1k-9.el8.x86_64 2023-03-29T10:56:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-libs-1:1.1.1k-9.el8.x86_64 2023-03-29T10:56:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-2.32.1-41.el8.x86_64 2023-03-29T10:56:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: util-linux-user-2.32.1-41.el8.x86_64 2023-03-29T10:56:11.077 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-03-29T10:56:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: automake-1.16.1-7.el8.noarch 2023-03-29T10:56:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: bison-3.0.4-10.el8.x86_64 2023-03-29T10:56:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.077 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel-0.188-3.el8.x86_64 2023-03-29T10:56:11.078 INFO:teuthology.orchestra.run.smithi119.stdout: flex-2.6.1-9.el8.x86_64 2023-03-29T10:56:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: fuse-2.9.7-16.el8.x86_64 2023-03-29T10:56:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-03-29T10:56:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: jq-1.6-6.el8.x86_64 2023-03-29T10:56:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-03-29T10:56:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-03-29T10:56:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-03-29T10:56:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-03-29T10:56:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-03-29T10:56:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: liboath-2.6.2-3.el8.x86_64 2023-03-29T10:56:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: libpq-13.5-1.el8.x86_64 2023-03-29T10:56:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-03-29T10:56:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-03-29T10:56:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.079 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-03-29T10:56:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: libselinux-devel-2.9-6.el8.x86_64 2023-03-29T10:56:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-03-29T10:56:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: libtool-2.4.6-25.el8.x86_64 2023-03-29T10:56:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind-1.3.1-3.el8.x86_64 2023-03-29T10:56:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel-2.32.1-41.el8.x86_64 2023-03-29T10:56:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-03-29T10:56:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-03-29T10:56:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-03-29T10:56:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-0.3.15-4.el8.x86_64 2023-03-29T10:56:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-03-29T10:56:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-03-29T10:56:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-03-29T10:56:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-03-29T10:56:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-03-29T10:56:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-10.23-1.module_el8+300+2c53004c.x86_64 2023-03-29T10:56:11.080 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib-10.23-1.module_el8+300+2c53004c.x86_64 2023-03-29T10:56:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server-10.23-1.module_el8+300+2c53004c.x86_64 2023-03-29T10:56:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-03-29T10:56:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel-2.5.1-7.el8.noarch 2023-03-29T10:56:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-03-29T10:56:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-03-29T10:56:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-03-29T10:56:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-03-29T10:56:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-03-29T10:56:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-03-29T10:56:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-03-29T10:56:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-03-29T10:56:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-03-29T10:56:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-6.2-6.el8.noarch 2023-03-29T10:56:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-03-29T10:56:11.081 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-03-29T10:56:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-03-29T10:56:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-03-29T10:56:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-03-29T10:56:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-03-29T10:56:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-03-29T10:56:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako-1.0.6-14.el8.noarch 2023-03-29T10:56:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-03-29T10:56:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-03-29T10:56:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-03-29T10:56:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-03-29T10:56:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-03-29T10:56:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-03-29T10:56:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-03-29T10:56:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend-2.6-1.el8.noarch 2023-03-29T10:56:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-03-29T10:56:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-03-29T10:56:11.082 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-03-29T10:56:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-03-29T10:56:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser-2.14-14.el8.noarch 2023-03-29T10:56:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz-2017.2-9.el8.noarch 2023-03-29T10:56:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-03-29T10:56:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-03-29T10:56:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes-2.4.1-12.el8.noarch 2023-03-29T10:56:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa-4.8-1.el8.noarch 2023-03-29T10:56:11.083 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-03-29T10:56:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-03-29T10:56:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-03-29T10:56:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-03-29T10:56:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-03-29T10:56:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-03-29T10:56:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-03-29T10:56:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-03-29T10:56:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-03-29T10:56:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-03-29T10:56:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-03-29T10:56:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: socat-1.7.4.1-1.el8.x86_64 2023-03-29T10:56:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: thrift-0.13.0-2.el8.x86_64 2023-03-29T10:56:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: uuid-1.6.2-43.el8.x86_64 2023-03-29T10:56:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-03-29T10:56:11.084 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-03-29T10:56:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel-1.2.11-19.el8.x86_64 2023-03-29T10:56:11.085 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:11.085 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-03-29T10:56:11.163 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : bison-3.0.4-10.el8.x86_64 142/157 2023-03-29T10:56:11.189 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 142/157 2023-03-29T10:56:11.369 DEBUG:teuthology.parallel:result is None 2023-03-29T10:56:11.406 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-1:1.1.1k-7.el8.x86_64 143/157 2023-03-29T10:56:11.573 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : elfutils-debuginfod-client-0.187-4.el8.x86_64 144/157 2023-03-29T10:56:11.780 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : elfutils-libs-0.187-4.el8.x86_64 145/157 2023-03-29T10:56:11.780 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 146/157 2023-03-29T10:56:12.032 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 146/157 2023-03-29T10:56:12.046 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.39.12-1.el 147/157 2023-03-29T10:56:12.161 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.39.12-1.el 147/157 2023-03-29T10:56:12.161 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 148/157 2023-03-29T10:56:12.458 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 148/157 2023-03-29T10:56:12.652 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : util-linux-user-2.32.1-38.el8.x86_64 149/157 2023-03-29T10:56:12.956 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : util-linux-2.32.1-38.el8.x86_64 150/157 2023-03-29T10:56:12.957 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : libfdisk-2.32.1-38.el8.x86_64 151/157 2023-03-29T10:56:13.208 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libfdisk-2.32.1-38.el8.x86_64 151/157 2023-03-29T10:56:13.208 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : libmount-2.32.1-38.el8.x86_64 152/157 2023-03-29T10:56:13.449 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libmount-2.32.1-38.el8.x86_64 152/157 2023-03-29T10:56:13.450 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : libblkid-2.32.1-38.el8.x86_64 153/157 2023-03-29T10:56:13.683 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libblkid-2.32.1-38.el8.x86_64 153/157 2023-03-29T10:56:13.683 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : libuuid-2.32.1-38.el8.x86_64 154/157 2023-03-29T10:56:13.916 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libuuid-2.32.1-38.el8.x86_64 154/157 2023-03-29T10:56:13.916 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : libsmartcols-2.32.1-38.el8.x86_64 155/157 2023-03-29T10:56:14.124 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libsmartcols-2.32.1-38.el8.x86_64 155/157 2023-03-29T10:56:14.317 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : elfutils-libelf-0.187-4.el8.x86_64 156/157 2023-03-29T10:56:14.317 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : openssl-libs-1:1.1.1k-7.el8.x86_64 157/157 2023-03-29T10:56:14.846 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8.x86_64 157/157 2023-03-29T10:56:14.846 INFO:teuthology.orchestra.run.smithi111.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-03-29T10:56:14.846 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:15.525 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-2:18.0.0-3066.gf03d69a8.el8.x86_64 1/157 2023-03-29T10:56:15.525 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-base-2:18.0.0-3066.gf03d69a8.el8.x86_64 2/157 2023-03-29T10:56:15.525 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-common-2:18.0.0-3066.gf03d69a8.el8.x86_64 3/157 2023-03-29T10:56:15.525 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-fuse-2:18.0.0-3066.gf03d69a8.el8.x86_64 4/157 2023-03-29T10:56:15.525 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3066.gf03d6 5/157 2023-03-29T10:56:15.525 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mds-2:18.0.0-3066.gf03d69a8.el8.x86_64 6/157 2023-03-29T10:56:15.525 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-2:18.0.0-3066.gf03d69a8.el8.x86_64 7/157 2023-03-29T10:56:15.525 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mon-2:18.0.0-3066.gf03d69a8.el8.x86_64 8/157 2023-03-29T10:56:15.526 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-osd-2:18.0.0-3066.gf03d69a8.el8.x86_64 9/157 2023-03-29T10:56:15.526 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-radosgw-2:18.0.0-3066.gf03d69a8.el8.x86_64 10/157 2023-03-29T10:56:15.526 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-selinux-2:18.0.0-3066.gf03d69a8.el8.x86_64 11/157 2023-03-29T10:56:15.526 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-test-2:18.0.0-3066.gf03d69a8.el8.x86_64 12/157 2023-03-29T10:56:15.526 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-mirror-2:18.0.0-3066.gf03d69a8.el8.x86_64 13/157 2023-03-29T10:56:15.526 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs-devel-2:18.0.0-3066.gf03d69a8.el8.x86_ 14/157 2023-03-29T10:56:15.526 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephfs2-2:18.0.0-3066.gf03d69a8.el8.x86_64 15/157 2023-03-29T10:56:15.526 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcephsqlite-2:18.0.0-3066.gf03d69a8.el8.x86_64 16/157 2023-03-29T10:56:15.526 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados-devel-2:18.0.0-3066.gf03d69a8.el8.x86_6 17/157 2023-03-29T10:56:15.526 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libradosstriper1-2:18.0.0-3066.gf03d69a8.el8.x86 18/157 2023-03-29T10:56:15.526 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librgw2-2:18.0.0-3066.gf03d69a8.el8.x86_64 19/157 2023-03-29T10:56:15.526 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3066.gf03d69a8.el 20/157 2023-03-29T10:56:15.527 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-ceph-common-2:18.0.0-3066.gf03d69a8.el8. 21/157 2023-03-29T10:56:15.527 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cephfs-2:18.0.0-3066.gf03d69a8.el8.x86_6 22/157 2023-03-29T10:56:15.527 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rados-2:18.0.0-3066.gf03d69a8.el8.x86_64 23/157 2023-03-29T10:56:15.527 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rbd-2:18.0.0-3066.gf03d69a8.el8.x86_64 24/157 2023-03-29T10:56:15.527 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rgw-2:18.0.0-3066.gf03d69a8.el8.x86_64 25/157 2023-03-29T10:56:15.527 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-fuse-2:18.0.0-3066.gf03d69a8.el8.x86_64 26/157 2023-03-29T10:56:15.527 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-mirror-2:18.0.0-3066.gf03d69a8.el8.x86_64 27/157 2023-03-29T10:56:15.527 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rbd-nbd-2:18.0.0-3066.gf03d69a8.el8.x86_64 28/157 2023-03-29T10:56:15.527 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3066.gf03d69a8. 29/157 2023-03-29T10:56:15.527 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3066.gf03d69a8.el8.noa 30/157 2023-03-29T10:56:15.536 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3066.gf03d69a8.el8.n 31/157 2023-03-29T10:56:15.536 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3066.gf03 32/157 2023-03-29T10:56:15.536 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3066.gf03d69a8.el 33/157 2023-03-29T10:56:15.536 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3066.gf03d69a8.el8.noarch 34/157 2023-03-29T10:56:15.536 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3066.gf03d69a8.e 35/157 2023-03-29T10:56:15.536 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ceph-volume-2:18.0.0-3066.gf03d69a8.el8.noarch 36/157 2023-03-29T10:56:15.536 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephadm-2:18.0.0-3066.gf03d69a8.el8.noarch 37/157 2023-03-29T10:56:15.537 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cephfs-top-2:18.0.0-3066.gf03d69a8.el8.noarch 38/157 2023-03-29T10:56:15.537 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : automake-1.16.1-7.el8.noarch 39/157 2023-03-29T10:56:15.537 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/157 2023-03-29T10:56:15.537 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/157 2023-03-29T10:56:15.537 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : jq-1.6-6.el8.x86_64 42/157 2023-03-29T10:56:15.537 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libpq-13.5-1.el8.x86_64 43/157 2023-03-29T10:56:15.537 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 44/157 2023-03-29T10:56:15.537 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 45/157 2023-03-29T10:56:15.537 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 46/157 2023-03-29T10:56:15.537 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 47/157 2023-03-29T10:56:15.537 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 48/157 2023-03-29T10:56:15.538 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 49/157 2023-03-29T10:56:15.538 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-10.23-1.module_el8+300+2c53004c.x86_6 50/157 2023-03-29T10:56:15.538 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-contrib-10.23-1.module_el8+300+2c5300 51/157 2023-03-29T10:56:15.538 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : postgresql-server-10.23-1.module_el8+300+2c53004 52/157 2023-03-29T10:56:15.538 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 53/157 2023-03-29T10:56:15.538 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 54/157 2023-03-29T10:56:15.538 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 55/157 2023-03-29T10:56:15.538 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 56/157 2023-03-29T10:56:15.539 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 57/157 2023-03-29T10:56:15.539 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 58/157 2023-03-29T10:56:15.539 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 59/157 2023-03-29T10:56:15.539 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 60/157 2023-03-29T10:56:15.539 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 61/157 2023-03-29T10:56:15.539 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/157 2023-03-29T10:56:15.539 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 63/157 2023-03-29T10:56:15.539 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 64/157 2023-03-29T10:56:15.539 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 65/157 2023-03-29T10:56:15.539 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 66/157 2023-03-29T10:56:15.539 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 67/157 2023-03-29T10:56:15.539 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 68/157 2023-03-29T10:56:15.540 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 69/157 2023-03-29T10:56:15.540 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 70/157 2023-03-29T10:56:15.540 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-devel-0.188-3.el8.x86_64 71/157 2023-03-29T10:56:15.540 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 72/157 2023-03-29T10:56:15.540 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 73/157 2023-03-29T10:56:15.540 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 74/157 2023-03-29T10:56:15.540 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 75/157 2023-03-29T10:56:15.540 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 76/157 2023-03-29T10:56:15.540 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 77/157 2023-03-29T10:56:15.540 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 78/157 2023-03-29T10:56:15.540 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 79/157 2023-03-29T10:56:15.540 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libselinux-devel-2.9-6.el8.x86_64 80/157 2023-03-29T10:56:15.540 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 81/157 2023-03-29T10:56:15.541 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libuuid-devel-2.32.1-41.el8.x86_64 82/157 2023-03-29T10:56:15.541 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 83/157 2023-03-29T10:56:15.541 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 84/157 2023-03-29T10:56:15.541 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 85/157 2023-03-29T10:56:15.541 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 86/157 2023-03-29T10:56:15.541 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 87/157 2023-03-29T10:56:15.541 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 88/157 2023-03-29T10:56:15.541 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 89/157 2023-03-29T10:56:15.541 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 90/157 2023-03-29T10:56:15.541 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 91/157 2023-03-29T10:56:15.541 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 92/157 2023-03-29T10:56:15.541 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 93/157 2023-03-29T10:56:15.541 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 94/157 2023-03-29T10:56:15.541 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : zlib-devel-1.2.11-19.el8.x86_64 95/157 2023-03-29T10:56:15.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 96/157 2023-03-29T10:56:15.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 97/157 2023-03-29T10:56:15.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 98/157 2023-03-29T10:56:15.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 99/157 2023-03-29T10:56:15.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 100/157 2023-03-29T10:56:15.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 101/157 2023-03-29T10:56:15.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 102/157 2023-03-29T10:56:15.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 103/157 2023-03-29T10:56:15.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 104/157 2023-03-29T10:56:15.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 105/157 2023-03-29T10:56:15.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 106/157 2023-03-29T10:56:15.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 107/157 2023-03-29T10:56:15.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 108/157 2023-03-29T10:56:15.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 109/157 2023-03-29T10:56:15.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 110/157 2023-03-29T10:56:15.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 111/157 2023-03-29T10:56:15.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 112/157 2023-03-29T10:56:15.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 113/157 2023-03-29T10:56:15.542 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-portend-2.6-1.el8.noarch 114/157 2023-03-29T10:56:15.543 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 115/157 2023-03-29T10:56:15.543 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 116/157 2023-03-29T10:56:15.543 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 117/157 2023-03-29T10:56:15.543 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 118/157 2023-03-29T10:56:15.543 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 119/157 2023-03-29T10:56:15.543 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 120/157 2023-03-29T10:56:15.543 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 121/157 2023-03-29T10:56:15.543 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 122/157 2023-03-29T10:56:15.543 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 123/157 2023-03-29T10:56:15.543 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 124/157 2023-03-29T10:56:15.543 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 125/157 2023-03-29T10:56:15.543 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 126/157 2023-03-29T10:56:15.543 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 127/157 2023-03-29T10:56:15.543 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-2:18.0.0-3066.gf03d69a8.el8.x86_64 128/157 2023-03-29T10:56:15.543 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 129/157 2023-03-29T10:56:15.543 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-2:18.0.0-3066.gf03d69a8.el8.x86_64 130/157 2023-03-29T10:56:15.544 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 131/157 2023-03-29T10:56:15.544 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-1.el 132/157 2023-03-29T10:56:15.544 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : NetworkManager-initscripts-updown-1:1.39.12-1.el 133/157 2023-03-29T10:56:15.544 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-debuginfod-client-0.188-3.el8.x86_64 134/157 2023-03-29T10:56:15.544 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-debuginfod-client-0.187-4.el8.x86_64 135/157 2023-03-29T10:56:15.544 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-0.188-3.el8.x86_64 136/157 2023-03-29T10:56:15.544 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libelf-0.187-4.el8.x86_64 137/157 2023-03-29T10:56:15.544 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libs-0.188-3.el8.x86_64 138/157 2023-03-29T10:56:15.544 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : elfutils-libs-0.187-4.el8.x86_64 139/157 2023-03-29T10:56:15.544 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 140/157 2023-03-29T10:56:15.544 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libblkid-2.32.1-38.el8.x86_64 141/157 2023-03-29T10:56:15.544 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 142/157 2023-03-29T10:56:15.544 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libfdisk-2.32.1-38.el8.x86_64 143/157 2023-03-29T10:56:15.544 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 144/157 2023-03-29T10:56:15.544 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libmount-2.32.1-38.el8.x86_64 145/157 2023-03-29T10:56:15.544 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 146/157 2023-03-29T10:56:15.544 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libsmartcols-2.32.1-38.el8.x86_64 147/157 2023-03-29T10:56:15.544 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 148/157 2023-03-29T10:56:15.545 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libuuid-2.32.1-38.el8.x86_64 149/157 2023-03-29T10:56:15.545 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:1.1.1k-9.el8.x86_64 150/157 2023-03-29T10:56:15.545 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-1:1.1.1k-7.el8.x86_64 151/157 2023-03-29T10:56:15.545 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8.x86_64 152/157 2023-03-29T10:56:15.545 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8.x86_64 153/157 2023-03-29T10:56:15.545 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 154/157 2023-03-29T10:56:15.545 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : util-linux-2.32.1-38.el8.x86_64 155/157 2023-03-29T10:56:15.545 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 156/157 2023-03-29T10:56:17.588 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : util-linux-user-2.32.1-38.el8.x86_64 157/157 2023-03-29T10:56:17.588 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:17.588 INFO:teuthology.orchestra.run.smithi111.stdout:Upgraded: 2023-03-29T10:56:17.588 INFO:teuthology.orchestra.run.smithi111.stdout: NetworkManager-initscripts-updown-1:1.40.16-1.el8.noarch 2023-03-29T10:56:17.588 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-debuginfod-client-0.188-3.el8.x86_64 2023-03-29T10:56:17.588 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-0.188-3.el8.x86_64 2023-03-29T10:56:17.588 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libs-0.188-3.el8.x86_64 2023-03-29T10:56:17.588 INFO:teuthology.orchestra.run.smithi111.stdout: libblkid-2.32.1-41.el8.x86_64 2023-03-29T10:56:17.588 INFO:teuthology.orchestra.run.smithi111.stdout: libfdisk-2.32.1-41.el8.x86_64 2023-03-29T10:56:17.588 INFO:teuthology.orchestra.run.smithi111.stdout: libmount-2.32.1-41.el8.x86_64 2023-03-29T10:56:17.588 INFO:teuthology.orchestra.run.smithi111.stdout: librados2-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.588 INFO:teuthology.orchestra.run.smithi111.stdout: librbd1-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.588 INFO:teuthology.orchestra.run.smithi111.stdout: libsmartcols-2.32.1-41.el8.x86_64 2023-03-29T10:56:17.588 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-2.32.1-41.el8.x86_64 2023-03-29T10:56:17.589 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-1:1.1.1k-9.el8.x86_64 2023-03-29T10:56:17.589 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-libs-1:1.1.1k-9.el8.x86_64 2023-03-29T10:56:17.589 INFO:teuthology.orchestra.run.smithi111.stdout: util-linux-2.32.1-41.el8.x86_64 2023-03-29T10:56:17.589 INFO:teuthology.orchestra.run.smithi111.stdout: util-linux-user-2.32.1-41.el8.x86_64 2023-03-29T10:56:17.589 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2023-03-29T10:56:17.589 INFO:teuthology.orchestra.run.smithi111.stdout: automake-1.16.1-7.el8.noarch 2023-03-29T10:56:17.589 INFO:teuthology.orchestra.run.smithi111.stdout: bison-3.0.4-10.el8.x86_64 2023-03-29T10:56:17.589 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.589 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-base-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.589 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-common-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.589 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-fuse-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.589 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-grafana-dashboards-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:17.589 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-immutable-object-cache-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.589 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mds-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.589 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.589 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-cephadm-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:17.589 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-dashboard-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:17.589 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:17.590 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-modules-core-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:17.590 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mgr-rook-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:17.590 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-mon-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.590 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-osd-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.590 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-prometheus-alerts-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:17.590 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-radosgw-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.590 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-selinux-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.590 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-test-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.590 INFO:teuthology.orchestra.run.smithi111.stdout: ceph-volume-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:17.590 INFO:teuthology.orchestra.run.smithi111.stdout: cephadm-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:17.590 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-mirror-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.590 INFO:teuthology.orchestra.run.smithi111.stdout: cephfs-top-2:18.0.0-3066.gf03d69a8.el8.noarch 2023-03-29T10:56:17.590 INFO:teuthology.orchestra.run.smithi111.stdout: elfutils-libelf-devel-0.188-3.el8.x86_64 2023-03-29T10:56:17.590 INFO:teuthology.orchestra.run.smithi111.stdout: flex-2.6.1-9.el8.x86_64 2023-03-29T10:56:17.590 INFO:teuthology.orchestra.run.smithi111.stdout: fuse-2.9.7-16.el8.x86_64 2023-03-29T10:56:17.590 INFO:teuthology.orchestra.run.smithi111.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-03-29T10:56:17.590 INFO:teuthology.orchestra.run.smithi111.stdout: jq-1.6-6.el8.x86_64 2023-03-29T10:56:17.590 INFO:teuthology.orchestra.run.smithi111.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-03-29T10:56:17.591 INFO:teuthology.orchestra.run.smithi111.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-03-29T10:56:17.591 INFO:teuthology.orchestra.run.smithi111.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-03-29T10:56:17.591 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs-devel-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.591 INFO:teuthology.orchestra.run.smithi111.stdout: libcephfs2-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.591 INFO:teuthology.orchestra.run.smithi111.stdout: libcephsqlite-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.591 INFO:teuthology.orchestra.run.smithi111.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-03-29T10:56:17.591 INFO:teuthology.orchestra.run.smithi111.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-03-29T10:56:17.591 INFO:teuthology.orchestra.run.smithi111.stdout: liboath-2.6.2-3.el8.x86_64 2023-03-29T10:56:17.591 INFO:teuthology.orchestra.run.smithi111.stdout: libpq-13.5-1.el8.x86_64 2023-03-29T10:56:17.591 INFO:teuthology.orchestra.run.smithi111.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-03-29T10:56:17.591 INFO:teuthology.orchestra.run.smithi111.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-03-29T10:56:17.591 INFO:teuthology.orchestra.run.smithi111.stdout: librados-devel-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.591 INFO:teuthology.orchestra.run.smithi111.stdout: libradosstriper1-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.591 INFO:teuthology.orchestra.run.smithi111.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-03-29T10:56:17.591 INFO:teuthology.orchestra.run.smithi111.stdout: librgw2-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.591 INFO:teuthology.orchestra.run.smithi111.stdout: libselinux-devel-2.9-6.el8.x86_64 2023-03-29T10:56:17.591 INFO:teuthology.orchestra.run.smithi111.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-03-29T10:56:17.592 INFO:teuthology.orchestra.run.smithi111.stdout: libtool-2.4.6-25.el8.x86_64 2023-03-29T10:56:17.592 INFO:teuthology.orchestra.run.smithi111.stdout: libunwind-1.3.1-3.el8.x86_64 2023-03-29T10:56:17.592 INFO:teuthology.orchestra.run.smithi111.stdout: libuuid-devel-2.32.1-41.el8.x86_64 2023-03-29T10:56:17.592 INFO:teuthology.orchestra.run.smithi111.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-03-29T10:56:17.592 INFO:teuthology.orchestra.run.smithi111.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-03-29T10:56:17.592 INFO:teuthology.orchestra.run.smithi111.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-03-29T10:56:17.592 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-0.3.15-4.el8.x86_64 2023-03-29T10:56:17.592 INFO:teuthology.orchestra.run.smithi111.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-03-29T10:56:17.592 INFO:teuthology.orchestra.run.smithi111.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-03-29T10:56:17.592 INFO:teuthology.orchestra.run.smithi111.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-03-29T10:56:17.592 INFO:teuthology.orchestra.run.smithi111.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-03-29T10:56:17.592 INFO:teuthology.orchestra.run.smithi111.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-03-29T10:56:17.592 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-10.23-1.module_el8+300+2c53004c.x86_64 2023-03-29T10:56:17.592 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-contrib-10.23-1.module_el8+300+2c53004c.x86_64 2023-03-29T10:56:17.592 INFO:teuthology.orchestra.run.smithi111.stdout: postgresql-server-10.23-1.module_el8+300+2c53004c.x86_64 2023-03-29T10:56:17.592 INFO:teuthology.orchestra.run.smithi111.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-03-29T10:56:17.592 INFO:teuthology.orchestra.run.smithi111.stdout: python3-babel-2.5.1-7.el8.noarch 2023-03-29T10:56:17.593 INFO:teuthology.orchestra.run.smithi111.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-03-29T10:56:17.593 INFO:teuthology.orchestra.run.smithi111.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-03-29T10:56:17.593 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-03-29T10:56:17.593 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-argparse-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.593 INFO:teuthology.orchestra.run.smithi111.stdout: python3-ceph-common-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.593 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cephfs-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.593 INFO:teuthology.orchestra.run.smithi111.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-03-29T10:56:17.593 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-03-29T10:56:17.593 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-03-29T10:56:17.593 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-03-29T10:56:17.593 INFO:teuthology.orchestra.run.smithi111.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-03-29T10:56:17.593 INFO:teuthology.orchestra.run.smithi111.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-03-29T10:56:17.593 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-6.2-6.el8.noarch 2023-03-29T10:56:17.593 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-03-29T10:56:17.593 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-03-29T10:56:17.593 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-03-29T10:56:17.593 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-03-29T10:56:17.593 INFO:teuthology.orchestra.run.smithi111.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-03-29T10:56:17.594 INFO:teuthology.orchestra.run.smithi111.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-03-29T10:56:17.594 INFO:teuthology.orchestra.run.smithi111.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-03-29T10:56:17.594 INFO:teuthology.orchestra.run.smithi111.stdout: python3-mako-1.0.6-14.el8.noarch 2023-03-29T10:56:17.594 INFO:teuthology.orchestra.run.smithi111.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-03-29T10:56:17.594 INFO:teuthology.orchestra.run.smithi111.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-03-29T10:56:17.594 INFO:teuthology.orchestra.run.smithi111.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-03-29T10:56:17.594 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-03-29T10:56:17.594 INFO:teuthology.orchestra.run.smithi111.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-03-29T10:56:17.594 INFO:teuthology.orchestra.run.smithi111.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-03-29T10:56:17.594 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-03-29T10:56:17.594 INFO:teuthology.orchestra.run.smithi111.stdout: python3-portend-2.6-1.el8.noarch 2023-03-29T10:56:17.594 INFO:teuthology.orchestra.run.smithi111.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-03-29T10:56:17.594 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-03-29T10:56:17.594 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-03-29T10:56:17.594 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-03-29T10:56:17.594 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pycparser-2.14-14.el8.noarch 2023-03-29T10:56:17.594 INFO:teuthology.orchestra.run.smithi111.stdout: python3-pytz-2017.2-9.el8.noarch 2023-03-29T10:56:17.595 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rados-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.595 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rbd-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.595 INFO:teuthology.orchestra.run.smithi111.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-03-29T10:56:17.595 INFO:teuthology.orchestra.run.smithi111.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-03-29T10:56:17.595 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rgw-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.595 INFO:teuthology.orchestra.run.smithi111.stdout: python3-routes-2.4.1-12.el8.noarch 2023-03-29T10:56:17.595 INFO:teuthology.orchestra.run.smithi111.stdout: python3-rsa-4.8-1.el8.noarch 2023-03-29T10:56:17.595 INFO:teuthology.orchestra.run.smithi111.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-03-29T10:56:17.595 INFO:teuthology.orchestra.run.smithi111.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-03-29T10:56:17.595 INFO:teuthology.orchestra.run.smithi111.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-03-29T10:56:17.595 INFO:teuthology.orchestra.run.smithi111.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-03-29T10:56:17.595 INFO:teuthology.orchestra.run.smithi111.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-03-29T10:56:17.595 INFO:teuthology.orchestra.run.smithi111.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-03-29T10:56:17.595 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-03-29T10:56:17.595 INFO:teuthology.orchestra.run.smithi111.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-03-29T10:56:17.595 INFO:teuthology.orchestra.run.smithi111.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-03-29T10:56:17.595 INFO:teuthology.orchestra.run.smithi111.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-03-29T10:56:17.595 INFO:teuthology.orchestra.run.smithi111.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-03-29T10:56:17.596 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-fuse-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.596 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-mirror-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.596 INFO:teuthology.orchestra.run.smithi111.stdout: rbd-nbd-2:18.0.0-3066.gf03d69a8.el8.x86_64 2023-03-29T10:56:17.596 INFO:teuthology.orchestra.run.smithi111.stdout: socat-1.7.4.1-1.el8.x86_64 2023-03-29T10:56:17.596 INFO:teuthology.orchestra.run.smithi111.stdout: thrift-0.13.0-2.el8.x86_64 2023-03-29T10:56:17.596 INFO:teuthology.orchestra.run.smithi111.stdout: uuid-1.6.2-43.el8.x86_64 2023-03-29T10:56:17.596 INFO:teuthology.orchestra.run.smithi111.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-03-29T10:56:17.596 INFO:teuthology.orchestra.run.smithi111.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-03-29T10:56:17.596 INFO:teuthology.orchestra.run.smithi111.stdout: zlib-devel-1.2.11-19.el8.x86_64 2023-03-29T10:56:17.596 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:17.596 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2023-03-29T10:56:19.217 DEBUG:teuthology.parallel:result is None 2023-03-29T10:56:19.218 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f03d69a8741c605ae6711afe2c9991ac46095f7d 2023-03-29T10:56:19.354 DEBUG:teuthology.orchestra.run.smithi111:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-03-29T10:56:19.404 INFO:teuthology.orchestra.run.smithi111.stdout:18.0.0-3066.gf03d69a8.el8 2023-03-29T10:56:19.404 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3066.gf03d69a8.el8 2023-03-29T10:56:19.404 INFO:teuthology.task.install:The correct ceph version 18.0.0-3066.gf03d69a8 is installed. 2023-03-29T10:56:19.405 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=f03d69a8741c605ae6711afe2c9991ac46095f7d 2023-03-29T10:56:19.545 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-03-29T10:56:19.600 INFO:teuthology.orchestra.run.smithi119.stdout:18.0.0-3066.gf03d69a8.el8 2023-03-29T10:56:19.601 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3066.gf03d69a8.el8 2023-03-29T10:56:19.601 INFO:teuthology.task.install:The correct ceph version 18.0.0-3066.gf03d69a8 is installed. 2023-03-29T10:56:19.601 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-03-29T10:56:19.601 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:56:19.601 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-03-29T10:56:19.667 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:56:19.667 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-03-29T10:56:19.737 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-03-29T10:56:19.738 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:56:19.738 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/daemon-helper 2023-03-29T10:56:19.801 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-03-29T10:56:19.905 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:56:19.905 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2023-03-29T10:56:19.970 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-03-29T10:56:20.071 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-03-29T10:56:20.072 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:56:20.072 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/usr/bin/adjust-ulimits 2023-03-29T10:56:20.135 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-03-29T10:56:20.237 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:56:20.237 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2023-03-29T10:56:20.297 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-03-29T10:56:20.398 INFO:teuthology.run_tasks:Running task ceph... 2023-03-29T10:56:20.464 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-03-29T10:56:20.464 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 777 /var/log/ceph 2023-03-29T10:56:20.468 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /var/log/ceph 2023-03-29T10:56:20.529 INFO:tasks.ceph:Disabling ceph logrotate... 2023-03-29T10:56:20.529 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -f -- /etc/logrotate.d/ceph 2023-03-29T10:56:20.532 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/logrotate.d/ceph 2023-03-29T10:56:20.593 INFO:tasks.ceph:Creating extra log directories... 2023-03-29T10:56:20.593 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-03-29T10:56:20.596 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-03-29T10:56:20.663 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-03-29T10:56:20.663 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-03-29T10:56:20.664 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-03-29T10:56:20.664 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:56:20.664 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-03-29T10:56:20.704 DEBUG:teuthology.orchestra.run.smithi111:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-03-29T10:56:20.900 DEBUG:teuthology.orchestra.run.smithi111:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-03-29T10:56:20.962 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:56:20.962 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-03-29T10:56:21.003 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-03-29T10:56:21.191 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-03-29T10:56:21.257 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-03-29T10:56:21.257 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-03-29T10:56:21.258 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2023-03-29_10:31:35-fs-wip-vshankar-testing-20230317.095222-testing-default-smithi/7225143', 'branch': 'wip-vshankar-testing-20230317.095222', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/xfstests-dev}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7225143', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '1776e8f494e94fd3414d7679e5782ec22e83c95f'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2023-03-29_10:31:35-fs-wip-vshankar-testing-20230317.095222-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20230317.095222'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'f03d69a8741c605ae6711afe2c9991ac46095f7d'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'f03d69a8741c605ae6711afe2c9991ac46095f7d'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20230317.095222', 'sha1': 'f03d69a8741c605ae6711afe2c9991ac46095f7d'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 30, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': 'f03d69a8741c605ae6711afe2c9991ac46095f7d', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20230317.095222', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_f03d69a8741c605ae6711afe2c9991ac46095f7d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'f03d69a8741c605ae6711afe2c9991ac46095f7d', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi111.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDlY8v0fYkp8mXTemAm3+xgrF3RHMFPpqbu5eVeI11m/QA3a2Cn4kO8mS7G4jEaRIghy7Gqk3mSZ0Eb2C/J9Yk5MeS70I5kzk7QdKDfydRx+fB81rS9SldGQrHCkut00Vb66ZrdV4X5Hcnmd7CNSn8VXcY/l3GikDCuejWB4I2nzhoaXIxBS6pha2wsovbi5gr8TxKEAfv6tp/bZ9Ypty5Nb565FDyhGY2dmotiyFSiWV7xJIBbfBSbW53ueYcx7KzEwuhjL76oy942Uqs9VWfHQAO0/Ge0gyDj1B+Gp8MCndRlFojL2uezsu/Qlm1bKj3laNv7Roij5FUfPe/ROxxJ2qT3sOPNaExi0WrDNfYIl9Zlu9KCTXtM9OgMtoFy4HtJMTsu4JtxIEgND/rRrCUEwzpZR3sW2FsJNrq/ETslEc4dDDDrra3B3yg3ryvJG0la4+0YVdpx1MQG6Y/N1ef1/vxSALcmobQAV4XFIs64gNcyX7c2Q4xBCvRP90EOQEE=', 'smithi119.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6r7aIQgpyhdYAjjhoFwotCukZ0xwUNAjXcC7AbsOUpeSzdaPbKFC8vx7JqTb3OKG6g9Gs7JinOu4o5otVVEMz7zMUeoTmAj2r8ZD9fuLP85rvPgyHWaQLx/uyPXlZSw9kU4/MWNspML5fmhvx3WJ2J6poT9LWZ5nzuzGLm6QWhU9kNffj2xvxumMOxzXij55J+2UYvckDruUrH1f4NQkmOEJ+CqmgpcdBRW0hD2SfebaUiZ8KmUQH6QYPG4Sxj5a1nSH+jNl8EZUon0P2v8jOYtclslXZFBz1FhZ8Q8omf3UVWgoseNoLigazJMTCxNL96RuakWCYzIPUw2lzvwVwNVI/tLKFAuu3utmaywOh3t+qC1QBslB7bT9oIt09D+0rXLplxZZIqba/5JrjIGb3HlxuHdiKEQ82ABzztub6rZHVyCG1JeFmKlhVlg5ZPAr9FeClwisrmr9TXvLGAyRy2nF6gT+yvDAo5gOG8J45qylWCO2x5/7ZxiJ1Qi4ntJk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '1776e8f494e94fd3414d7679e5782ec22e83c95f'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'f03d69a8741c605ae6711afe2c9991ac46095f7d'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.tests_from_xfstests_dev']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e3729161eecf8e6d09a4069acbb883cdcd5e9f50', 'timestamp': '2023-03-29_10:31:35', 'tube': 'smithi', 'use_shaman': True, 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.803'} 2023-03-29T10:56:21.258 DEBUG:teuthology.orchestra.run.smithi111:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-03-29T10:56:21.262 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-03-29T10:56:21.302 DEBUG:teuthology.orchestra.run.smithi111:> sudo install -d -m0777 -- /var/run/ceph 2023-03-29T10:56:21.344 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/run/ceph 2023-03-29T10:56:21.406 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:56:21.406 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/scratch_devs of=/dev/stdout 2023-03-29T10:56:21.447 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-03-29T10:56:21.448 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_1 2023-03-29T10:56:21.529 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-03-29T10:56:21.529 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-29T10:56:21.529 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 444 Links: 1 2023-03-29T10:56:21.529 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-29T10:56:21.529 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2023-03-29T10:56:21.529 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2023-03-29 10:56:14.989810519 +0000 2023-03-29T10:56:21.529 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2023-03-29 10:53:06.992331180 +0000 2023-03-29T10:56:21.529 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2023-03-29 10:53:06.992331180 +0000 2023-03-29T10:56:21.529 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2023-03-29 10:53:06.967331598 +0000 2023-03-29T10:56:21.529 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-03-29T10:56:21.630 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2023-03-29T10:56:21.630 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2023-03-29T10:56:21.630 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000298387 s, 1.7 MB/s 2023-03-29T10:56:21.632 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-03-29T10:56:21.676 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_2 2023-03-29T10:56:21.756 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-03-29T10:56:21.756 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-29T10:56:21.756 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 450 Links: 1 2023-03-29T10:56:21.756 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-29T10:56:21.756 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2023-03-29T10:56:21.756 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2023-03-29 10:56:14.989810519 +0000 2023-03-29T10:56:21.756 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2023-03-29 10:53:07.757318391 +0000 2023-03-29T10:56:21.756 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2023-03-29 10:53:07.757318391 +0000 2023-03-29T10:56:21.756 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2023-03-29 10:53:07.732318809 +0000 2023-03-29T10:56:21.756 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-03-29T10:56:21.855 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2023-03-29T10:56:21.855 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2023-03-29T10:56:21.856 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000283517 s, 1.8 MB/s 2023-03-29T10:56:21.857 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-03-29T10:56:21.899 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_3 2023-03-29T10:56:21.982 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-03-29T10:56:21.982 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-29T10:56:21.982 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 456 Links: 1 2023-03-29T10:56:21.982 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-29T10:56:21.982 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2023-03-29T10:56:21.982 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2023-03-29 10:56:14.990810503 +0000 2023-03-29T10:56:21.982 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2023-03-29 10:53:08.544305235 +0000 2023-03-29T10:56:21.982 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2023-03-29 10:53:08.544305235 +0000 2023-03-29T10:56:21.982 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2023-03-29 10:53:08.520305636 +0000 2023-03-29T10:56:21.982 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-03-29T10:56:22.082 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2023-03-29T10:56:22.082 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2023-03-29T10:56:22.082 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000271862 s, 1.9 MB/s 2023-03-29T10:56:22.084 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-03-29T10:56:22.129 DEBUG:teuthology.orchestra.run.smithi111:> stat /dev/vg_nvme/lv_4 2023-03-29T10:56:22.212 INFO:teuthology.orchestra.run.smithi111.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-03-29T10:56:22.212 INFO:teuthology.orchestra.run.smithi111.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-29T10:56:22.212 INFO:teuthology.orchestra.run.smithi111.stdout:Device: 5h/5d Inode: 462 Links: 1 2023-03-29T10:56:22.212 INFO:teuthology.orchestra.run.smithi111.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-29T10:56:22.212 INFO:teuthology.orchestra.run.smithi111.stdout:Context: system_u:object_r:device_t:s0 2023-03-29T10:56:22.212 INFO:teuthology.orchestra.run.smithi111.stdout:Access: 2023-03-29 10:56:14.990810503 +0000 2023-03-29T10:56:22.212 INFO:teuthology.orchestra.run.smithi111.stdout:Modify: 2023-03-29 10:53:09.365291510 +0000 2023-03-29T10:56:22.213 INFO:teuthology.orchestra.run.smithi111.stdout:Change: 2023-03-29 10:53:09.365291510 +0000 2023-03-29T10:56:22.213 INFO:teuthology.orchestra.run.smithi111.stdout: Birth: 2023-03-29 10:53:09.341291912 +0000 2023-03-29T10:56:22.213 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-03-29T10:56:22.311 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records in 2023-03-29T10:56:22.311 INFO:teuthology.orchestra.run.smithi111.stderr:1+0 records out 2023-03-29T10:56:22.311 INFO:teuthology.orchestra.run.smithi111.stderr:512 bytes copied, 0.000258799 s, 2.0 MB/s 2023-03-29T10:56:22.313 DEBUG:teuthology.orchestra.run.smithi111:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-03-29T10:56:22.357 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-29T10:56:22.357 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:56:22.357 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2023-03-29T10:56:22.402 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-03-29T10:56:22.402 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2023-03-29T10:56:22.485 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-03-29T10:56:22.486 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-29T10:56:22.486 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 447 Links: 1 2023-03-29T10:56:22.486 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-29T10:56:22.486 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-03-29T10:56:22.486 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-03-29 10:56:08.476373964 +0000 2023-03-29T10:56:22.486 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-03-29 10:53:06.979365622 +0000 2023-03-29T10:56:22.486 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-03-29 10:53:06.979365622 +0000 2023-03-29T10:56:22.486 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2023-03-29 10:53:06.953366266 +0000 2023-03-29T10:56:22.486 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-03-29T10:56:22.586 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-03-29T10:56:22.586 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-03-29T10:56:22.586 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000291143 s, 1.8 MB/s 2023-03-29T10:56:22.588 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-03-29T10:56:22.631 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2023-03-29T10:56:22.711 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-03-29T10:56:22.712 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-29T10:56:22.712 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 453 Links: 1 2023-03-29T10:56:22.712 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-29T10:56:22.712 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-03-29T10:56:22.712 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-03-29 10:56:08.476373964 +0000 2023-03-29T10:56:22.712 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-03-29 10:53:07.765346157 +0000 2023-03-29T10:56:22.712 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-03-29 10:53:07.765346157 +0000 2023-03-29T10:56:22.712 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2023-03-29 10:53:07.742346727 +0000 2023-03-29T10:56:22.712 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-03-29T10:56:22.810 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-03-29T10:56:22.810 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-03-29T10:56:22.810 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000289166 s, 1.8 MB/s 2023-03-29T10:56:22.812 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-03-29T10:56:22.854 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2023-03-29T10:56:22.935 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-03-29T10:56:22.935 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-29T10:56:22.935 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 459 Links: 1 2023-03-29T10:56:22.936 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-29T10:56:22.936 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-03-29T10:56:22.936 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-03-29 10:56:08.476373964 +0000 2023-03-29T10:56:22.936 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-03-29 10:53:08.553326644 +0000 2023-03-29T10:56:22.936 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-03-29 10:53:08.553326644 +0000 2023-03-29T10:56:22.936 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2023-03-29 10:53:08.527327287 +0000 2023-03-29T10:56:22.936 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-03-29T10:56:23.034 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-03-29T10:56:23.035 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-03-29T10:56:23.035 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000298136 s, 1.7 MB/s 2023-03-29T10:56:23.037 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-03-29T10:56:23.079 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2023-03-29T10:56:23.159 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-03-29T10:56:23.159 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-03-29T10:56:23.159 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 465 Links: 1 2023-03-29T10:56:23.159 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-03-29T10:56:23.159 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-03-29T10:56:23.159 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-03-29 10:56:08.477373939 +0000 2023-03-29T10:56:23.159 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-03-29 10:53:09.341307129 +0000 2023-03-29T10:56:23.159 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-03-29 10:53:09.341307129 +0000 2023-03-29T10:56:23.159 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2023-03-29 10:53:09.318307699 +0000 2023-03-29T10:56:23.160 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-03-29T10:56:23.258 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-03-29T10:56:23.258 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-03-29T10:56:23.258 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.00028275 s, 1.8 MB/s 2023-03-29T10:56:23.260 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-03-29T10:56:23.301 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-29T10:56:23.302 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi111.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-03-29T10:56:23.302 INFO:tasks.ceph:Generating config... 2023-03-29T10:56:23.303 INFO:tasks.ceph:[client] client mount timeout = 600 2023-03-29T10:56:23.303 INFO:tasks.ceph:[client] debug client = 20 2023-03-29T10:56:23.303 INFO:tasks.ceph:[client] debug ms = 1 2023-03-29T10:56:23.303 INFO:tasks.ceph:[client] pid file = 2023-03-29T10:56:23.303 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-03-29T10:56:23.303 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-03-29T10:56:23.304 INFO:tasks.ceph:[mds] debug mds = 20 2023-03-29T10:56:23.304 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-03-29T10:56:23.304 INFO:tasks.ceph:[mds] debug ms = 1 2023-03-29T10:56:23.304 INFO:tasks.ceph:[mds] mds debug frag = True 2023-03-29T10:56:23.304 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-03-29T10:56:23.304 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-03-29T10:56:23.304 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-03-29T10:56:23.304 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-03-29T10:56:23.304 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-03-29T10:56:23.304 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-03-29T10:56:23.304 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-03-29T10:56:23.304 INFO:tasks.ceph:[mgr] debug ms = 1 2023-03-29T10:56:23.304 INFO:tasks.ceph:[mon] debug mon = 20 2023-03-29T10:56:23.304 INFO:tasks.ceph:[mon] debug ms = 1 2023-03-29T10:56:23.304 INFO:tasks.ceph:[mon] debug paxos = 20 2023-03-29T10:56:23.305 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-03-29T10:56:23.305 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-03-29T10:56:23.305 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-03-29T10:56:23.305 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-03-29T10:56:23.305 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-03-29T10:56:23.305 INFO:tasks.ceph:[osd] debug ms = 1 2023-03-29T10:56:23.305 INFO:tasks.ceph:[osd] debug osd = 20 2023-03-29T10:56:23.305 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-03-29T10:56:23.305 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-03-29T10:56:23.305 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-03-29T10:56:23.305 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-03-29T10:56:23.305 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-03-29T10:56:23.305 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-03-29T10:56:23.305 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-03-29T10:56:23.306 INFO:tasks.ceph:Setting up mon.a... 2023-03-29T10:56:23.306 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-03-29T10:56:23.404 INFO:teuthology.orchestra.run.smithi111.stdout:creating /etc/ceph/ceph.keyring 2023-03-29T10:56:23.411 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-03-29T10:56:23.503 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-29T10:56:23.564 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.111'), ('mon.b', '172.21.15.119'), ('mon.c', '[v2:172.21.15.119:3301,v1:172.21.15.119:6790]')] 2023-03-29T10:56:23.564 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.111,172.21.15.119,[v2:172.21.15.119:3301,v1:172.21.15.119:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-03-29T10:56:23.565 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:56:23.565 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-03-29T10:56:23.605 DEBUG:teuthology.orchestra.run.smithi111:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.111 --add b 172.21.15.119 --addv c '[v2:172.21.15.119:3301,v1:172.21.15.119:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-03-29T10:56:23.726 INFO:teuthology.orchestra.run.smithi111.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-03-29T10:56:23.727 INFO:teuthology.orchestra.run.smithi111.stdout:monmaptool: generated fsid c8526f71-9b58-4b2e-978f-708199ec8aef 2023-03-29T10:56:23.727 INFO:teuthology.orchestra.run.smithi111.stdout:setting min_mon_release = pacific 2023-03-29T10:56:23.727 INFO:teuthology.orchestra.run.smithi111.stdout:epoch 0 2023-03-29T10:56:23.727 INFO:teuthology.orchestra.run.smithi111.stdout:fsid c8526f71-9b58-4b2e-978f-708199ec8aef 2023-03-29T10:56:23.727 INFO:teuthology.orchestra.run.smithi111.stdout:last_changed 2023-03-29T10:56:23.727486+0000 2023-03-29T10:56:23.727 INFO:teuthology.orchestra.run.smithi111.stdout:created 2023-03-29T10:56:23.727486+0000 2023-03-29T10:56:23.727 INFO:teuthology.orchestra.run.smithi111.stdout:min_mon_release 16 (pacific) 2023-03-29T10:56:23.727 INFO:teuthology.orchestra.run.smithi111.stdout:election_strategy: 1 2023-03-29T10:56:23.727 INFO:teuthology.orchestra.run.smithi111.stdout:0: [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] mon.a 2023-03-29T10:56:23.727 INFO:teuthology.orchestra.run.smithi111.stdout:1: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.b 2023-03-29T10:56:23.727 INFO:teuthology.orchestra.run.smithi111.stdout:2: [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] mon.c 2023-03-29T10:56:23.727 INFO:teuthology.orchestra.run.smithi111.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-03-29T10:56:23.729 DEBUG:teuthology.orchestra.run.smithi111:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-03-29T10:56:23.769 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c8526f71-9b58-4b2e-978f-708199ec8aef... 2023-03-29T10:56:23.770 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-03-29T10:56:23.812 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-03-29T10:56:23.913 INFO:teuthology.orchestra.run.smithi111.stdout:[global] 2023-03-29T10:56:23.914 INFO:teuthology.orchestra.run.smithi111.stdout: chdir = "" 2023-03-29T10:56:23.914 INFO:teuthology.orchestra.run.smithi111.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-03-29T10:56:23.914 INFO:teuthology.orchestra.run.smithi111.stdout: auth supported = cephx 2023-03-29T10:56:23.914 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.914 INFO:teuthology.orchestra.run.smithi111.stdout: filestore xattr use omap = true 2023-03-29T10:56:23.914 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.914 INFO:teuthology.orchestra.run.smithi111.stdout: mon clock drift allowed = 1.000 2023-03-29T10:56:23.915 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.915 INFO:teuthology.orchestra.run.smithi111.stdout: osd crush chooseleaf type = 0 2023-03-29T10:56:23.915 INFO:teuthology.orchestra.run.smithi111.stdout: auth debug = true 2023-03-29T10:56:23.915 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.915 INFO:teuthology.orchestra.run.smithi111.stdout: ms die on old message = true 2023-03-29T10:56:23.915 INFO:teuthology.orchestra.run.smithi111.stdout: ms die on bug = true 2023-03-29T10:56:23.915 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.915 INFO:teuthology.orchestra.run.smithi111.stdout: mon max pg per osd = 10000 # >= luminous 2023-03-29T10:56:23.915 INFO:teuthology.orchestra.run.smithi111.stdout: mon pg warn max object skew = 0 2023-03-29T10:56:23.915 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.915 INFO:teuthology.orchestra.run.smithi111.stdout: # disable pg_autoscaler by default for new pools 2023-03-29T10:56:23.915 INFO:teuthology.orchestra.run.smithi111.stdout: osd_pool_default_pg_autoscale_mode = off 2023-03-29T10:56:23.915 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.915 INFO:teuthology.orchestra.run.smithi111.stdout: osd pool default size = 2 2023-03-29T10:56:23.915 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.915 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd allow primary affinity = true 2023-03-29T10:56:23.915 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd allow pg remap = true 2023-03-29T10:56:23.915 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on legacy crush tunables = false 2023-03-29T10:56:23.916 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on crush straw calc version zero = false 2023-03-29T10:56:23.916 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on no sortbitwise = false 2023-03-29T10:56:23.916 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on osd down out interval zero = false 2023-03-29T10:56:23.916 INFO:teuthology.orchestra.run.smithi111.stdout: mon warn on too few osds = false 2023-03-29T10:56:23.916 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-03-29T10:56:23.916 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_pool_no_redundancy = false 2023-03-29T10:56:23.916 INFO:teuthology.orchestra.run.smithi111.stdout: mon_allow_pool_size_one = true 2023-03-29T10:56:23.916 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.916 INFO:teuthology.orchestra.run.smithi111.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-03-29T10:56:23.916 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.916 INFO:teuthology.orchestra.run.smithi111.stdout: osd default data pool replay window = 5 2023-03-29T10:56:23.916 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.917 INFO:teuthology.orchestra.run.smithi111.stdout: mon allow pool delete = true 2023-03-29T10:56:23.917 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.917 INFO:teuthology.orchestra.run.smithi111.stdout: mon cluster log file level = debug 2023-03-29T10:56:23.917 INFO:teuthology.orchestra.run.smithi111.stdout: debug asserts on shutdown = true 2023-03-29T10:56:23.917 INFO:teuthology.orchestra.run.smithi111.stdout: mon health detail to clog = false 2023-03-29T10:56:23.917 INFO:teuthology.orchestra.run.smithi111.stdout: mon host = "172.21.15.111,172.21.15.119,[v2:172.21.15.119:3301,v1:172.21.15.119:6790]" 2023-03-29T10:56:23.917 INFO:teuthology.orchestra.run.smithi111.stdout: fsid = c8526f71-9b58-4b2e-978f-708199ec8aef 2023-03-29T10:56:23.917 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.917 INFO:teuthology.orchestra.run.smithi111.stdout:[osd] 2023-03-29T10:56:23.917 INFO:teuthology.orchestra.run.smithi111.stdout: osd journal size = 100 2023-03-29T10:56:23.917 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.917 INFO:teuthology.orchestra.run.smithi111.stdout: osd scrub load threshold = 5.0 2023-03-29T10:56:23.917 INFO:teuthology.orchestra.run.smithi111.stdout: osd scrub max interval = 600 2023-03-29T10:56:23.917 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.917 INFO:teuthology.orchestra.run.smithi111.stdout: osd recover clone overlap = true 2023-03-29T10:56:23.917 INFO:teuthology.orchestra.run.smithi111.stdout: osd recovery max chunk = 1048576 2023-03-29T10:56:23.917 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.918 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug shutdown = true 2023-03-29T10:56:23.918 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug op order = true 2023-03-29T10:56:23.918 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug verify stray on activate = true 2023-03-29T10:56:23.918 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.918 INFO:teuthology.orchestra.run.smithi111.stdout: osd open classes on start = true 2023-03-29T10:56:23.918 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug pg log writeout = true 2023-03-29T10:56:23.918 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.918 INFO:teuthology.orchestra.run.smithi111.stdout: osd deep scrub update digest min age = 30 2023-03-29T10:56:23.918 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.918 INFO:teuthology.orchestra.run.smithi111.stdout: osd map max advance = 10 2023-03-29T10:56:23.918 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.918 INFO:teuthology.orchestra.run.smithi111.stdout: journal zero on create = true 2023-03-29T10:56:23.918 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.918 INFO:teuthology.orchestra.run.smithi111.stdout: filestore ondisk finisher threads = 3 2023-03-29T10:56:23.918 INFO:teuthology.orchestra.run.smithi111.stdout: filestore apply finisher threads = 3 2023-03-29T10:56:23.918 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.919 INFO:teuthology.orchestra.run.smithi111.stdout: bdev debug aio = true 2023-03-29T10:56:23.919 INFO:teuthology.orchestra.run.smithi111.stdout: osd debug misdirected ops = true 2023-03-29T10:56:23.919 INFO:teuthology.orchestra.run.smithi111.stdout: bluestore block size = 96636764160 2023-03-29T10:56:23.919 INFO:teuthology.orchestra.run.smithi111.stdout: bluestore fsck on mount = True 2023-03-29T10:56:23.919 INFO:teuthology.orchestra.run.smithi111.stdout: debug bluefs = 20 2023-03-29T10:56:23.919 INFO:teuthology.orchestra.run.smithi111.stdout: debug bluestore = 20 2023-03-29T10:56:23.919 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2023-03-29T10:56:23.919 INFO:teuthology.orchestra.run.smithi111.stdout: debug osd = 20 2023-03-29T10:56:23.919 INFO:teuthology.orchestra.run.smithi111.stdout: debug rocksdb = 10 2023-03-29T10:56:23.919 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd backfillfull_ratio = 0.85 2023-03-29T10:56:23.919 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd full ratio = 0.9 2023-03-29T10:56:23.919 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd nearfull ratio = 0.8 2023-03-29T10:56:23.919 INFO:teuthology.orchestra.run.smithi111.stdout: osd failsafe full ratio = 0.95 2023-03-29T10:56:23.919 INFO:teuthology.orchestra.run.smithi111.stdout: osd objectstore = bluestore 2023-03-29T10:56:23.919 INFO:teuthology.orchestra.run.smithi111.stdout: osd op complaint time = 180 2023-03-29T10:56:23.919 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.919 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr] 2023-03-29T10:56:23.919 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2023-03-29T10:56:23.920 INFO:teuthology.orchestra.run.smithi111.stdout: debug mgr = 20 2023-03-29T10:56:23.920 INFO:teuthology.orchestra.run.smithi111.stdout: debug mon = 20 2023-03-29T10:56:23.920 INFO:teuthology.orchestra.run.smithi111.stdout: debug auth = 20 2023-03-29T10:56:23.920 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min pgs per osd = 4 2023-03-29T10:56:23.920 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min bytes per osd = 10 2023-03-29T10:56:23.920 INFO:teuthology.orchestra.run.smithi111.stdout: mgr/telemetry/nag = false 2023-03-29T10:56:23.920 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.920 INFO:teuthology.orchestra.run.smithi111.stdout:[mon] 2023-03-29T10:56:23.920 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2023-03-29T10:56:23.920 INFO:teuthology.orchestra.run.smithi111.stdout: debug mon = 20 2023-03-29T10:56:23.920 INFO:teuthology.orchestra.run.smithi111.stdout: debug paxos = 20 2023-03-29T10:56:23.920 INFO:teuthology.orchestra.run.smithi111.stdout: debug auth = 20 2023-03-29T10:56:23.920 INFO:teuthology.orchestra.run.smithi111.stdout: mon data avail warn = 5 2023-03-29T10:56:23.920 INFO:teuthology.orchestra.run.smithi111.stdout: mon mgr mkfs grace = 240 2023-03-29T10:56:23.920 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min pgs per osd = 4 2023-03-29T10:56:23.920 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd reporter subtree level = osd 2023-03-29T10:56:23.920 INFO:teuthology.orchestra.run.smithi111.stdout: mon osd prime pg temp = true 2023-03-29T10:56:23.920 INFO:teuthology.orchestra.run.smithi111.stdout: mon reweight min bytes per osd = 10 2023-03-29T10:56:23.921 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.921 INFO:teuthology.orchestra.run.smithi111.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-03-29T10:56:23.921 INFO:teuthology.orchestra.run.smithi111.stdout: auth mon ticket ttl = 660 # 11m 2023-03-29T10:56:23.921 INFO:teuthology.orchestra.run.smithi111.stdout: auth service ticket ttl = 240 # 4m 2023-03-29T10:56:23.921 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.921 INFO:teuthology.orchestra.run.smithi111.stdout: # don't complain about insecure global_id in the test suite 2023-03-29T10:56:23.921 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-03-29T10:56:23.921 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-03-29T10:56:23.921 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.921 INFO:teuthology.orchestra.run.smithi111.stdout: # 1m isn't quite enough 2023-03-29T10:56:23.921 INFO:teuthology.orchestra.run.smithi111.stdout: mon_down_mkfs_grace = 2m 2023-03-29T10:56:23.921 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.921 INFO:teuthology.orchestra.run.smithi111.stdout: mon_warn_on_filestore_osds = false 2023-03-29T10:56:23.921 INFO:teuthology.orchestra.run.smithi111.stdout: mon op complaint time = 120 2023-03-29T10:56:23.921 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:56:23.921 INFO:teuthology.orchestra.run.smithi111.stdout:[client] 2023-03-29T10:56:23.922 INFO:teuthology.orchestra.run.smithi111.stdout: rgw cache enabled = true 2023-03-29T10:56:23.922 INFO:teuthology.orchestra.run.smithi111.stdout: rgw enable ops log = true 2023-03-29T10:56:23.922 INFO:teuthology.orchestra.run.smithi111.stdout: rgw enable usage log = true 2023-03-29T10:56:23.922 INFO:teuthology.orchestra.run.smithi111.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-03-29T10:56:23.922 INFO:teuthology.orchestra.run.smithi111.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-03-29T10:56:23.922 INFO:teuthology.orchestra.run.smithi111.stdout: client mount timeout = 600 2023-03-29T10:56:23.922 INFO:teuthology.orchestra.run.smithi111.stdout: debug client = 20 2023-03-29T10:56:23.922 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2023-03-29T10:56:23.922 INFO:teuthology.orchestra.run.smithi111.stdout: pid file = "" 2023-03-29T10:56:23.922 INFO:teuthology.orchestra.run.smithi111.stdout: rados mon op timeout = 900 2023-03-29T10:56:23.922 INFO:teuthology.orchestra.run.smithi111.stdout: rados osd op timeout = 900 2023-03-29T10:56:23.922 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.a] 2023-03-29T10:56:23.922 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.b] 2023-03-29T10:56:23.922 INFO:teuthology.orchestra.run.smithi111.stdout:[mon.c] 2023-03-29T10:56:23.922 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.a] 2023-03-29T10:56:23.922 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.c] 2023-03-29T10:56:23.922 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.b] 2023-03-29T10:56:23.923 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.d] 2023-03-29T10:56:23.923 INFO:teuthology.orchestra.run.smithi111.stdout:[mds] 2023-03-29T10:56:23.923 INFO:teuthology.orchestra.run.smithi111.stdout: debug mds = 20 2023-03-29T10:56:23.923 INFO:teuthology.orchestra.run.smithi111.stdout: debug mds balancer = 20 2023-03-29T10:56:23.923 INFO:teuthology.orchestra.run.smithi111.stdout: debug ms = 1 2023-03-29T10:56:23.923 INFO:teuthology.orchestra.run.smithi111.stdout: mds debug frag = True 2023-03-29T10:56:23.923 INFO:teuthology.orchestra.run.smithi111.stdout: mds debug scatterstat = True 2023-03-29T10:56:23.923 INFO:teuthology.orchestra.run.smithi111.stdout: mds op complaint time = 180 2023-03-29T10:56:23.923 INFO:teuthology.orchestra.run.smithi111.stdout: mds verify scatter = True 2023-03-29T10:56:23.923 INFO:teuthology.orchestra.run.smithi111.stdout: osd op complaint time = 180 2023-03-29T10:56:23.923 INFO:teuthology.orchestra.run.smithi111.stdout: rados mon op timeout = 900 2023-03-29T10:56:23.923 INFO:teuthology.orchestra.run.smithi111.stdout: rados osd op timeout = 900 2023-03-29T10:56:23.926 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2023-03-29T10:56:23.926 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2023-03-29T10:56:23.926 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-03-29T10:56:23.926 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2023-03-29T10:56:23.926 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.926 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2023-03-29T10:56:23.926 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.926 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2023-03-29T10:56:23.926 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.926 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2023-03-29T10:56:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2023-03-29T10:56:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2023-03-29T10:56:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2023-03-29T10:56:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2023-03-29T10:56:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2023-03-29T10:56:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2023-03-29T10:56:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2023-03-29T10:56:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2023-03-29T10:56:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2023-03-29T10:56:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2023-03-29T10:56:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2023-03-29T10:56:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2023-03-29T10:56:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2023-03-29T10:56:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2023-03-29T10:56:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2023-03-29T10:56:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-03-29T10:56:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2023-03-29T10:56:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2023-03-29T10:56:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-03-29T10:56:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2023-03-29T10:56:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2023-03-29T10:56:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2023-03-29T10:56:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2023-03-29T10:56:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2023-03-29T10:56:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = "172.21.15.111,172.21.15.119,[v2:172.21.15.119:3301,v1:172.21.15.119:6790]" 2023-03-29T10:56:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = c8526f71-9b58-4b2e-978f-708199ec8aef 2023-03-29T10:56:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.929 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2023-03-29T10:56:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2023-03-29T10:56:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2023-03-29T10:56:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2023-03-29T10:56:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2023-03-29T10:56:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2023-03-29T10:56:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2023-03-29T10:56:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2023-03-29T10:56:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2023-03-29T10:56:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2023-03-29T10:56:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2023-03-29T10:56:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2023-03-29T10:56:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2023-03-29T10:56:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2023-03-29T10:56:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2023-03-29T10:56:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2023-03-29T10:56:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2023-03-29T10:56:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2023-03-29T10:56:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore block size = 96636764160 2023-03-29T10:56:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore fsck on mount = True 2023-03-29T10:56:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluefs = 20 2023-03-29T10:56:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluestore = 20 2023-03-29T10:56:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-03-29T10:56:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 20 2023-03-29T10:56:23.931 INFO:teuthology.orchestra.run.smithi119.stdout: debug rocksdb = 10 2023-03-29T10:56:23.931 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd backfillfull_ratio = 0.85 2023-03-29T10:56:23.931 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd full ratio = 0.9 2023-03-29T10:56:23.931 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd nearfull ratio = 0.8 2023-03-29T10:56:23.931 INFO:teuthology.orchestra.run.smithi119.stdout: osd failsafe full ratio = 0.95 2023-03-29T10:56:23.931 INFO:teuthology.orchestra.run.smithi119.stdout: osd objectstore = bluestore 2023-03-29T10:56:23.931 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2023-03-29T10:56:23.931 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.931 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2023-03-29T10:56:23.931 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-03-29T10:56:23.931 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2023-03-29T10:56:23.931 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2023-03-29T10:56:23.931 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2023-03-29T10:56:23.931 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2023-03-29T10:56:23.931 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2023-03-29T10:56:23.931 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2023-03-29T10:56:23.931 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.931 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2023-03-29T10:56:23.931 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-03-29T10:56:23.932 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2023-03-29T10:56:23.932 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2023-03-29T10:56:23.932 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2023-03-29T10:56:23.932 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2023-03-29T10:56:23.932 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2023-03-29T10:56:23.932 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2023-03-29T10:56:23.932 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2023-03-29T10:56:23.932 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2023-03-29T10:56:23.932 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2023-03-29T10:56:23.932 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.932 INFO:teuthology.orchestra.run.smithi119.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-03-29T10:56:23.932 INFO:teuthology.orchestra.run.smithi119.stdout: auth mon ticket ttl = 660 # 11m 2023-03-29T10:56:23.932 INFO:teuthology.orchestra.run.smithi119.stdout: auth service ticket ttl = 240 # 4m 2023-03-29T10:56:23.932 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.932 INFO:teuthology.orchestra.run.smithi119.stdout: # don't complain about insecure global_id in the test suite 2023-03-29T10:56:23.932 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-03-29T10:56:23.932 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-03-29T10:56:23.932 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.933 INFO:teuthology.orchestra.run.smithi119.stdout: # 1m isn't quite enough 2023-03-29T10:56:23.933 INFO:teuthology.orchestra.run.smithi119.stdout: mon_down_mkfs_grace = 2m 2023-03-29T10:56:23.933 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.933 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_filestore_osds = false 2023-03-29T10:56:23.933 INFO:teuthology.orchestra.run.smithi119.stdout: mon op complaint time = 120 2023-03-29T10:56:23.933 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-03-29T10:56:23.933 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2023-03-29T10:56:23.933 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2023-03-29T10:56:23.933 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2023-03-29T10:56:23.933 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2023-03-29T10:56:23.933 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-03-29T10:56:23.933 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-03-29T10:56:23.933 INFO:teuthology.orchestra.run.smithi119.stdout: client mount timeout = 600 2023-03-29T10:56:23.933 INFO:teuthology.orchestra.run.smithi119.stdout: debug client = 20 2023-03-29T10:56:23.933 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-03-29T10:56:23.933 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = "" 2023-03-29T10:56:23.933 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2023-03-29T10:56:23.933 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2023-03-29T10:56:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2023-03-29T10:56:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.b] 2023-03-29T10:56:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.c] 2023-03-29T10:56:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a] 2023-03-29T10:56:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c] 2023-03-29T10:56:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b] 2023-03-29T10:56:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d] 2023-03-29T10:56:23.934 INFO:teuthology.orchestra.run.smithi119.stdout:[mds] 2023-03-29T10:56:23.934 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds = 20 2023-03-29T10:56:23.934 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds balancer = 20 2023-03-29T10:56:23.934 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-03-29T10:56:23.934 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug frag = True 2023-03-29T10:56:23.934 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug scatterstat = True 2023-03-29T10:56:23.934 INFO:teuthology.orchestra.run.smithi119.stdout: mds op complaint time = 180 2023-03-29T10:56:23.934 INFO:teuthology.orchestra.run.smithi119.stdout: mds verify scatter = True 2023-03-29T10:56:23.934 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2023-03-29T10:56:23.934 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2023-03-29T10:56:23.935 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2023-03-29T10:56:23.952 INFO:tasks.ceph:Creating admin key on mon.a... 2023-03-29T10:56:23.952 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-03-29T10:56:24.057 INFO:tasks.ceph:Copying monmap to all nodes... 2023-03-29T10:56:24.057 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:56:24.057 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-03-29T10:56:24.098 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:56:24.098 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-03-29T10:56:24.179 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi111.front.sepia.ceph.com 2023-03-29T10:56:24.179 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:56:24.179 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/etc/ceph/ceph.keyring 2023-03-29T10:56:24.179 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-29T10:56:24.371 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:56:24.371 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-03-29T10:56:24.410 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi119.front.sepia.ceph.com 2023-03-29T10:56:24.410 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:56:24.410 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.keyring 2023-03-29T10:56:24.410 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-03-29T10:56:24.504 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:56:24.504 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-03-29T10:56:24.586 INFO:tasks.ceph:Setting up mon nodes... 2023-03-29T10:56:24.587 INFO:tasks.ceph:Setting up mgr nodes... 2023-03-29T10:56:24.587 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-03-29T10:56:24.696 INFO:teuthology.orchestra.run.smithi111.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-03-29T10:56:24.701 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-03-29T10:56:24.822 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-03-29T10:56:24.828 INFO:tasks.ceph:Setting up mds nodes... 2023-03-29T10:56:24.829 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-03-29T10:56:24.938 INFO:teuthology.orchestra.run.smithi111.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-03-29T10:56:24.943 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-03-29T10:56:25.005 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-03-29T10:56:25.114 INFO:teuthology.orchestra.run.smithi111.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-03-29T10:56:25.119 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-03-29T10:56:25.180 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-03-29T10:56:25.290 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-03-29T10:56:25.295 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-03-29T10:56:25.356 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-03-29T10:56:25.465 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-03-29T10:56:25.470 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-03-29T10:56:25.530 INFO:tasks.ceph_client:Setting up client nodes... 2023-03-29T10:56:25.531 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-03-29T10:56:25.618 INFO:teuthology.orchestra.run.smithi111.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-03-29T10:56:25.644 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-03-29T10:56:25.730 INFO:teuthology.orchestra.run.smithi111.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-03-29T10:56:25.757 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-03-29T10:56:25.844 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-03-29T10:56:25.871 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-03-29T10:56:25.957 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-03-29T10:56:25.984 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-03-29T10:56:25.984 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi111.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-03-29T10:56:25.984 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-03-29T10:56:26.046 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-29T10:56:26.046 INFO:tasks.ceph:role: osd.0 2023-03-29T10:56:26.046 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi111.front.sepia.ceph.com 2023-03-29T10:56:26.047 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-03-29T10:56:26.115 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-03-29T10:56:26.115 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-29T10:56:26.115 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-29T10:56:26.115 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-29T10:56:26.115 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-29T10:56:26.115 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2023-03-29T10:56:26.115 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-29T10:56:26.115 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-29T10:56:26.115 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-29T10:56:26.115 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-29T10:56:26.539 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2023-03-29T10:56:26.543 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2023-03-29T10:56:26.543 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-03-29T10:56:26.719 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-03-29T10:56:26.791 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-03-29T10:56:26.853 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-29T10:56:26.853 INFO:tasks.ceph:role: osd.1 2023-03-29T10:56:26.853 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi111.front.sepia.ceph.com 2023-03-29T10:56:26.853 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-03-29T10:56:26.922 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-03-29T10:56:26.922 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-29T10:56:26.922 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-29T10:56:26.922 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-29T10:56:26.922 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-29T10:56:26.922 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2023-03-29T10:56:26.922 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-29T10:56:26.922 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-29T10:56:26.923 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-29T10:56:26.923 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-29T10:56:27.324 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2023-03-29T10:56:27.327 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2023-03-29T10:56:27.327 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-03-29T10:56:27.447 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-03-29T10:56:27.514 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-03-29T10:56:27.575 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-29T10:56:27.575 INFO:tasks.ceph:role: osd.2 2023-03-29T10:56:27.575 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi111.front.sepia.ceph.com 2023-03-29T10:56:27.575 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-03-29T10:56:27.644 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-03-29T10:56:27.644 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-29T10:56:27.644 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-29T10:56:27.644 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-29T10:56:27.644 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-29T10:56:27.644 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2023-03-29T10:56:27.645 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-29T10:56:27.645 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-29T10:56:27.645 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-29T10:56:27.645 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-29T10:56:28.097 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2023-03-29T10:56:28.101 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2023-03-29T10:56:28.101 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-03-29T10:56:28.281 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-03-29T10:56:28.351 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-03-29T10:56:28.412 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-03-29T10:56:28.412 INFO:tasks.ceph:role: osd.3 2023-03-29T10:56:28.412 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi111.front.sepia.ceph.com 2023-03-29T10:56:28.412 DEBUG:teuthology.orchestra.run.smithi111:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-03-29T10:56:28.480 INFO:teuthology.orchestra.run.smithi111.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-03-29T10:56:28.481 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-29T10:56:28.481 INFO:teuthology.orchestra.run.smithi111.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-29T10:56:28.481 INFO:teuthology.orchestra.run.smithi111.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-29T10:56:28.481 INFO:teuthology.orchestra.run.smithi111.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-29T10:56:28.481 INFO:teuthology.orchestra.run.smithi111.stdout: = sunit=0 swidth=0 blks 2023-03-29T10:56:28.481 INFO:teuthology.orchestra.run.smithi111.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-29T10:56:28.481 INFO:teuthology.orchestra.run.smithi111.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-29T10:56:28.481 INFO:teuthology.orchestra.run.smithi111.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-29T10:56:28.481 INFO:teuthology.orchestra.run.smithi111.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-29T10:56:28.913 INFO:teuthology.orchestra.run.smithi111.stdout:Discarding blocks...Done. 2023-03-29T10:56:28.917 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi111.front.sepia.ceph.com -o noatime 2023-03-29T10:56:28.917 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-03-29T10:56:29.083 DEBUG:teuthology.orchestra.run.smithi111:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-03-29T10:56:29.150 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-29T10:56:29.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:29.254+0000 7f6c7986a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-03-29T10:56:29.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:29.255+0000 7f6c7986a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-03-29T10:56:29.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:29.255+0000 7f6c7986a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-03-29T10:56:29.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:29.255+0000 7f6c7986a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-03-29T10:56:29.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:29.255+0000 7f6c7986a640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-03-29T10:56:29.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:29.257+0000 7f6c7986a640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-03-29T10:56:32.136 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-03-29T10:56:32.200 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-29T10:56:32.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:32.295+0000 7f86297ca640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-03-29T10:56:32.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:32.295+0000 7f86297ca640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-03-29T10:56:32.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:32.295+0000 7f86297ca640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-03-29T10:56:32.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:32.295+0000 7f86297ca640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-03-29T10:56:32.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:32.295+0000 7f86297ca640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-03-29T10:56:32.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:32.297+0000 7f86297ca640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-03-29T10:56:35.169 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-03-29T10:56:35.231 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-29T10:56:35.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:35.326+0000 7f41257ad640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-03-29T10:56:35.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:35.326+0000 7f41257ad640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-03-29T10:56:35.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:35.326+0000 7f41257ad640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-03-29T10:56:35.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:35.326+0000 7f41257ad640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-03-29T10:56:35.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:35.326+0000 7f41257ad640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-03-29T10:56:35.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:35.328+0000 7f41257ad640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-03-29T10:56:38.215 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-03-29T10:56:38.281 DEBUG:teuthology.orchestra.run.smithi111:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-29T10:56:38.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:38.378+0000 7ffa9719f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-03-29T10:56:38.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:38.378+0000 7ffa9719f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-03-29T10:56:38.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:38.378+0000 7ffa9719f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-03-29T10:56:38.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:38.378+0000 7ffa9719f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-03-29T10:56:38.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:38.379+0000 7ffa9719f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-03-29T10:56:38.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:56:38.380+0000 7ffa9719f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-03-29T10:56:41.262 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-03-29T10:56:41.333 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-03-29T10:56:41.401 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-29T10:56:41.401 INFO:tasks.ceph:role: osd.4 2023-03-29T10:56:41.401 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com 2023-03-29T10:56:41.401 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-03-29T10:56:41.472 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-03-29T10:56:41.472 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-29T10:56:41.472 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-29T10:56:41.472 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-29T10:56:41.472 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-29T10:56:41.472 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2023-03-29T10:56:41.472 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-29T10:56:41.473 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-29T10:56:41.473 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-29T10:56:41.473 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-29T10:56:41.911 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2023-03-29T10:56:41.915 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2023-03-29T10:56:41.915 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-03-29T10:56:42.088 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-03-29T10:56:42.158 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-03-29T10:56:42.220 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-29T10:56:42.220 INFO:tasks.ceph:role: osd.5 2023-03-29T10:56:42.220 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com 2023-03-29T10:56:42.220 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-03-29T10:56:42.288 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-03-29T10:56:42.288 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-29T10:56:42.288 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-29T10:56:42.288 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-29T10:56:42.288 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-29T10:56:42.288 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2023-03-29T10:56:42.288 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-29T10:56:42.288 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-29T10:56:42.289 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-29T10:56:42.289 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-29T10:56:42.699 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2023-03-29T10:56:42.702 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2023-03-29T10:56:42.703 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-03-29T10:56:42.832 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-03-29T10:56:42.901 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-03-29T10:56:42.962 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-29T10:56:42.962 INFO:tasks.ceph:role: osd.6 2023-03-29T10:56:42.962 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com 2023-03-29T10:56:42.962 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-03-29T10:56:43.030 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-03-29T10:56:43.030 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-29T10:56:43.030 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-29T10:56:43.030 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-29T10:56:43.030 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-29T10:56:43.030 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2023-03-29T10:56:43.030 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-29T10:56:43.030 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-29T10:56:43.030 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-29T10:56:43.030 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-29T10:56:43.485 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2023-03-29T10:56:43.488 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2023-03-29T10:56:43.488 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-03-29T10:56:43.655 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-03-29T10:56:43.722 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-03-29T10:56:43.783 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-03-29T10:56:43.784 INFO:tasks.ceph:role: osd.7 2023-03-29T10:56:43.784 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi119.front.sepia.ceph.com 2023-03-29T10:56:43.784 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-03-29T10:56:43.852 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-03-29T10:56:43.853 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2023-03-29T10:56:43.853 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-03-29T10:56:43.853 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-03-29T10:56:43.853 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-03-29T10:56:43.853 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2023-03-29T10:56:43.853 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-03-29T10:56:43.853 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-03-29T10:56:43.853 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-03-29T10:56:43.853 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-03-29T10:56:44.290 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2023-03-29T10:56:44.293 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2023-03-29T10:56:44.294 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-03-29T10:56:44.473 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-03-29T10:56:44.540 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-29T10:56:44.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:44.658+0000 7f96d0dd2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-03-29T10:56:44.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:44.658+0000 7f96d0dd2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-03-29T10:56:44.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:44.658+0000 7f96d0dd2640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-03-29T10:56:44.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:44.658+0000 7f96d0dd2640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-03-29T10:56:44.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:44.658+0000 7f96d0dd2640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-03-29T10:56:44.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:44.660+0000 7f96d0dd2640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-03-29T10:56:47.546 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-03-29T10:56:47.612 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-29T10:56:47.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:47.709+0000 7f34c90d8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-03-29T10:56:47.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:47.709+0000 7f34c90d8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-03-29T10:56:47.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:47.709+0000 7f34c90d8640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-03-29T10:56:47.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:47.709+0000 7f34c90d8640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-03-29T10:56:47.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:47.709+0000 7f34c90d8640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-03-29T10:56:47.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:47.711+0000 7f34c90d8640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-03-29T10:56:50.586 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-03-29T10:56:50.652 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-29T10:56:50.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:50.748+0000 7f47934cb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-03-29T10:56:50.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:50.748+0000 7f47934cb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-03-29T10:56:50.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:50.748+0000 7f47934cb640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-03-29T10:56:50.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:50.748+0000 7f47934cb640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-03-29T10:56:50.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:50.748+0000 7f47934cb640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-03-29T10:56:50.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:50.750+0000 7f47934cb640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-03-29T10:56:51.259 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T10:56:51.263 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T10:56:53.635 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-03-29T10:56:53.703 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-03-29T10:56:53.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:53.800+0000 7eff05bb7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-03-29T10:56:53.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:53.800+0000 7eff05bb7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-03-29T10:56:53.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:53.800+0000 7eff05bb7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-03-29T10:56:53.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:53.801+0000 7eff05bb7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-03-29T10:56:53.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:53.801+0000 7eff05bb7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-03-29T10:56:53.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:56:53.802+0000 7eff05bb7640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-03-29T10:56:56.683 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-03-29T10:56:56.745 INFO:tasks.ceph:Reading keys from all nodes... 2023-03-29T10:56:56.745 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:56:56.745 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-03-29T10:56:56.813 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:56:56.813 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-03-29T10:56:56.917 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:56:56.917 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-03-29T10:56:57.019 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:56:57.019 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-03-29T10:56:57.119 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:56:57.120 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-03-29T10:56:57.220 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:56:57.220 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-03-29T10:56:57.321 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:56:57.322 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-03-29T10:56:57.422 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:56:57.423 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-03-29T10:56:57.487 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:56:57.487 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-03-29T10:56:57.587 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:56:57.587 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-03-29T10:56:57.688 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:56:57.688 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-03-29T10:56:57.789 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:56:57.789 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-03-29T10:56:57.890 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:56:57.890 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-03-29T10:56:57.991 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:56:57.991 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-03-29T10:56:58.092 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:56:58.092 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-03-29T10:56:58.135 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:56:58.135 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-03-29T10:56:58.217 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:56:58.217 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-03-29T10:56:58.257 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:56:58.257 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-03-29T10:56:58.337 INFO:tasks.ceph:Adding keys to all mons... 2023-03-29T10:56:58.337 DEBUG:teuthology.orchestra.run.smithi111:> sudo tee -a /etc/ceph/ceph.keyring 2023-03-29T10:56:58.341 DEBUG:teuthology.orchestra.run.smithi119:> sudo tee -a /etc/ceph/ceph.keyring 2023-03-29T10:56:58.396 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr.y] 2023-03-29T10:56:58.397 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBYGSRkJ2+PKRAAGoCnOjmVtYroJKEzZHd84A== 2023-03-29T10:56:58.397 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.a] 2023-03-29T10:56:58.397 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBYGSRkbCL9NxAAPlUk/kxFQWaRb0KWzZgdzQ== 2023-03-29T10:56:58.397 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.c] 2023-03-29T10:56:58.397 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBZGSRkOlnjBhAAYuibQZYXCSfiZ+5g2HMMhg== 2023-03-29T10:56:58.397 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.0] 2023-03-29T10:56:58.397 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBdGSRkfthGDxAAu0H6zYyBtIiGcZhhgfA5tg== 2023-03-29T10:56:58.397 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.1] 2023-03-29T10:56:58.397 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBgGSRkTA6hERAAzWcslS2JXfC3Veik8854Kw== 2023-03-29T10:56:58.398 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.2] 2023-03-29T10:56:58.398 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBjGSRkIXt/ExAAKv910aQ0DXwlapS9BaLW5Q== 2023-03-29T10:56:58.398 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.3] 2023-03-29T10:56:58.398 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBmGSRkVSmZFhAAAOK61H8Drx0cx8YEsMP9Fg== 2023-03-29T10:56:58.398 INFO:teuthology.orchestra.run.smithi111.stdout:[mgr.x] 2023-03-29T10:56:58.398 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBYGSRkG0YRMRAAvfVgZ0GOj/NHocPhqdfBtA== 2023-03-29T10:56:58.398 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.b] 2023-03-29T10:56:58.398 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBZGSRkS+5iERAAMiKMA0iWZgOLY8o7YY5HBg== 2023-03-29T10:56:58.398 INFO:teuthology.orchestra.run.smithi111.stdout:[mds.d] 2023-03-29T10:56:58.398 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBZGSRkpWrNGxAA6QPfNxtu8SOwefAkYTq7NQ== 2023-03-29T10:56:58.399 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.4] 2023-03-29T10:56:58.399 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBsGSRkV31DJxAAiWiGcWrjbX5Bh38gDLG2Og== 2023-03-29T10:56:58.399 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.5] 2023-03-29T10:56:58.399 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBvGSRkP5JQKhAAaoAfEw3cCrBEcQhRNmwLrA== 2023-03-29T10:56:58.399 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.6] 2023-03-29T10:56:58.399 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQByGSRkhbiiLBAARXXU1Bwk7M3fLEDGnO+1Lw== 2023-03-29T10:56:58.399 INFO:teuthology.orchestra.run.smithi111.stdout:[osd.7] 2023-03-29T10:56:58.399 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQB1GSRkShfBLxAA/FxkMObHsaXgDF3CIv354A== 2023-03-29T10:56:58.399 INFO:teuthology.orchestra.run.smithi111.stdout:[client.0] 2023-03-29T10:56:58.399 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBZGSRkTCPmJBAAoBPWEMDsr6r1DUWLIYeCHw== 2023-03-29T10:56:58.400 INFO:teuthology.orchestra.run.smithi111.stdout:[client.2] 2023-03-29T10:56:58.400 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBZGSRkuR6dKxAAn3ZN4MVSbqXTTQQlwUVLHA== 2023-03-29T10:56:58.400 INFO:teuthology.orchestra.run.smithi111.stdout:[client.1] 2023-03-29T10:56:58.400 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBZGSRkhqZmMhAASfrTwS+BZGF/E6S34W+z6Q== 2023-03-29T10:56:58.400 INFO:teuthology.orchestra.run.smithi111.stdout:[client.3] 2023-03-29T10:56:58.400 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBZGSRkQfseORAAGOTyPhNRfqi0NxE+EvVRbw== 2023-03-29T10:56:58.436 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.y] 2023-03-29T10:56:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBYGSRkJ2+PKRAAGoCnOjmVtYroJKEzZHd84A== 2023-03-29T10:56:58.436 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a] 2023-03-29T10:56:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBYGSRkbCL9NxAAPlUk/kxFQWaRb0KWzZgdzQ== 2023-03-29T10:56:58.436 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c] 2023-03-29T10:56:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBZGSRkOlnjBhAAYuibQZYXCSfiZ+5g2HMMhg== 2023-03-29T10:56:58.436 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.0] 2023-03-29T10:56:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBdGSRkfthGDxAAu0H6zYyBtIiGcZhhgfA5tg== 2023-03-29T10:56:58.436 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.1] 2023-03-29T10:56:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBgGSRkTA6hERAAzWcslS2JXfC3Veik8854Kw== 2023-03-29T10:56:58.436 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.2] 2023-03-29T10:56:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBjGSRkIXt/ExAAKv910aQ0DXwlapS9BaLW5Q== 2023-03-29T10:56:58.436 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.3] 2023-03-29T10:56:58.436 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBmGSRkVSmZFhAAAOK61H8Drx0cx8YEsMP9Fg== 2023-03-29T10:56:58.437 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.x] 2023-03-29T10:56:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBYGSRkG0YRMRAAvfVgZ0GOj/NHocPhqdfBtA== 2023-03-29T10:56:58.437 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b] 2023-03-29T10:56:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBZGSRkS+5iERAAMiKMA0iWZgOLY8o7YY5HBg== 2023-03-29T10:56:58.437 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d] 2023-03-29T10:56:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBZGSRkpWrNGxAA6QPfNxtu8SOwefAkYTq7NQ== 2023-03-29T10:56:58.437 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.4] 2023-03-29T10:56:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBsGSRkV31DJxAAiWiGcWrjbX5Bh38gDLG2Og== 2023-03-29T10:56:58.437 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.5] 2023-03-29T10:56:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBvGSRkP5JQKhAAaoAfEw3cCrBEcQhRNmwLrA== 2023-03-29T10:56:58.437 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.6] 2023-03-29T10:56:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQByGSRkhbiiLBAARXXU1Bwk7M3fLEDGnO+1Lw== 2023-03-29T10:56:58.437 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.7] 2023-03-29T10:56:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQB1GSRkShfBLxAA/FxkMObHsaXgDF3CIv354A== 2023-03-29T10:56:58.437 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2023-03-29T10:56:58.437 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBZGSRkTCPmJBAAoBPWEMDsr6r1DUWLIYeCHw== 2023-03-29T10:56:58.438 INFO:teuthology.orchestra.run.smithi119.stdout:[client.2] 2023-03-29T10:56:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBZGSRkuR6dKxAAn3ZN4MVSbqXTTQQlwUVLHA== 2023-03-29T10:56:58.438 INFO:teuthology.orchestra.run.smithi119.stdout:[client.1] 2023-03-29T10:56:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBZGSRkhqZmMhAASfrTwS+BZGF/E6S34W+z6Q== 2023-03-29T10:56:58.438 INFO:teuthology.orchestra.run.smithi119.stdout:[client.3] 2023-03-29T10:56:58.438 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBZGSRkQfseORAAGOTyPhNRfqi0NxE+EvVRbw== 2023-03-29T10:56:58.439 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-29T10:56:58.445 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-29T10:56:58.576 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-29T10:56:58.579 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-29T10:56:58.680 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-29T10:56:58.683 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-29T10:56:58.784 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T10:56:58.787 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T10:56:58.886 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T10:56:58.889 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T10:56:58.987 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T10:56:58.990 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T10:56:59.088 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T10:56:59.092 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T10:56:59.189 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-29T10:56:59.193 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-03-29T10:56:59.291 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-29T10:56:59.294 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-29T10:56:59.396 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-29T10:56:59.400 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-03-29T10:56:59.501 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T10:56:59.505 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T10:56:59.603 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T10:56:59.607 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T10:56:59.705 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T10:56:59.709 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T10:56:59.807 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T10:56:59.810 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-03-29T10:56:59.908 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-29T10:56:59.911 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-29T10:57:00.012 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-29T10:57:00.015 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-29T10:57:00.115 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-29T10:57:00.119 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-29T10:57:00.222 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-29T10:57:00.226 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-03-29T10:57:00.328 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-03-29T10:57:00.328 DEBUG:teuthology.orchestra.run.smithi111:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-03-29T10:57:00.389 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-03-29T10:57:00.502 DEBUG:teuthology.orchestra.run.smithi111:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-03-29T10:57:00.564 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-03-29T10:57:00.626 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-03-29T10:57:00.737 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-03-29T10:57:00.799 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-03-29T10:57:00.859 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-03-29T10:57:00.971 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-03-29T10:57:01.032 DEBUG:teuthology.orchestra.run.smithi111:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-03-29T10:57:01.036 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-03-29T10:57:01.075 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-03-29T10:57:01.076 INFO:tasks.ceph.mon.a:Restarting daemon 2023-03-29T10:57:01.076 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-03-29T10:57:01.116 INFO:tasks.ceph.mon.a:Started 2023-03-29T10:57:01.117 INFO:tasks.ceph.mon.b:Restarting daemon 2023-03-29T10:57:01.117 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-03-29T10:57:01.120 INFO:tasks.ceph.mon.b:Started 2023-03-29T10:57:01.121 INFO:tasks.ceph.mon.c:Restarting daemon 2023-03-29T10:57:01.121 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-03-29T10:57:01.124 INFO:tasks.ceph.mon.c:Started 2023-03-29T10:57:01.124 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-03-29T10:57:01.124 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-03-29T10:57:01.124 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-03-29T10:57:01.127 INFO:tasks.ceph.mgr.y:Started 2023-03-29T10:57:01.128 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-03-29T10:57:01.128 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-03-29T10:57:01.131 INFO:tasks.ceph.mgr.x:Started 2023-03-29T10:57:01.131 INFO:tasks.ceph:Setting crush tunables to default 2023-03-29T10:57:01.131 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd crush tunables default 2023-03-29T10:57:01.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.317+0000 7f5ea07ff700 1 Processor -- start 2023-03-29T10:57:01.318 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.317+0000 7f5ea07ff700 1 -- start start 2023-03-29T10:57:01.318 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.318+0000 7f5ea07ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5e9815a5a0 0x7f5e9806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:01.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.319+0000 7f5ea07ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5e98071200 0x7f5e9806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:01.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.319+0000 7f5ea07ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e980cb280 con 0x7f5e9806a920 2023-03-29T10:57:01.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.319+0000 7f5ea07ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e9806adc0 con 0x7f5e98071200 2023-03-29T10:57:01.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.319+0000 7f5ea07ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e9815a970 con 0x7f5e9815a5a0 2023-03-29T10:57:01.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.319+0000 7f5e9ed9c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5e98071200 0x7f5e9806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:01.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.319+0000 7f5e9ed9c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f5e98071200 0x7f5e9806d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57014/0 (socket says 172.21.15.111:57014) 2023-03-29T10:57:01.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.319+0000 7f5e9dd9a700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f5e9806a920 0x7f5e98070af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:33586/0 (socket says 172.21.15.111:33586) 2023-03-29T10:57:01.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.319+0000 7f5e9ed9c700 1 -- 172.21.15.111:0/963893050 learned_addr learned my addr 172.21.15.111:0/963893050 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:01.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.319+0000 7f5e9ed9c700 1 -- 172.21.15.111:0/963893050 >> v2:172.21.15.111:3300/0 conn(0x7f5e98071200 msgr2=0x7f5e9806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-03-29T10:57:01.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.319+0000 7f5e9ed9c700 1 -- 172.21.15.111:0/963893050 >> v2:172.21.15.111:3300/0 conn(0x7f5e98071200 msgr2=0x7f5e9806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-29T10:57:01.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.319+0000 7f5e9ed9c700 1 --2- 172.21.15.111:0/963893050 >> v2:172.21.15.111:3300/0 conn(0x7f5e98071200 0x7f5e9806d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-29T10:57:01.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.319+0000 7f5e9ed9c700 1 --2- 172.21.15.111:0/963893050 >> v2:172.21.15.111:3300/0 conn(0x7f5e98071200 0x7f5e9806d1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-29T10:57:01.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.319+0000 7f5e9e59b700 1 --2- 172.21.15.111:0/963893050 >> v2:172.21.15.119:3300/0 conn(0x7f5e9815a5a0 0x7f5e9806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:01.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.320+0000 7f5e9e59b700 1 -- 172.21.15.111:0/963893050 >> v2:172.21.15.119:3300/0 conn(0x7f5e9815a5a0 msgr2=0x7f5e9806a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-03-29T10:57:01.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.320+0000 7f5e9e59b700 1 -- 172.21.15.111:0/963893050 >> v2:172.21.15.119:3300/0 conn(0x7f5e9815a5a0 msgr2=0x7f5e9806a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-03-29T10:57:01.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.320+0000 7f5e9e59b700 1 --2- 172.21.15.111:0/963893050 >> v2:172.21.15.119:3300/0 conn(0x7f5e9815a5a0 0x7f5e9806a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-03-29T10:57:01.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.320+0000 7f5e9e59b700 1 --2- 172.21.15.111:0/963893050 >> v2:172.21.15.119:3300/0 conn(0x7f5e9815a5a0 0x7f5e9806a3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-03-29T10:57:01.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.516+0000 7f5e9d599700 1 -- 172.21.15.111:0/963893050 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830158910 0 0) 0x7f5e880011d0 con 0x7f5e9806a920 2023-03-29T10:57:01.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.516+0000 7f5e9d599700 1 -- 172.21.15.111:0/963893050 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e840035b0 con 0x7f5e9806a920 2023-03-29T10:57:01.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.517+0000 7f5e9d599700 1 -- 172.21.15.111:0/963893050 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1562071372 0 0) 0x7f5e840035b0 con 0x7f5e9806a920 2023-03-29T10:57:01.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.517+0000 7f5e9d599700 1 -- 172.21.15.111:0/963893050 >> v2:172.21.15.119:3300/0 conn(0x7f5e9815a5a0 msgr2=0x7f5e9806a3e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-29T10:57:01.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.517+0000 7f5e9d599700 1 --2- 172.21.15.111:0/963893050 >> v2:172.21.15.119:3300/0 conn(0x7f5e9815a5a0 0x7f5e9806a3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:01.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.517+0000 7f5e9d599700 1 -- 172.21.15.111:0/963893050 >> v2:172.21.15.111:3300/0 conn(0x7f5e98071200 msgr2=0x7f5e9806d1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-29T10:57:01.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.517+0000 7f5e9d599700 1 --2- 172.21.15.111:0/963893050 >> v2:172.21.15.111:3300/0 conn(0x7f5e98071200 0x7f5e9806d1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:01.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.517+0000 7f5e9d599700 1 -- 172.21.15.111:0/963893050 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e98071700 con 0x7f5e9806a920 2023-03-29T10:57:01.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.517+0000 7f5e9d599700 1 -- 172.21.15.111:0/963893050 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f5e880018c0 con 0x7f5e9806a920 2023-03-29T10:57:01.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.517+0000 7f5e9d599700 1 -- 172.21.15.111:0/963893050 >> v1:172.21.15.111:6789/0 conn(0x7f5e9806a920 legacy=0x7f5e98070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:01.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.517+0000 7f5e9d599700 1 --2- 172.21.15.111:0/963893050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e84003a80 0x7f5e84003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:01.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.517+0000 7f5e9d599700 1 --2- 172.21.15.111:0/963893050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e84004850 0x7f5e84004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:01.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.517+0000 7f5e9d599700 1 --2- 172.21.15.111:0/963893050 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5e84005200 0x7f5e8400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:01.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.517+0000 7f5e9d599700 1 -- 172.21.15.111:0/963893050 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9815a970 con 0x7f5e84003a80 2023-03-29T10:57:01.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.517+0000 7f5e9d599700 1 -- 172.21.15.111:0/963893050 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8400c0e0 con 0x7f5e84004850 2023-03-29T10:57:01.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.517+0000 7f5e9d599700 1 -- 172.21.15.111:0/963893050 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8400c2a0 con 0x7f5e84005200 2023-03-29T10:57:01.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.517+0000 7f5e9dd9a700 1 --2- 172.21.15.111:0/963893050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e84004850 0x7f5e84004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:01.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.517+0000 7f5e9e59b700 1 --2- 172.21.15.111:0/963893050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e84003a80 0x7f5e84003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:01.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.517+0000 7f5e9ed9c700 1 --2- 172.21.15.111:0/963893050 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5e84005200 0x7f5e8400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:01.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.518+0000 7f5e9e59b700 1 -- 172.21.15.111:0/963893050 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5e84005200 msgr2=0x7f5e8400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:01.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.518+0000 7f5e9e59b700 1 --2- 172.21.15.111:0/963893050 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5e84005200 0x7f5e8400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:01.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.518+0000 7f5e9e59b700 1 -- 172.21.15.111:0/963893050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e84004850 msgr2=0x7f5e84004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:01.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.518+0000 7f5e9e59b700 1 --2- 172.21.15.111:0/963893050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e84004850 0x7f5e84004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:01.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.518+0000 7f5e9e59b700 1 -- 172.21.15.111:0/963893050 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e840035b0 con 0x7f5e84003a80 2023-03-29T10:57:01.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.518+0000 7f5e9ed9c700 1 -- 172.21.15.111:0/963893050 reap_dead start 2023-03-29T10:57:01.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.518+0000 7f5e9e59b700 1 --2- 172.21.15.111:0/963893050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e84003a80 0x7f5e84003e90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009860 tx=0x7f5e84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c320ebd242b8aa server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:01.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.519+0000 7f5e9d599700 1 -- 172.21.15.111:0/963893050 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9400c070 con 0x7f5e84003a80 2023-03-29T10:57:01.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.519+0000 7f5e9d599700 1 -- 172.21.15.111:0/963893050 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e94004020 con 0x7f5e84003a80 2023-03-29T10:57:01.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.519+0000 7f5e9d599700 1 -- 172.21.15.111:0/963893050 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e940043b0 con 0x7f5e84003a80 2023-03-29T10:57:01.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.520+0000 7f5ea07ff700 1 -- 172.21.15.111:0/963893050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e84003a80 msgr2=0x7f5e84003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:01.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.520+0000 7f5ea07ff700 1 --2- 172.21.15.111:0/963893050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e84003a80 0x7f5e84003e90 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009860 tx=0x7f5e84005880 comp rx=0 tx=0).stop 2023-03-29T10:57:01.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.520+0000 7f5ea07ff700 1 -- 172.21.15.111:0/963893050 shutdown_connections 2023-03-29T10:57:01.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.520+0000 7f5ea07ff700 1 --2- 172.21.15.111:0/963893050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e84003a80 0x7f5e84003e90 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:01.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.520+0000 7f5ea07ff700 1 -- 172.21.15.111:0/963893050 >> 172.21.15.111:0/963893050 conn(0x7f5e980cf720 msgr2=0x7f5e980cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:01.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.520+0000 7f5ea07ff700 1 -- 172.21.15.111:0/963893050 shutdown_connections 2023-03-29T10:57:01.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.520+0000 7f5ea07ff700 1 -- 172.21.15.111:0/963893050 wait complete. 2023-03-29T10:57:01.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.523+0000 7f5ea07ff700 1 Processor -- start 2023-03-29T10:57:01.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.523+0000 7f5ea07ff700 1 -- start start 2023-03-29T10:57:01.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.525+0000 7f5ea07ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e84003a80 0x7f5e980ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:01.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.525+0000 7f5ea07ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e98151370 0x7f5e981fa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:01.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.525+0000 7f5ea07ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5e98155340 0x7f5e981557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:01.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.525+0000 7f5ea07ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e98155d20 con 0x7f5e98151370 2023-03-29T10:57:01.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.525+0000 7f5ea07ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e981fad30 con 0x7f5e84003a80 2023-03-29T10:57:01.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.525+0000 7f5ea07ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e980c2530 con 0x7f5e98155340 2023-03-29T10:57:01.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.525+0000 7f5e9dd9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e98151370 0x7f5e981fa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:01.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.525+0000 7f5e9e59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e84003a80 0x7f5e980ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:01.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.525+0000 7f5e9dd9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e98151370 0x7f5e981fa7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57044/0 (socket says 172.21.15.111:57044) 2023-03-29T10:57:01.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.525+0000 7f5e9dd9a700 1 -- 172.21.15.111:0/2304617908 learned_addr learned my addr 172.21.15.111:0/2304617908 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:01.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.525+0000 7f5e9ed9c700 1 --2- 172.21.15.111:0/2304617908 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5e98155340 0x7f5e981557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:01.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.526+0000 7f5e9dd9a700 1 -- 172.21.15.111:0/2304617908 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5e98155340 msgr2=0x7f5e981557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:01.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.526+0000 7f5e9dd9a700 1 --2- 172.21.15.111:0/2304617908 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5e98155340 0x7f5e981557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:01.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.526+0000 7f5e9dd9a700 1 -- 172.21.15.111:0/2304617908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e84003a80 msgr2=0x7f5e980ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:01.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.526+0000 7f5e9dd9a700 1 --2- 172.21.15.111:0/2304617908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e84003a80 0x7f5e980ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:01.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.526+0000 7f5e9dd9a700 1 -- 172.21.15.111:0/2304617908 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9806c6c0 con 0x7f5e98151370 2023-03-29T10:57:01.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.526+0000 7f5e9e59b700 1 --2- 172.21.15.111:0/2304617908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e84003a80 0x7f5e980ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:01.527 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.526+0000 7f5e9dd9a700 1 --2- 172.21.15.111:0/2304617908 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e98151370 0x7f5e981fa7f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5e88007fd0 tx=0x7f5e8800fb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:01.527 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.526+0000 7f5e8effd700 1 -- 172.21.15.111:0/2304617908 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e880072e0 con 0x7f5e98151370 2023-03-29T10:57:01.527 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.526+0000 7f5e8effd700 1 -- 172.21.15.111:0/2304617908 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e88007440 con 0x7f5e98151370 2023-03-29T10:57:01.527 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.526+0000 7f5ea07ff700 1 -- 172.21.15.111:0/2304617908 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e980719e0 con 0x7f5e98151370 2023-03-29T10:57:01.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.526+0000 7f5e8effd700 1 -- 172.21.15.111:0/2304617908 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e88005830 con 0x7f5e98151370 2023-03-29T10:57:01.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.526+0000 7f5ea07ff700 1 -- 172.21.15.111:0/2304617908 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e9806c9a0 con 0x7f5e98151370 2023-03-29T10:57:01.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.527+0000 7f5e8effd700 1 -- 172.21.15.111:0/2304617908 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5e88005c90 con 0x7f5e98151370 2023-03-29T10:57:01.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.527+0000 7f5e8effd700 1 -- 172.21.15.111:0/2304617908 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5e8801dde0 con 0x7f5e98151370 2023-03-29T10:57:01.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.527+0000 7f5ea07ff700 1 -- 172.21.15.111:0/2304617908 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e980ca1e0 con 0x7f5e98151370 2023-03-29T10:57:01.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.530+0000 7f5e8effd700 1 -- 172.21.15.111:0/2304617908 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f5e980cb280 con 0x7f5e98151370 2023-03-29T10:57:01.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.606+0000 7f5ea07ff700 1 -- 172.21.15.111:0/2304617908 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f5e980cbcf0 con 0x7f5e98151370 2023-03-29T10:57:01.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.609+0000 7f5e8effd700 1 -- 172.21.15.111:0/2304617908 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f5e8801d460 con 0x7f5e98151370 2023-03-29T10:57:01.610 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.610+0000 7f5ea07ff700 1 -- 172.21.15.111:0/2304617908 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e98151370 msgr2=0x7f5e981fa7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:01.610 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.610+0000 7f5ea07ff700 1 --2- 172.21.15.111:0/2304617908 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e98151370 0x7f5e981fa7f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5e88007fd0 tx=0x7f5e8800fb90 comp rx=0 tx=0).stop 2023-03-29T10:57:01.610 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.610+0000 7f5ea07ff700 1 -- 172.21.15.111:0/2304617908 shutdown_connections 2023-03-29T10:57:01.610 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.610+0000 7f5ea07ff700 1 --2- 172.21.15.111:0/2304617908 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5e98155340 0x7f5e981557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:01.610 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.610+0000 7f5ea07ff700 1 --2- 172.21.15.111:0/2304617908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e84003a80 0x7f5e980ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:01.610 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.610+0000 7f5ea07ff700 1 --2- 172.21.15.111:0/2304617908 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5e98151370 0x7f5e981fa7f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:01.610 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.610+0000 7f5ea07ff700 1 -- 172.21.15.111:0/2304617908 >> 172.21.15.111:0/2304617908 conn(0x7f5e980cf720 msgr2=0x7f5e980b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:01.610 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.610+0000 7f5ea07ff700 1 -- 172.21.15.111:0/2304617908 shutdown_connections 2023-03-29T10:57:01.610 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:01.610+0000 7f5ea07ff700 1 -- 172.21.15.111:0/2304617908 wait complete. 2023-03-29T10:57:01.610 INFO:teuthology.orchestra.run.smithi111.stderr:adjusted tunables profile to default 2023-03-29T10:57:01.627 INFO:tasks.ceph:check_enable_crimson: False 2023-03-29T10:57:01.627 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-03-29T10:57:01.627 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:57:01.627 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-03-29T10:57:01.689 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:57:01.689 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-03-29T10:57:01.790 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:57:01.790 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-03-29T10:57:01.891 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:57:01.891 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-03-29T10:57:01.993 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:57:01.993 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-03-29T10:57:02.054 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:57:02.054 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-03-29T10:57:02.156 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:57:02.156 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-03-29T10:57:02.258 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:57:02.258 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-03-29T10:57:02.361 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 07696850-0517-429c-be4a-d6f8c2cf3d01 0 2023-03-29T10:57:02.585 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:02.585+0000 7fa172b1e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-03-29T10:57:02.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.586+0000 7f1f15bff700 1 Processor -- start 2023-03-29T10:57:02.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.586+0000 7f1f15bff700 1 -- start start 2023-03-29T10:57:02.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.587+0000 7f1f15bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1f100b87a0 0x7f1f100cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:02.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.587+0000 7f1f15bff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1f10160d40 0x7f1f100d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:02.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.588+0000 7f1f15bff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f100cdc60 con 0x7f1f101601e0 2023-03-29T10:57:02.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.588+0000 7f1f15bff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f100d02d0 con 0x7f1f10160d40 2023-03-29T10:57:02.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.588+0000 7f1f15bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f100d0cb0 con 0x7f1f100b87a0 2023-03-29T10:57:02.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.588+0000 7f1f0f7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1f100b87a0 0x7f1f100cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:02.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.588+0000 7f1f0f7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1f100b87a0 0x7f1f100cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51732/0 (socket says 172.21.15.119:51732) 2023-03-29T10:57:02.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.588+0000 7f1f0f7fe700 1 -- 172.21.15.119:0/3031381951 learned_addr learned my addr 172.21.15.119:0/3031381951 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-03-29T10:57:02.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.588+0000 7f1f0ffff700 1 --2- 172.21.15.119:0/3031381951 >> v2:172.21.15.111:3300/0 conn(0x7f1f10160d40 0x7f1f100d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:02.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.588+0000 7f1f0f7fe700 1 -- 172.21.15.119:0/3031381951 >> v2:172.21.15.111:3300/0 conn(0x7f1f10160d40 msgr2=0x7f1f100d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:02.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.588+0000 7f1f0f7fe700 1 --2- 172.21.15.119:0/3031381951 >> v2:172.21.15.111:3300/0 conn(0x7f1f10160d40 0x7f1f100d1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:02.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.588+0000 7f1f0f7fe700 1 -- 172.21.15.119:0/3031381951 >> v1:172.21.15.111:6789/0 conn(0x7f1f101601e0 legacy=0x7f1f10160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:02.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.588+0000 7f1f0f7fe700 1 -- 172.21.15.119:0/3031381951 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f100cd6a0 con 0x7f1f100b87a0 2023-03-29T10:57:02.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.588+0000 7f1f0f7fe700 1 --2- 172.21.15.119:0/3031381951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f100b87a0 0x7f1f100cfbf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f100b4ac0 tx=0x7f1f00009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db86db4df737c037 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:02.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.589+0000 7f1f0effd700 1 -- 172.21.15.119:0/3031381951 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00011830 con 0x7f1f100b87a0 2023-03-29T10:57:02.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.589+0000 7f1f0effd700 1 -- 172.21.15.119:0/3031381951 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f00011990 con 0x7f1f100b87a0 2023-03-29T10:57:02.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.589+0000 7f1f0effd700 1 -- 172.21.15.119:0/3031381951 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00011d20 con 0x7f1f100b87a0 2023-03-29T10:57:02.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.589+0000 7f1f15bff700 1 -- 172.21.15.119:0/3031381951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f100b87a0 msgr2=0x7f1f100cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:02.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.589+0000 7f1f15bff700 1 --2- 172.21.15.119:0/3031381951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f100b87a0 0x7f1f100cfbf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f100b4ac0 tx=0x7f1f00009bd0 comp rx=0 tx=0).stop 2023-03-29T10:57:02.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.589+0000 7f1f15bff700 1 -- 172.21.15.119:0/3031381951 shutdown_connections 2023-03-29T10:57:02.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.589+0000 7f1f15bff700 1 --2- 172.21.15.119:0/3031381951 >> v2:172.21.15.111:3300/0 conn(0x7f1f10160d40 0x7f1f100d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:02.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.589+0000 7f1f15bff700 1 --2- 172.21.15.119:0/3031381951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f100b87a0 0x7f1f100cfbf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:02.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.589+0000 7f1f15bff700 1 -- 172.21.15.119:0/3031381951 >> 172.21.15.119:0/3031381951 conn(0x7f1f10071b10 msgr2=0x7f1f10071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:02.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.589+0000 7f1f15bff700 1 -- 172.21.15.119:0/3031381951 shutdown_connections 2023-03-29T10:57:02.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.590+0000 7f1f15bff700 1 -- 172.21.15.119:0/3031381951 wait complete. 2023-03-29T10:57:02.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.591+0000 7f1f15bff700 1 Processor -- start 2023-03-29T10:57:02.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.591+0000 7f1f15bff700 1 -- start start 2023-03-29T10:57:02.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.592+0000 7f1f15bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1f100b87a0 0x7f1f1014ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:02.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.592+0000 7f1f15bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f101601e0 0x7f1f100cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:02.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.592+0000 7f1f15bff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f10160d40 0x7f1f100caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:02.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.592+0000 7f1f15bff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f100c9f80 con 0x7f1f10160d40 2023-03-29T10:57:02.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.592+0000 7f1f15bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f100ca1d0 con 0x7f1f101601e0 2023-03-29T10:57:02.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.592+0000 7f1f15bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f100ca4b0 con 0x7f1f100b87a0 2023-03-29T10:57:02.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.592+0000 7f1f07fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f101601e0 0x7f1f100cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:02.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.593+0000 7f1f07fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f101601e0 0x7f1f100cc550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51736/0 (socket says 172.21.15.119:51736) 2023-03-29T10:57:02.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.593+0000 7f1f07fff700 1 -- 172.21.15.119:0/2728061842 learned_addr learned my addr 172.21.15.119:0/2728061842 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-03-29T10:57:02.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.593+0000 7f1f0f7fe700 1 --2- 172.21.15.119:0/2728061842 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1f100b87a0 0x7f1f1014ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:02.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.593+0000 7f1f0ffff700 1 --2- 172.21.15.119:0/2728061842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f10160d40 0x7f1f100caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:02.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.593+0000 7f1f07fff700 1 -- 172.21.15.119:0/2728061842 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1f100b87a0 msgr2=0x7f1f1014ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:02.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.593+0000 7f1f07fff700 1 --2- 172.21.15.119:0/2728061842 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1f100b87a0 0x7f1f1014ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:02.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.593+0000 7f1f07fff700 1 -- 172.21.15.119:0/2728061842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f10160d40 msgr2=0x7f1f100caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:02.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.593+0000 7f1f07fff700 1 --2- 172.21.15.119:0/2728061842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f10160d40 0x7f1f100caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:02.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.593+0000 7f1f07fff700 1 -- 172.21.15.119:0/2728061842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f00009550 con 0x7f1f101601e0 2023-03-29T10:57:02.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.593+0000 7f1f0f7fe700 1 --2- 172.21.15.119:0/2728061842 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1f100b87a0 0x7f1f1014ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:02.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.593+0000 7f1f07fff700 1 --2- 172.21.15.119:0/2728061842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f101601e0 0x7f1f100cc550 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800c4e0 tx=0x7f1ef800c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:02.596 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:02.594+0000 7fe1bee6a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-03-29T10:57:02.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.593+0000 7f1f0cff9700 1 -- 172.21.15.119:0/2728061842 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef80052d0 con 0x7f1f101601e0 2023-03-29T10:57:02.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.593+0000 7f1f0cff9700 1 -- 172.21.15.119:0/2728061842 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef8005430 con 0x7f1f101601e0 2023-03-29T10:57:02.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.593+0000 7f1f0cff9700 1 -- 172.21.15.119:0/2728061842 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef80038c0 con 0x7f1f101601e0 2023-03-29T10:57:02.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.593+0000 7f1f15bff700 1 -- 172.21.15.119:0/2728061842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f100d1640 con 0x7f1f101601e0 2023-03-29T10:57:02.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.594+0000 7f1f15bff700 1 -- 172.21.15.119:0/2728061842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f100cd980 con 0x7f1f101601e0 2023-03-29T10:57:02.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.594+0000 7f1f0cff9700 1 -- 172.21.15.119:0/2728061842 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1ef8003d20 con 0x7f1f101601e0 2023-03-29T10:57:02.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.594+0000 7f1f0cff9700 1 -- 172.21.15.119:0/2728061842 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1ef801c780 con 0x7f1f101601e0 2023-03-29T10:57:02.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.595+0000 7f1f15bff700 1 -- 172.21.15.119:0/2728061842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f100d17d0 con 0x7f1f101601e0 2023-03-29T10:57:02.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.597+0000 7f1f0cff9700 1 -- 172.21.15.119:0/2728061842 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f1ef8017070 con 0x7f1f101601e0 2023-03-29T10:57:02.675 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:02.675+0000 7fa172b1e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-03-29T10:57:02.681 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:02.682+0000 7fe1bee6a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-03-29T10:57:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.684+0000 7f1f15bff700 1 -- 172.21.15.119:0/2728061842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "07696850-0517-429c-be4a-d6f8c2cf3d01", "id": 0} v 0) v1 -- 0x7f1f10074430 con 0x7f1f101601e0 2023-03-29T10:57:02.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.689+0000 7f1f0cff9700 1 -- 172.21.15.119:0/2728061842 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "07696850-0517-429c-be4a-d6f8c2cf3d01", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1ef8011d60 con 0x7f1f101601e0 2023-03-29T10:57:02.689 INFO:teuthology.orchestra.run.smithi119.stdout:0 2023-03-29T10:57:02.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.690+0000 7f1f15bff700 1 -- 172.21.15.119:0/2728061842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f101601e0 msgr2=0x7f1f100cc550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:02.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.690+0000 7f1f15bff700 1 --2- 172.21.15.119:0/2728061842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f101601e0 0x7f1f100cc550 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800c4e0 tx=0x7f1ef800c7f0 comp rx=0 tx=0).stop 2023-03-29T10:57:02.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.690+0000 7f1f15bff700 1 -- 172.21.15.119:0/2728061842 shutdown_connections 2023-03-29T10:57:02.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.690+0000 7f1f15bff700 1 --2- 172.21.15.119:0/2728061842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f10160d40 0x7f1f100caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:02.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.690+0000 7f1f15bff700 1 --2- 172.21.15.119:0/2728061842 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1f100b87a0 0x7f1f1014ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:02.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.690+0000 7f1f15bff700 1 --2- 172.21.15.119:0/2728061842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f101601e0 0x7f1f100cc550 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:02.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.690+0000 7f1f15bff700 1 -- 172.21.15.119:0/2728061842 >> 172.21.15.119:0/2728061842 conn(0x7f1f10071b10 msgr2=0x7f1f100d33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:02.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.691+0000 7f1f15bff700 1 -- 172.21.15.119:0/2728061842 shutdown_connections 2023-03-29T10:57:02.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.691+0000 7f1f15bff700 1 -- 172.21.15.119:0/2728061842 wait complete. 2023-03-29T10:57:02.707 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 818f1f56-1dcf-4d4c-870f-5ca3c9830c45 1 2023-03-29T10:57:02.780 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:02.781+0000 7fa172b1e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-03-29T10:57:02.783 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:02.784+0000 7fe1bee6a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-03-29T10:57:02.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.889+0000 7ff452bff700 1 Processor -- start 2023-03-29T10:57:02.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.890+0000 7ff452bff700 1 -- start start 2023-03-29T10:57:02.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.891+0000 7ff452bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff44c15c590 0x7ff44c160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:02.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.891+0000 7ff452bff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff44c0cd520 con 0x7ff44c0cf010 2023-03-29T10:57:02.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.891+0000 7ff452bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff44c0d2640 con 0x7ff44c0cff50 2023-03-29T10:57:02.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.891+0000 7ff452bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff44c1610d0 con 0x7ff44c15c590 2023-03-29T10:57:02.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.891+0000 7ff45099b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff44c15c590 0x7ff44c160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:02.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.891+0000 7ff45099b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff44c15c590 0x7ff44c160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51748/0 (socket says 172.21.15.119:51748) 2023-03-29T10:57:02.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.891+0000 7ff45099b700 1 -- 172.21.15.119:0/445730245 learned_addr learned my addr 172.21.15.119:0/445730245 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-03-29T10:57:02.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.892+0000 7ff45099b700 1 -- 172.21.15.119:0/445730245 >> v1:172.21.15.119:6789/0 conn(0x7ff44c0cff50 legacy=0x7ff44c0d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:02.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.892+0000 7ff45099b700 1 -- 172.21.15.119:0/445730245 >> v1:172.21.15.111:6789/0 conn(0x7ff44c0cf010 legacy=0x7ff44c0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:02.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.892+0000 7ff45099b700 1 -- 172.21.15.119:0/445730245 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44c0b23c0 con 0x7ff44c15c590 2023-03-29T10:57:02.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.892+0000 7ff44b7fe700 1 -- 172.21.15.119:0/445730245 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1035963659 0 0) 0x7ff44c0cd520 con 0x7ff44c0cf010 2023-03-29T10:57:02.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.892+0000 7ff45099b700 1 --2- 172.21.15.119:0/445730245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff44c15c590 0x7ff44c160b90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff4400098f0 tx=0x7ff440009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32f31977a04e3a9a server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:02.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.892+0000 7ff44b7fe700 1 -- 172.21.15.119:0/445730245 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44000b8a0 con 0x7ff44c15c590 2023-03-29T10:57:02.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.892+0000 7ff44b7fe700 1 -- 172.21.15.119:0/445730245 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44000ba00 con 0x7ff44c15c590 2023-03-29T10:57:02.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.892+0000 7ff44b7fe700 1 -- 172.21.15.119:0/445730245 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44000bd90 con 0x7ff44c15c590 2023-03-29T10:57:02.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.893+0000 7ff452bff700 1 -- 172.21.15.119:0/445730245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff44c15c590 msgr2=0x7ff44c160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:02.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.893+0000 7ff452bff700 1 --2- 172.21.15.119:0/445730245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff44c15c590 0x7ff44c160b90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff4400098f0 tx=0x7ff440009c00 comp rx=0 tx=0).stop 2023-03-29T10:57:02.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.893+0000 7ff452bff700 1 -- 172.21.15.119:0/445730245 shutdown_connections 2023-03-29T10:57:02.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.893+0000 7ff452bff700 1 --2- 172.21.15.119:0/445730245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff44c15c590 0x7ff44c160b90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:02.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.893+0000 7ff452bff700 1 -- 172.21.15.119:0/445730245 >> 172.21.15.119:0/445730245 conn(0x7ff44c071b10 msgr2=0x7ff44c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:02.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.893+0000 7ff452bff700 1 -- 172.21.15.119:0/445730245 shutdown_connections 2023-03-29T10:57:02.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.893+0000 7ff452bff700 1 -- 172.21.15.119:0/445730245 wait complete. 2023-03-29T10:57:02.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.895+0000 7ff452bff700 1 Processor -- start 2023-03-29T10:57:02.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.895+0000 7ff452bff700 1 -- start start 2023-03-29T10:57:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.897+0000 7ff452bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff44c0cf010 0x7ff44c14d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.897+0000 7ff452bff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff44c0cff50 0x7ff44c0c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.897+0000 7ff452bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff44c15c590 0x7ff44c0b7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.897+0000 7ff452bff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c14d850 con 0x7ff44c0cff50 2023-03-29T10:57:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.897+0000 7ff452bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0b6270 con 0x7ff44c0cf010 2023-03-29T10:57:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.897+0000 7ff452bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff44c0b6550 con 0x7ff44c15c590 2023-03-29T10:57:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.897+0000 7ff45099b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff44c0cf010 0x7ff44c14d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.897+0000 7ff45099b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff44c0cf010 0x7ff44c14d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51762/0 (socket says 172.21.15.119:51762) 2023-03-29T10:57:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.897+0000 7ff45099b700 1 -- 172.21.15.119:0/3355286241 learned_addr learned my addr 172.21.15.119:0/3355286241 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-03-29T10:57:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.897+0000 7ff44bfff700 1 --2- 172.21.15.119:0/3355286241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff44c0cff50 0x7ff44c0c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.897+0000 7ff45119c700 1 --2- 172.21.15.119:0/3355286241 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff44c15c590 0x7ff44c0b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:02.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.897+0000 7ff45099b700 1 -- 172.21.15.119:0/3355286241 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff44c15c590 msgr2=0x7ff44c0b7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:02.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.897+0000 7ff45099b700 1 --2- 172.21.15.119:0/3355286241 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff44c15c590 0x7ff44c0b7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:02.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.897+0000 7ff45099b700 1 -- 172.21.15.119:0/3355286241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff44c0cff50 msgr2=0x7ff44c0c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:02.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.897+0000 7ff45099b700 1 --2- 172.21.15.119:0/3355286241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff44c0cff50 0x7ff44c0c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:02.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.897+0000 7ff45099b700 1 -- 172.21.15.119:0/3355286241 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff440009550 con 0x7ff44c0cf010 2023-03-29T10:57:02.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.898+0000 7ff45099b700 1 --2- 172.21.15.119:0/3355286241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff44c0cf010 0x7ff44c14d310 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff440014040 tx=0x7ff440004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:02.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.898+0000 7ff4497fa700 1 -- 172.21.15.119:0/3355286241 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff440011970 con 0x7ff44c0cf010 2023-03-29T10:57:02.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.898+0000 7ff4497fa700 1 -- 172.21.15.119:0/3355286241 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44001e070 con 0x7ff44c0cf010 2023-03-29T10:57:02.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.898+0000 7ff4497fa700 1 -- 172.21.15.119:0/3355286241 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff440011cd0 con 0x7ff44c0cf010 2023-03-29T10:57:02.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.898+0000 7ff452bff700 1 -- 172.21.15.119:0/3355286241 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff44c0cd770 con 0x7ff44c0cf010 2023-03-29T10:57:02.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.898+0000 7ff452bff700 1 -- 172.21.15.119:0/3355286241 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44c0cf560 con 0x7ff44c0cf010 2023-03-29T10:57:02.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.898+0000 7ff4497fa700 1 -- 172.21.15.119:0/3355286241 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff44001a720 con 0x7ff44c0cf010 2023-03-29T10:57:02.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.898+0000 7ff4497fa700 1 -- 172.21.15.119:0/3355286241 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7ff44001ad80 con 0x7ff44c0cf010 2023-03-29T10:57:02.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.899+0000 7ff452bff700 1 -- 172.21.15.119:0/3355286241 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44c15c960 con 0x7ff44c0cf010 2023-03-29T10:57:02.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.901+0000 7ff4497fa700 1 -- 172.21.15.119:0/3355286241 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7ff440020070 con 0x7ff44c0cf010 2023-03-29T10:57:02.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.986+0000 7ff452bff700 1 -- 172.21.15.119:0/3355286241 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "818f1f56-1dcf-4d4c-870f-5ca3c9830c45", "id": 1} v 0) v1 -- 0x7ff44c0b1f00 con 0x7ff44c0cf010 2023-03-29T10:57:02.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.994+0000 7ff4497fa700 1 -- 172.21.15.119:0/3355286241 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "818f1f56-1dcf-4d4c-870f-5ca3c9830c45", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff440015020 con 0x7ff44c0cf010 2023-03-29T10:57:02.994 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-03-29T10:57:02.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.995+0000 7ff452bff700 1 -- 172.21.15.119:0/3355286241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff44c0cf010 msgr2=0x7ff44c14d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:02.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.995+0000 7ff452bff700 1 --2- 172.21.15.119:0/3355286241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff44c0cf010 0x7ff44c14d310 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff440014040 tx=0x7ff440004750 comp rx=0 tx=0).stop 2023-03-29T10:57:02.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.995+0000 7ff452bff700 1 -- 172.21.15.119:0/3355286241 shutdown_connections 2023-03-29T10:57:02.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.995+0000 7ff452bff700 1 --2- 172.21.15.119:0/3355286241 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff44c15c590 0x7ff44c0b7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:02.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.995+0000 7ff452bff700 1 --2- 172.21.15.119:0/3355286241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff44c0cf010 0x7ff44c14d310 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:02.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.995+0000 7ff452bff700 1 --2- 172.21.15.119:0/3355286241 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff44c0cff50 0x7ff44c0c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:02.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.995+0000 7ff452bff700 1 -- 172.21.15.119:0/3355286241 >> 172.21.15.119:0/3355286241 conn(0x7ff44c071b10 msgr2=0x7ff44c15b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:02.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.995+0000 7ff452bff700 1 -- 172.21.15.119:0/3355286241 shutdown_connections 2023-03-29T10:57:02.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:02.995+0000 7ff452bff700 1 -- 172.21.15.119:0/3355286241 wait complete. 2023-03-29T10:57:03.007 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:03.007+0000 7fe1bee6a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-03-29T10:57:03.011 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 7943699f-e24a-4905-a599-f7139d3811ba 2 2023-03-29T10:57:03.014 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:03.014+0000 7fa172b1e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-03-29T10:57:03.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.191+0000 7fe8f4fff700 1 Processor -- start 2023-03-29T10:57:03.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.191+0000 7fe8f4fff700 1 -- start start 2023-03-29T10:57:03.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.192+0000 7fe8f4fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe8f00b1d50 0x7fe8f006b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:03.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.192+0000 7fe8f4fff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f006a5f0 con 0x7fe8f00cd790 2023-03-29T10:57:03.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.192+0000 7fe8f4fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f006a840 con 0x7fe8f00ca210 2023-03-29T10:57:03.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.192+0000 7fe8f4fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f00ca6b0 con 0x7fe8f00b1d50 2023-03-29T10:57:03.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.193+0000 7fe8e7fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe8f00ca210 0x7fe8f00b1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59162/0 (socket says 172.21.15.119:59162) 2023-03-29T10:57:03.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.193+0000 7fe8e7fff700 1 -- 172.21.15.119:0/3521242475 learned_addr learned my addr 172.21.15.119:0/3521242475 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-03-29T10:57:03.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.193+0000 7fe8eed9d700 1 --2- 172.21.15.119:0/3521242475 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe8f00b1d50 0x7fe8f006b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:03.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.193+0000 7fe8edd9b700 1 -- 172.21.15.119:0/3521242475 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190865114 0 0) 0x7fe8f006a840 con 0x7fe8f00ca210 2023-03-29T10:57:03.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.193+0000 7fe8eed9d700 1 -- 172.21.15.119:0/3521242475 >> v1:172.21.15.119:6789/0 conn(0x7fe8f00ca210 legacy=0x7fe8f00b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:03.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.193+0000 7fe8eed9d700 1 -- 172.21.15.119:0/3521242475 >> v1:172.21.15.111:6789/0 conn(0x7fe8f00cd790 legacy=0x7fe8f00cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:03.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.193+0000 7fe8eed9d700 1 -- 172.21.15.119:0/3521242475 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f006ab20 con 0x7fe8f00b1d50 2023-03-29T10:57:03.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.193+0000 7fe8eed9d700 1 --2- 172.21.15.119:0/3521242475 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe8f00b1d50 0x7fe8f006b600 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009df0 tx=0x7fe8e800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7c39b64f7130801 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:03.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.194+0000 7fe8edd9b700 1 -- 172.21.15.119:0/3521242475 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e800e070 con 0x7fe8f00b1d50 2023-03-29T10:57:03.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.194+0000 7fe8edd9b700 1 -- 172.21.15.119:0/3521242475 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e800bad0 con 0x7fe8f00b1d50 2023-03-29T10:57:03.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.194+0000 7fe8edd9b700 1 -- 172.21.15.119:0/3521242475 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e800be60 con 0x7fe8f00b1d50 2023-03-29T10:57:03.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.194+0000 7fe8f4fff700 1 -- 172.21.15.119:0/3521242475 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe8f00b1d50 msgr2=0x7fe8f006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:03.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.194+0000 7fe8f4fff700 1 --2- 172.21.15.119:0/3521242475 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe8f00b1d50 0x7fe8f006b600 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009df0 tx=0x7fe8e800b330 comp rx=0 tx=0).stop 2023-03-29T10:57:03.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.194+0000 7fe8f4fff700 1 -- 172.21.15.119:0/3521242475 shutdown_connections 2023-03-29T10:57:03.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.194+0000 7fe8f4fff700 1 --2- 172.21.15.119:0/3521242475 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe8f00b1d50 0x7fe8f006b600 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.194+0000 7fe8f4fff700 1 -- 172.21.15.119:0/3521242475 >> 172.21.15.119:0/3521242475 conn(0x7fe8f006e980 msgr2=0x7fe8f00cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:03.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.194+0000 7fe8f4fff700 1 -- 172.21.15.119:0/3521242475 shutdown_connections 2023-03-29T10:57:03.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.194+0000 7fe8f4fff700 1 -- 172.21.15.119:0/3521242475 wait complete. 2023-03-29T10:57:03.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.196+0000 7fe8f4fff700 1 Processor -- start 2023-03-29T10:57:03.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.196+0000 7fe8f4fff700 1 -- start start 2023-03-29T10:57:03.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.197+0000 7fe8f4fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f00b1d50 0x7fe8f015aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:03.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.197+0000 7fe8f4fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8f00ca210 0x7fe8f00c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:03.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.197+0000 7fe8f4fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe8f00cd790 0x7fe8f00c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:03.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.197+0000 7fe8f4fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f014e6b0 con 0x7fe8f00ca210 2023-03-29T10:57:03.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.197+0000 7fe8f4fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f015b9c0 con 0x7fe8f00b1d50 2023-03-29T10:57:03.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.197+0000 7fe8f4fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f00c3b90 con 0x7fe8f00cd790 2023-03-29T10:57:03.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.197+0000 7fe8ee59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f00b1d50 0x7fe8f015aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:03.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.197+0000 7fe8ee59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f00b1d50 0x7fe8f015aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51776/0 (socket says 172.21.15.119:51776) 2023-03-29T10:57:03.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.197+0000 7fe8ee59c700 1 -- 172.21.15.119:0/1154385617 learned_addr learned my addr 172.21.15.119:0/1154385617 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-03-29T10:57:03.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.197+0000 7fe8eed9d700 1 --2- 172.21.15.119:0/1154385617 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe8f00cd790 0x7fe8f00c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:03.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.198+0000 7fe8e7fff700 1 --2- 172.21.15.119:0/1154385617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8f00ca210 0x7fe8f00c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:03.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.198+0000 7fe8ee59c700 1 -- 172.21.15.119:0/1154385617 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe8f00cd790 msgr2=0x7fe8f00c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:03.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.198+0000 7fe8ee59c700 1 --2- 172.21.15.119:0/1154385617 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe8f00cd790 0x7fe8f00c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.198+0000 7fe8ee59c700 1 -- 172.21.15.119:0/1154385617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8f00ca210 msgr2=0x7fe8f00c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:03.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.198+0000 7fe8ee59c700 1 --2- 172.21.15.119:0/1154385617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8f00ca210 0x7fe8f00c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.198+0000 7fe8ee59c700 1 -- 172.21.15.119:0/1154385617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e8009a70 con 0x7fe8f00b1d50 2023-03-29T10:57:03.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.198+0000 7fe8eed9d700 1 --2- 172.21.15.119:0/1154385617 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe8f00cd790 0x7fe8f00c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:03.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.198+0000 7fe8ee59c700 1 --2- 172.21.15.119:0/1154385617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f00b1d50 0x7fe8f015aa20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000c750 tx=0x7fe8e000ca60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:03.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.198+0000 7fe8e6ffd700 1 -- 172.21.15.119:0/1154385617 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00054e0 con 0x7fe8f00b1d50 2023-03-29T10:57:03.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.198+0000 7fe8e6ffd700 1 -- 172.21.15.119:0/1154385617 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e0003680 con 0x7fe8f00b1d50 2023-03-29T10:57:03.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.198+0000 7fe8e6ffd700 1 -- 172.21.15.119:0/1154385617 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0003a30 con 0x7fe8f00b1d50 2023-03-29T10:57:03.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.198+0000 7fe8f4fff700 1 -- 172.21.15.119:0/1154385617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f006b0e0 con 0x7fe8f00b1d50 2023-03-29T10:57:03.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.199+0000 7fe8f4fff700 1 -- 172.21.15.119:0/1154385617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f006ae00 con 0x7fe8f00b1d50 2023-03-29T10:57:03.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.199+0000 7fe8e6ffd700 1 -- 172.21.15.119:0/1154385617 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe8e001c440 con 0x7fe8f00b1d50 2023-03-29T10:57:03.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.199+0000 7fe8e6ffd700 1 -- 172.21.15.119:0/1154385617 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fe8e001cb10 con 0x7fe8f00b1d50 2023-03-29T10:57:03.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.200+0000 7fe8f4fff700 1 -- 172.21.15.119:0/1154385617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f015ca20 con 0x7fe8f00b1d50 2023-03-29T10:57:03.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.202+0000 7fe8e6ffd700 1 -- 172.21.15.119:0/1154385617 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fe8e0017070 con 0x7fe8f00b1d50 2023-03-29T10:57:03.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.289+0000 7fe8f4fff700 1 -- 172.21.15.119:0/1154385617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7943699f-e24a-4905-a599-f7139d3811ba", "id": 2} v 0) v1 -- 0x7fe8f0072f40 con 0x7fe8f00b1d50 2023-03-29T10:57:03.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.296+0000 7fe8e6ffd700 1 -- 172.21.15.119:0/1154385617 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7943699f-e24a-4905-a599-f7139d3811ba", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe8e001cda0 con 0x7fe8f00b1d50 2023-03-29T10:57:03.296 INFO:teuthology.orchestra.run.smithi119.stdout:2 2023-03-29T10:57:03.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.297+0000 7fe8f4fff700 1 -- 172.21.15.119:0/1154385617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f00b1d50 msgr2=0x7fe8f015aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:03.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.297+0000 7fe8f4fff700 1 --2- 172.21.15.119:0/1154385617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f00b1d50 0x7fe8f015aa20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000c750 tx=0x7fe8e000ca60 comp rx=0 tx=0).stop 2023-03-29T10:57:03.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.297+0000 7fe8f4fff700 1 -- 172.21.15.119:0/1154385617 shutdown_connections 2023-03-29T10:57:03.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.297+0000 7fe8f4fff700 1 --2- 172.21.15.119:0/1154385617 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe8f00cd790 0x7fe8f00c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.297+0000 7fe8f4fff700 1 --2- 172.21.15.119:0/1154385617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f00b1d50 0x7fe8f015aa20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.297+0000 7fe8f4fff700 1 --2- 172.21.15.119:0/1154385617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe8f00ca210 0x7fe8f00c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.298+0000 7fe8f4fff700 1 -- 172.21.15.119:0/1154385617 >> 172.21.15.119:0/1154385617 conn(0x7fe8f006e980 msgr2=0x7fe8f0160b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:03.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.298+0000 7fe8f4fff700 1 -- 172.21.15.119:0/1154385617 shutdown_connections 2023-03-29T10:57:03.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.298+0000 7fe8f4fff700 1 -- 172.21.15.119:0/1154385617 wait complete. 2023-03-29T10:57:03.313 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 73e253f3-86e1-4da0-b800-ce2b864835df 3 2023-03-29T10:57:03.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.493+0000 7f9268fff700 1 Processor -- start 2023-03-29T10:57:03.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.493+0000 7f9268fff700 1 -- start start 2023-03-29T10:57:03.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.494+0000 7f9268fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f926415a140 0x7f92640cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:03.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.494+0000 7f9268fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f92640cfff0 0x7f92640b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:03.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.494+0000 7f9268fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926415d660 con 0x7f9264160ba0 2023-03-29T10:57:03.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.494+0000 7f9268fff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926415cd70 con 0x7f926415a140 2023-03-29T10:57:03.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.494+0000 7f9268fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926415d010 con 0x7f92640cfff0 2023-03-29T10:57:03.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.494+0000 7f9262d9d700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f92640cfff0 0x7f92640b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:03.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.494+0000 7f9261d9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9264160ba0 0x7f92640cf8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59174/0 (socket says 172.21.15.119:59174) 2023-03-29T10:57:03.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.495+0000 7f9261d9b700 1 -- 172.21.15.119:0/4112020964 learned_addr learned my addr 172.21.15.119:0/4112020964 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-03-29T10:57:03.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.495+0000 7f926259c700 1 --2- 172.21.15.119:0/4112020964 >> v2:172.21.15.111:3300/0 conn(0x7f926415a140 0x7f92640cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:03.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.495+0000 7f9262d9d700 1 -- 172.21.15.119:0/4112020964 >> v2:172.21.15.111:3300/0 conn(0x7f926415a140 msgr2=0x7f92640cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:03.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.495+0000 7f9262d9d700 1 --2- 172.21.15.119:0/4112020964 >> v2:172.21.15.111:3300/0 conn(0x7f926415a140 0x7f92640cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.495+0000 7f9262d9d700 1 -- 172.21.15.119:0/4112020964 >> v1:172.21.15.119:6789/0 conn(0x7f9264160ba0 legacy=0x7f92640cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:03.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.495+0000 7f9262d9d700 1 -- 172.21.15.119:0/4112020964 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926415d2f0 con 0x7f92640cfff0 2023-03-29T10:57:03.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.495+0000 7f926159a700 1 -- 172.21.15.119:0/4112020964 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216252920 0 0) 0x7f926415d660 con 0x7f9264160ba0 2023-03-29T10:57:03.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.495+0000 7f9262d9d700 1 --2- 172.21.15.119:0/4112020964 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f92640cfff0 0x7f92640b1b70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f925c009d90 tx=0x7f925c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f26df2b3d6fe0ed7 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:03.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.495+0000 7f926159a700 1 -- 172.21.15.119:0/4112020964 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c00e070 con 0x7f92640cfff0 2023-03-29T10:57:03.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.495+0000 7f926159a700 1 -- 172.21.15.119:0/4112020964 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925c00bb20 con 0x7f92640cfff0 2023-03-29T10:57:03.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.495+0000 7f926159a700 1 -- 172.21.15.119:0/4112020964 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c012430 con 0x7f92640cfff0 2023-03-29T10:57:03.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.496+0000 7f9268fff700 1 -- 172.21.15.119:0/4112020964 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f92640cfff0 msgr2=0x7f92640b1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:03.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.496+0000 7f9268fff700 1 --2- 172.21.15.119:0/4112020964 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f92640cfff0 0x7f92640b1b70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f925c009d90 tx=0x7f925c00b330 comp rx=0 tx=0).stop 2023-03-29T10:57:03.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.496+0000 7f9268fff700 1 -- 172.21.15.119:0/4112020964 shutdown_connections 2023-03-29T10:57:03.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.496+0000 7f9268fff700 1 --2- 172.21.15.119:0/4112020964 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f92640cfff0 0x7f92640b1b70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.496+0000 7f9268fff700 1 --2- 172.21.15.119:0/4112020964 >> v2:172.21.15.111:3300/0 conn(0x7f926415a140 0x7f92640cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.496+0000 7f9268fff700 1 -- 172.21.15.119:0/4112020964 >> 172.21.15.119:0/4112020964 conn(0x7f9264071b10 msgr2=0x7f9264071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:03.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.496+0000 7f9268fff700 1 -- 172.21.15.119:0/4112020964 shutdown_connections 2023-03-29T10:57:03.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.496+0000 7f9268fff700 1 -- 172.21.15.119:0/4112020964 wait complete. 2023-03-29T10:57:03.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.498+0000 7f9268fff700 1 Processor -- start 2023-03-29T10:57:03.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.498+0000 7f9268fff700 1 -- start start 2023-03-29T10:57:03.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.499+0000 7f9268fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92640cfff0 0x7f92640b6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:03.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.499+0000 7f9268fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926415a140 0x7f926414fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:03.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.499+0000 7f9268fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9264160ba0 0x7f926414ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:03.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.499+0000 7f9268fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9264151640 con 0x7f92640cfff0 2023-03-29T10:57:03.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.499+0000 7f9268fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92641f2230 con 0x7f926415a140 2023-03-29T10:57:03.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.499+0000 7f9268fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92641505a0 con 0x7f9264160ba0 2023-03-29T10:57:03.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.499+0000 7f9262d9d700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9264160ba0 0x7f926414ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:03.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.499+0000 7f926259c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92640cfff0 0x7f92640b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:03.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.499+0000 7f926259c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92640cfff0 0x7f92640b6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.119:56712/0 (socket says 172.21.15.119:56712) 2023-03-29T10:57:03.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.499+0000 7f9261d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926415a140 0x7f926414fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:03.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.499+0000 7f926259c700 1 -- 172.21.15.119:0/759199559 learned_addr learned my addr 172.21.15.119:0/759199559 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-03-29T10:57:03.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.499+0000 7f926259c700 1 -- 172.21.15.119:0/759199559 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9264160ba0 msgr2=0x7f926414ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:03.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.499+0000 7f926259c700 1 --2- 172.21.15.119:0/759199559 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9264160ba0 0x7f926414ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.499+0000 7f926259c700 1 -- 172.21.15.119:0/759199559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926415a140 msgr2=0x7f926414fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:03.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.499+0000 7f926259c700 1 --2- 172.21.15.119:0/759199559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926415a140 0x7f926414fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.499+0000 7f926259c700 1 -- 172.21.15.119:0/759199559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925c009a70 con 0x7f92640cfff0 2023-03-29T10:57:03.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.500+0000 7f926259c700 1 --2- 172.21.15.119:0/759199559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92640cfff0 0x7f92640b6960 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f925400e9b0 tx=0x7f925400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:03.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.500+0000 7f9252ffd700 1 -- 172.21.15.119:0/759199559 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925400ee50 con 0x7f92640cfff0 2023-03-29T10:57:03.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.500+0000 7f9252ffd700 1 -- 172.21.15.119:0/759199559 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925400c850 con 0x7f92640cfff0 2023-03-29T10:57:03.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.500+0000 7f9268fff700 1 -- 172.21.15.119:0/759199559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92640cdb00 con 0x7f92640cfff0 2023-03-29T10:57:03.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.500+0000 7f9252ffd700 1 -- 172.21.15.119:0/759199559 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925400cbc0 con 0x7f92640cfff0 2023-03-29T10:57:03.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.500+0000 7f9268fff700 1 -- 172.21.15.119:0/759199559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926415daf0 con 0x7f92640cfff0 2023-03-29T10:57:03.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.500+0000 7f9252ffd700 1 -- 172.21.15.119:0/759199559 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9254005d20 con 0x7f92640cfff0 2023-03-29T10:57:03.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.500+0000 7f9252ffd700 1 -- 172.21.15.119:0/759199559 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f9254006700 con 0x7f92640cfff0 2023-03-29T10:57:03.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.501+0000 7f9268fff700 1 -- 172.21.15.119:0/759199559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9264161020 con 0x7f92640cfff0 2023-03-29T10:57:03.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.504+0000 7f9252ffd700 1 -- 172.21.15.119:0/759199559 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f9254021060 con 0x7f92640cfff0 2023-03-29T10:57:03.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.590+0000 7f9268fff700 1 -- 172.21.15.119:0/759199559 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd new", "uuid": "73e253f3-86e1-4da0-b800-ce2b864835df", "id": 3} v 0) v1 -- 0x7f92640b51f0 con 0x7f92640cfff0 2023-03-29T10:57:03.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.597+0000 7f9252ffd700 1 -- 172.21.15.119:0/759199559 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "73e253f3-86e1-4da0-b800-ce2b864835df", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9254017070 con 0x7f92640cfff0 2023-03-29T10:57:03.597 INFO:teuthology.orchestra.run.smithi119.stdout:3 2023-03-29T10:57:03.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.598+0000 7f9268fff700 1 -- 172.21.15.119:0/759199559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92640cfff0 msgr2=0x7f92640b6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:03.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.598+0000 7f9268fff700 1 --2- 172.21.15.119:0/759199559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92640cfff0 0x7f92640b6960 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f925400e9b0 tx=0x7f925400ecc0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.598+0000 7f9268fff700 1 -- 172.21.15.119:0/759199559 shutdown_connections 2023-03-29T10:57:03.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.598+0000 7f9268fff700 1 --2- 172.21.15.119:0/759199559 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f92640cfff0 0x7f92640b6960 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.598+0000 7f9268fff700 1 --2- 172.21.15.119:0/759199559 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9264160ba0 0x7f926414ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.598+0000 7f9268fff700 1 --2- 172.21.15.119:0/759199559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926415a140 0x7f926414fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.598+0000 7f9268fff700 1 -- 172.21.15.119:0/759199559 >> 172.21.15.119:0/759199559 conn(0x7f9264071b10 msgr2=0x7f926415be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:03.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.598+0000 7f9268fff700 1 -- 172.21.15.119:0/759199559 shutdown_connections 2023-03-29T10:57:03.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.599+0000 7f9268fff700 1 -- 172.21.15.119:0/759199559 wait complete. 2023-03-29T10:57:03.616 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 095d437b-3816-4afb-86a9-6d78e19880c7 4 2023-03-29T10:57:03.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.817+0000 7f978919d700 1 Processor -- start 2023-03-29T10:57:03.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.817+0000 7f978919d700 1 -- start start 2023-03-29T10:57:03.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.819+0000 7f978919d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f97841617f0 0x7f97840ce120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:03.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.819+0000 7f978919d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f97840710e0 0x7f9784071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:03.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.819+0000 7f978919d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97840cc440 con 0x7f978415e2f0 2023-03-29T10:57:03.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.819+0000 7f978919d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97840ce660 con 0x7f97840710e0 2023-03-29T10:57:03.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.819+0000 7f978919d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97840adcf0 con 0x7f97841617f0 2023-03-29T10:57:03.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.820+0000 7f9783fff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f978415e2f0 0x7f978406ca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.119:51356/0 (socket says 172.21.15.119:51356) 2023-03-29T10:57:03.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.820+0000 7f9783fff700 1 -- 172.21.15.119:0/1748206394 learned_addr learned my addr 172.21.15.119:0/1748206394 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-03-29T10:57:03.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.820+0000 7f97837fe700 1 --2- 172.21.15.119:0/1748206394 >> v2:172.21.15.119:3300/0 conn(0x7f97841617f0 0x7f97840ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:03.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.820+0000 7f978899c700 1 --2- 172.21.15.119:0/1748206394 >> v2:172.21.15.111:3300/0 conn(0x7f97840710e0 0x7f9784071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:03.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.820+0000 7f9782ffd700 1 -- 172.21.15.119:0/1748206394 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171074965 0 0) 0x7f97840cc440 con 0x7f978415e2f0 2023-03-29T10:57:03.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.820+0000 7f9782ffd700 1 -- 172.21.15.119:0/1748206394 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97700035b0 con 0x7f978415e2f0 2023-03-29T10:57:03.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.820+0000 7f978899c700 1 -- 172.21.15.119:0/1748206394 >> v2:172.21.15.119:3300/0 conn(0x7f97841617f0 msgr2=0x7f97840ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:03.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.820+0000 7f9782ffd700 1 -- 172.21.15.119:0/1748206394 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 233734127 0 0) 0x7f97700035b0 con 0x7f978415e2f0 2023-03-29T10:57:03.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.820+0000 7f978899c700 1 --2- 172.21.15.119:0/1748206394 >> v2:172.21.15.119:3300/0 conn(0x7f97841617f0 0x7f97840ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.821+0000 7f978899c700 1 -- 172.21.15.119:0/1748206394 >> v1:172.21.15.111:6789/0 conn(0x7f978415e2f0 legacy=0x7f978406ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:03.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.821+0000 7f978899c700 1 -- 172.21.15.119:0/1748206394 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f978406e0e0 con 0x7f97840710e0 2023-03-29T10:57:03.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.821+0000 7f97837fe700 1 --2- 172.21.15.119:0/1748206394 >> v2:172.21.15.119:3300/0 conn(0x7f97841617f0 0x7f97840ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:03.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.821+0000 7f978899c700 1 --2- 172.21.15.119:0/1748206394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97840710e0 0x7f9784071500 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f97840cf950 tx=0x7f9774009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2fed2b80399ac1 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:03.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.821+0000 7f9782ffd700 1 -- 172.21.15.119:0/1748206394 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c001670 con 0x7f97840710e0 2023-03-29T10:57:03.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.821+0000 7f9782ffd700 1 -- 172.21.15.119:0/1748206394 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9774004030 con 0x7f97840710e0 2023-03-29T10:57:03.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.821+0000 7f9782ffd700 1 -- 172.21.15.119:0/1748206394 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97740043a0 con 0x7f97840710e0 2023-03-29T10:57:03.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.822+0000 7f978919d700 1 -- 172.21.15.119:0/1748206394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97840710e0 msgr2=0x7f9784071500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:03.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.822+0000 7f978919d700 1 --2- 172.21.15.119:0/1748206394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97840710e0 0x7f9784071500 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f97840cf950 tx=0x7f9774009b20 comp rx=0 tx=0).stop 2023-03-29T10:57:03.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.822+0000 7f978919d700 1 -- 172.21.15.119:0/1748206394 shutdown_connections 2023-03-29T10:57:03.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.822+0000 7f978919d700 1 --2- 172.21.15.119:0/1748206394 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97840710e0 0x7f9784071500 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.822+0000 7f978919d700 1 --2- 172.21.15.119:0/1748206394 >> v2:172.21.15.119:3300/0 conn(0x7f97841617f0 0x7f97840ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.822+0000 7f978919d700 1 -- 172.21.15.119:0/1748206394 >> 172.21.15.119:0/1748206394 conn(0x7f978406a3e0 msgr2=0x7f978406a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:03.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.822+0000 7f978919d700 1 -- 172.21.15.119:0/1748206394 shutdown_connections 2023-03-29T10:57:03.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.822+0000 7f978919d700 1 -- 172.21.15.119:0/1748206394 wait complete. 2023-03-29T10:57:03.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.825+0000 7f978919d700 1 Processor -- start 2023-03-29T10:57:03.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.825+0000 7f978919d700 1 -- start start 2023-03-29T10:57:03.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.827+0000 7f978919d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97840710e0 0x7f978413b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:03.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.827+0000 7f978919d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f978415e2f0 0x7f978413a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:03.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.827+0000 7f978919d700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97841617f0 0x7f97840b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:03.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.827+0000 7f978919d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97840b3300 con 0x7f97840710e0 2023-03-29T10:57:03.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.827+0000 7f978919d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97840b2a40 con 0x7f978415e2f0 2023-03-29T10:57:03.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.827+0000 7f978919d700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97840b2cb0 con 0x7f97841617f0 2023-03-29T10:57:03.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.827+0000 7f9783fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97840710e0 0x7f978413b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:03.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.827+0000 7f9783fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97840710e0 0x7f978413b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.119:56738/0 (socket says 172.21.15.119:56738) 2023-03-29T10:57:03.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.827+0000 7f9783fff700 1 -- 172.21.15.119:0/2293845980 learned_addr learned my addr 172.21.15.119:0/2293845980 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-03-29T10:57:03.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.827+0000 7f978899c700 1 --2- 172.21.15.119:0/2293845980 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97841617f0 0x7f97840b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:03.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.827+0000 7f9783fff700 1 -- 172.21.15.119:0/2293845980 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97841617f0 msgr2=0x7f97840b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:03.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.827+0000 7f9783fff700 1 --2- 172.21.15.119:0/2293845980 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97841617f0 0x7f97840b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.827+0000 7f9783fff700 1 -- 172.21.15.119:0/2293845980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f978415e2f0 msgr2=0x7f978413a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:03.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.827+0000 7f9783fff700 1 --2- 172.21.15.119:0/2293845980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f978415e2f0 0x7f978413a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.827+0000 7f9783fff700 1 -- 172.21.15.119:0/2293845980 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976c009a70 con 0x7f97840710e0 2023-03-29T10:57:03.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.828+0000 7f9783fff700 1 --2- 172.21.15.119:0/2293845980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97840710e0 0x7f978413b820 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f977c00c700 tx=0x7f977c00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:03.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.828+0000 7f9780ff9700 1 -- 172.21.15.119:0/2293845980 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c004c80 con 0x7f97840710e0 2023-03-29T10:57:03.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.828+0000 7f9780ff9700 1 -- 172.21.15.119:0/2293845980 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977c004de0 con 0x7f97840710e0 2023-03-29T10:57:03.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.828+0000 7f9780ff9700 1 -- 172.21.15.119:0/2293845980 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c0072f0 con 0x7f97840710e0 2023-03-29T10:57:03.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.828+0000 7f978919d700 1 -- 172.21.15.119:0/2293845980 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97740094a0 con 0x7f97840710e0 2023-03-29T10:57:03.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.828+0000 7f978919d700 1 -- 172.21.15.119:0/2293845980 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97840cc690 con 0x7f97840710e0 2023-03-29T10:57:03.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.828+0000 7f9780ff9700 1 -- 172.21.15.119:0/2293845980 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f977c01d3f0 con 0x7f97840710e0 2023-03-29T10:57:03.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.829+0000 7f9780ff9700 1 -- 172.21.15.119:0/2293845980 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f977c01dbb0 con 0x7f97840710e0 2023-03-29T10:57:03.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.830+0000 7f978919d700 1 -- 172.21.15.119:0/2293845980 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f978415e6c0 con 0x7f97840710e0 2023-03-29T10:57:03.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.832+0000 7f9780ff9700 1 -- 172.21.15.119:0/2293845980 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f977c01d6a0 con 0x7f97840710e0 2023-03-29T10:57:03.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.923+0000 7f978919d700 1 -- 172.21.15.119:0/2293845980 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd new", "uuid": "095d437b-3816-4afb-86a9-6d78e19880c7", "id": 4} v 0) v1 -- 0x7f97840cc970 con 0x7f97840710e0 2023-03-29T10:57:03.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.931+0000 7f9780ff9700 1 -- 172.21.15.119:0/2293845980 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "095d437b-3816-4afb-86a9-6d78e19880c7", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f977c01d910 con 0x7f97840710e0 2023-03-29T10:57:03.931 INFO:teuthology.orchestra.run.smithi119.stdout:4 2023-03-29T10:57:03.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.932+0000 7f978919d700 1 -- 172.21.15.119:0/2293845980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97840710e0 msgr2=0x7f978413b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:03.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.932+0000 7f978919d700 1 --2- 172.21.15.119:0/2293845980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97840710e0 0x7f978413b820 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f977c00c700 tx=0x7f977c00cac0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.932+0000 7f978919d700 1 -- 172.21.15.119:0/2293845980 shutdown_connections 2023-03-29T10:57:03.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.932+0000 7f978919d700 1 --2- 172.21.15.119:0/2293845980 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97840710e0 0x7f978413b820 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.932+0000 7f978919d700 1 --2- 172.21.15.119:0/2293845980 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97841617f0 0x7f97840b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.932+0000 7f978919d700 1 --2- 172.21.15.119:0/2293845980 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f978415e2f0 0x7f978413a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:03.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.932+0000 7f978919d700 1 -- 172.21.15.119:0/2293845980 >> 172.21.15.119:0/2293845980 conn(0x7f978406a3e0 msgr2=0x7f9784163750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:03.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.932+0000 7f978919d700 1 -- 172.21.15.119:0/2293845980 shutdown_connections 2023-03-29T10:57:03.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:03.932+0000 7f978919d700 1 -- 172.21.15.119:0/2293845980 wait complete. 2023-03-29T10:57:03.937 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:03.937+0000 7fe1bee6a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-03-29T10:57:03.948 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 3e7fe27b-3125-4d00-be7b-c7a93679bfaa 5 2023-03-29T10:57:03.984 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:03.983+0000 7fa172b1e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-03-29T10:57:04.107 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:04.106+0000 7fe1bee6a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-03-29T10:57:04.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.129+0000 7fba10bff700 1 Processor -- start 2023-03-29T10:57:04.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.129+0000 7fba10bff700 1 -- start start 2023-03-29T10:57:04.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.131+0000 7fba10bff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fba0c0cfbf0 0x7fba0c1608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:04.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.131+0000 7fba10bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fba0c0d1100 0x7fba0c0d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:04.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.131+0000 7fba10bff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba0c0d0020 con 0x7fba0c0b87a0 2023-03-29T10:57:04.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.131+0000 7fba10bff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0c0d02d0 con 0x7fba0c0cfbf0 2023-03-29T10:57:04.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.131+0000 7fba10bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c0d0cb0 con 0x7fba0c0d1100 2023-03-29T10:57:04.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.131+0000 7fba0ad9d700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fba0c0d1100 0x7fba0c0d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:04.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.131+0000 7fba09d9b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fba0c0cfbf0 0x7fba0c1608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:04.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.131+0000 7fba0ad9d700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fba0c0d1100 0x7fba0c0d1560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.119:35412/0 (socket says 172.21.15.119:35412) 2023-03-29T10:57:04.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.131+0000 7fba0ad9d700 1 -- 172.21.15.119:0/3797148021 learned_addr learned my addr 172.21.15.119:0/3797148021 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-03-29T10:57:04.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.131+0000 7fba0ad9d700 1 -- 172.21.15.119:0/3797148021 >> v2:172.21.15.111:3300/0 conn(0x7fba0c0cfbf0 msgr2=0x7fba0c1608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:04.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.131+0000 7fba0ad9d700 1 --2- 172.21.15.119:0/3797148021 >> v2:172.21.15.111:3300/0 conn(0x7fba0c0cfbf0 0x7fba0c1608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.131+0000 7fba0ad9d700 1 -- 172.21.15.119:0/3797148021 >> v1:172.21.15.111:6789/0 conn(0x7fba0c0b87a0 legacy=0x7fba0c1601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:04.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.131+0000 7fba0ad9d700 1 -- 172.21.15.119:0/3797148021 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0c0cdc60 con 0x7fba0c0d1100 2023-03-29T10:57:04.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.132+0000 7fba0ad9d700 1 --2- 172.21.15.119:0/3797148021 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fba0c0d1100 0x7fba0c0d1560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fba04009e50 tx=0x7fba0400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9341b1b5aeefa18e server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:04.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.132+0000 7fba0959a700 1 -- 172.21.15.119:0/3797148021 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0400e070 con 0x7fba0c0d1100 2023-03-29T10:57:04.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.132+0000 7fba0959a700 1 -- 172.21.15.119:0/3797148021 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0400b910 con 0x7fba0c0d1100 2023-03-29T10:57:04.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.132+0000 7fba0959a700 1 -- 172.21.15.119:0/3797148021 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0400bcc0 con 0x7fba0c0d1100 2023-03-29T10:57:04.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.132+0000 7fba10bff700 1 -- 172.21.15.119:0/3797148021 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fba0c0d1100 msgr2=0x7fba0c0d1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:04.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.132+0000 7fba10bff700 1 --2- 172.21.15.119:0/3797148021 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fba0c0d1100 0x7fba0c0d1560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fba04009e50 tx=0x7fba0400b330 comp rx=0 tx=0).stop 2023-03-29T10:57:04.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.132+0000 7fba10bff700 1 -- 172.21.15.119:0/3797148021 shutdown_connections 2023-03-29T10:57:04.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.132+0000 7fba10bff700 1 --2- 172.21.15.119:0/3797148021 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fba0c0d1100 0x7fba0c0d1560 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.132+0000 7fba10bff700 1 --2- 172.21.15.119:0/3797148021 >> v2:172.21.15.111:3300/0 conn(0x7fba0c0cfbf0 0x7fba0c1608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.132+0000 7fba10bff700 1 -- 172.21.15.119:0/3797148021 >> 172.21.15.119:0/3797148021 conn(0x7fba0c071b10 msgr2=0x7fba0c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:04.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.133+0000 7fba10bff700 1 -- 172.21.15.119:0/3797148021 shutdown_connections 2023-03-29T10:57:04.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.133+0000 7fba10bff700 1 -- 172.21.15.119:0/3797148021 wait complete. 2023-03-29T10:57:04.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.134+0000 7fba10bff700 1 Processor -- start 2023-03-29T10:57:04.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.134+0000 7fba10bff700 1 -- start start 2023-03-29T10:57:04.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.135+0000 7fba10bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba0c0b87a0 0x7fba0c14a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:04.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.135+0000 7fba10bff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba0c0cfbf0 0x7fba0c15e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:04.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.135+0000 7fba10bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fba0c0d1100 0x7fba0c15ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:04.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.135+0000 7fba10bff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c0ca140 con 0x7fba0c0cfbf0 2023-03-29T10:57:04.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.135+0000 7fba10bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c15fbe0 con 0x7fba0c0b87a0 2023-03-29T10:57:04.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.135+0000 7fba10bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c14a6d0 con 0x7fba0c0d1100 2023-03-29T10:57:04.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.135+0000 7fba0ad9d700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fba0c0d1100 0x7fba0c15ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:04.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.135+0000 7fba0a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba0c0b87a0 0x7fba0c14a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:04.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.136+0000 7fba0ad9d700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fba0c0d1100 0x7fba0c15ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.119:35420/0 (socket says 172.21.15.119:35420) 2023-03-29T10:57:04.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.136+0000 7fba0ad9d700 1 -- 172.21.15.119:0/540013304 learned_addr learned my addr 172.21.15.119:0/540013304 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-03-29T10:57:04.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.136+0000 7fba09d9b700 1 --2- 172.21.15.119:0/540013304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba0c0cfbf0 0x7fba0c15e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:04.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.136+0000 7fba0ad9d700 1 -- 172.21.15.119:0/540013304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba0c0b87a0 msgr2=0x7fba0c14a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:04.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.136+0000 7fba0ad9d700 1 --2- 172.21.15.119:0/540013304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba0c0b87a0 0x7fba0c14a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.136+0000 7fba0ad9d700 1 -- 172.21.15.119:0/540013304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba0c0cfbf0 msgr2=0x7fba0c15e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.136+0000 7fba0ad9d700 1 --2- 172.21.15.119:0/540013304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba0c0cfbf0 0x7fba0c15e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.136+0000 7fba0ad9d700 1 -- 172.21.15.119:0/540013304 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba04009b30 con 0x7fba0c0d1100 2023-03-29T10:57:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.136+0000 7fba0ad9d700 1 --2- 172.21.15.119:0/540013304 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fba0c0d1100 0x7fba0c15ed10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fba0c08be30 tx=0x7fba04004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.136+0000 7fb9faffd700 1 -- 172.21.15.119:0/540013304 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04004800 con 0x7fba0c0d1100 2023-03-29T10:57:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.136+0000 7fb9faffd700 1 -- 172.21.15.119:0/540013304 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba04004960 con 0x7fba0c0d1100 2023-03-29T10:57:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.136+0000 7fba10bff700 1 -- 172.21.15.119:0/540013304 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba0c0cd980 con 0x7fba0c0d1100 2023-03-29T10:57:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.137+0000 7fb9faffd700 1 -- 172.21.15.119:0/540013304 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0401b670 con 0x7fba0c0d1100 2023-03-29T10:57:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.137+0000 7fba10bff700 1 -- 172.21.15.119:0/540013304 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba0c0cd6a0 con 0x7fba0c0d1100 2023-03-29T10:57:04.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.137+0000 7fb9faffd700 1 -- 172.21.15.119:0/540013304 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fba0401bad0 con 0x7fba0c0d1100 2023-03-29T10:57:04.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.137+0000 7fb9faffd700 1 -- 172.21.15.119:0/540013304 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fba0401bd80 con 0x7fba0c0d1100 2023-03-29T10:57:04.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.138+0000 7fba10bff700 1 -- 172.21.15.119:0/540013304 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba0c160e30 con 0x7fba0c0d1100 2023-03-29T10:57:04.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.140+0000 7fb9faffd700 1 -- 172.21.15.119:0/540013304 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fba04043b10 con 0x7fba0c0d1100 2023-03-29T10:57:04.164 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:04.163+0000 7fa172b1e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-03-29T10:57:04.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.226+0000 7fba10bff700 1 -- 172.21.15.119:0/540013304 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3e7fe27b-3125-4d00-be7b-c7a93679bfaa", "id": 5} v 0) v1 -- 0x7fba0c0723f0 con 0x7fba0c0d1100 2023-03-29T10:57:04.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.234+0000 7fb9faffd700 1 -- 172.21.15.119:0/540013304 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3e7fe27b-3125-4d00-be7b-c7a93679bfaa", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fba04032020 con 0x7fba0c0d1100 2023-03-29T10:57:04.235 INFO:teuthology.orchestra.run.smithi119.stdout:5 2023-03-29T10:57:04.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.235+0000 7fba10bff700 1 -- 172.21.15.119:0/540013304 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fba0c0d1100 msgr2=0x7fba0c15ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:04.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.235+0000 7fba10bff700 1 --2- 172.21.15.119:0/540013304 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fba0c0d1100 0x7fba0c15ed10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fba0c08be30 tx=0x7fba04004ad0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.235+0000 7fba10bff700 1 -- 172.21.15.119:0/540013304 shutdown_connections 2023-03-29T10:57:04.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.235+0000 7fba10bff700 1 --2- 172.21.15.119:0/540013304 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fba0c0d1100 0x7fba0c15ed10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.235+0000 7fba10bff700 1 --2- 172.21.15.119:0/540013304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba0c0b87a0 0x7fba0c14a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.235+0000 7fba10bff700 1 --2- 172.21.15.119:0/540013304 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fba0c0cfbf0 0x7fba0c15e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.235+0000 7fba10bff700 1 -- 172.21.15.119:0/540013304 >> 172.21.15.119:0/540013304 conn(0x7fba0c071b10 msgr2=0x7fba0c15c400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:04.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.235+0000 7fba10bff700 1 -- 172.21.15.119:0/540013304 shutdown_connections 2023-03-29T10:57:04.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.235+0000 7fba10bff700 1 -- 172.21.15.119:0/540013304 wait complete. 2023-03-29T10:57:04.252 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new fb37ec18-5612-46aa-ae6b-682bc8fc784f 6 2023-03-29T10:57:04.364 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:04.363+0000 7fe1bee6a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-03-29T10:57:04.433 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:04.432+0000 7fa172b1e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-03-29T10:57:04.454 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:04.453+0000 7fe1bee6a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-03-29T10:57:04.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.472+0000 7efd0a9ff700 1 Processor -- start 2023-03-29T10:57:04.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.472+0000 7efd0a9ff700 1 -- start start 2023-03-29T10:57:04.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.474+0000 7efd0a9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efd040b87a0 0x7efd040cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:04.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.474+0000 7efd0a9ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd041601e0 0x7efd04160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:04.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.474+0000 7efd0a9ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd04160ff0 con 0x7efd04160b70 2023-03-29T10:57:04.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.474+0000 7efd0a9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd040d02d0 con 0x7efd040b87a0 2023-03-29T10:57:04.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.474+0000 7efd0a9ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd040d0cb0 con 0x7efd041601e0 2023-03-29T10:57:04.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.474+0000 7efd037fe700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd041601e0 0x7efd04160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:04.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.474+0000 7efd03fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efd040b87a0 0x7efd040cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:04.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.474+0000 7efd037fe700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd041601e0 0x7efd04160630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.119:35430/0 (socket says 172.21.15.119:35430) 2023-03-29T10:57:04.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.474+0000 7efd03fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efd040b87a0 0x7efd040cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51820/0 (socket says 172.21.15.119:51820) 2023-03-29T10:57:04.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.474+0000 7efd037fe700 1 -- 172.21.15.119:0/1906558046 learned_addr learned my addr 172.21.15.119:0/1906558046 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-03-29T10:57:04.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.474+0000 7efd03fff700 1 -- 172.21.15.119:0/1906558046 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd041601e0 msgr2=0x7efd04160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:04.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.474+0000 7efd03fff700 1 --2- 172.21.15.119:0/1906558046 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd041601e0 0x7efd04160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.475+0000 7efd03fff700 1 -- 172.21.15.119:0/1906558046 >> v1:172.21.15.119:6789/0 conn(0x7efd04160b70 legacy=0x7efd040d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:04.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.475+0000 7efd03fff700 1 -- 172.21.15.119:0/1906558046 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd040cdc60 con 0x7efd040b87a0 2023-03-29T10:57:04.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.475+0000 7efd037fe700 1 --2- 172.21.15.119:0/1906558046 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd041601e0 0x7efd04160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:04.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.475+0000 7efd03fff700 1 --2- 172.21.15.119:0/1906558046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd040b87a0 0x7efd040cfbf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009840 tx=0x7efcf4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7af5761aa665bb2d server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:04.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.475+0000 7efd02ffd700 1 -- 172.21.15.119:0/1906558046 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4011830 con 0x7efd040b87a0 2023-03-29T10:57:04.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.475+0000 7efd02ffd700 1 -- 172.21.15.119:0/1906558046 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf4011990 con 0x7efd040b87a0 2023-03-29T10:57:04.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.475+0000 7efd02ffd700 1 -- 172.21.15.119:0/1906558046 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4011d20 con 0x7efd040b87a0 2023-03-29T10:57:04.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.476+0000 7efd0a9ff700 1 -- 172.21.15.119:0/1906558046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd040b87a0 msgr2=0x7efd040cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:04.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.476+0000 7efd0a9ff700 1 --2- 172.21.15.119:0/1906558046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd040b87a0 0x7efd040cfbf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009840 tx=0x7efcf4009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:04.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.476+0000 7efd0a9ff700 1 -- 172.21.15.119:0/1906558046 shutdown_connections 2023-03-29T10:57:04.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.476+0000 7efd0a9ff700 1 --2- 172.21.15.119:0/1906558046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd040b87a0 0x7efd040cfbf0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.476+0000 7efd0a9ff700 1 --2- 172.21.15.119:0/1906558046 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd041601e0 0x7efd04160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.476+0000 7efd0a9ff700 1 -- 172.21.15.119:0/1906558046 >> 172.21.15.119:0/1906558046 conn(0x7efd04071b10 msgr2=0x7efd04071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:04.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.476+0000 7efd0a9ff700 1 -- 172.21.15.119:0/1906558046 shutdown_connections 2023-03-29T10:57:04.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.476+0000 7efd0a9ff700 1 -- 172.21.15.119:0/1906558046 wait complete. 2023-03-29T10:57:04.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.477+0000 7efd0a9ff700 1 Processor -- start 2023-03-29T10:57:04.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.478+0000 7efd0a9ff700 1 -- start start 2023-03-29T10:57:04.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.478+0000 7efd0a9ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd040b87a0 0x7efd040c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:04.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.478+0000 7efd0a9ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd041601e0 0x7efd040b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:04.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.478+0000 7efd0a9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd04160b70 0x7efd040b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:04.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.478+0000 7efd0a9ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd040c7c20 con 0x7efd040b87a0 2023-03-29T10:57:04.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.478+0000 7efd0a9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd040c7660 con 0x7efd04160b70 2023-03-29T10:57:04.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.478+0000 7efd0a9ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd040c7940 con 0x7efd041601e0 2023-03-29T10:57:04.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.479+0000 7efd037fe700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd041601e0 0x7efd040b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:04.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.479+0000 7efd037fe700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd041601e0 0x7efd040b3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.119:35442/0 (socket says 172.21.15.119:35442) 2023-03-29T10:57:04.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.479+0000 7efd037fe700 1 -- 172.21.15.119:0/2930342405 learned_addr learned my addr 172.21.15.119:0/2930342405 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-03-29T10:57:04.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.479+0000 7efd08f9c700 1 --2- 172.21.15.119:0/2930342405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd04160b70 0x7efd040b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:04.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.479+0000 7efd03fff700 1 --2- 172.21.15.119:0/2930342405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd040b87a0 0x7efd040c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:04.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.479+0000 7efd037fe700 1 -- 172.21.15.119:0/2930342405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd04160b70 msgr2=0x7efd040b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:04.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.479+0000 7efd037fe700 1 --2- 172.21.15.119:0/2930342405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd04160b70 0x7efd040b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.479+0000 7efd037fe700 1 -- 172.21.15.119:0/2930342405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd040b87a0 msgr2=0x7efd040c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:04.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.479+0000 7efd037fe700 1 --2- 172.21.15.119:0/2930342405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd040b87a0 0x7efd040c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.479+0000 7efd037fe700 1 -- 172.21.15.119:0/2930342405 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf8009550 con 0x7efd041601e0 2023-03-29T10:57:04.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.479+0000 7efd037fe700 1 --2- 172.21.15.119:0/2930342405 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd041601e0 0x7efd040b3960 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efcf800ebe0 tx=0x7efcf800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:04.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.480+0000 7efd00ff9700 1 -- 172.21.15.119:0/2930342405 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf800c850 con 0x7efd041601e0 2023-03-29T10:57:04.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.480+0000 7efd00ff9700 1 -- 172.21.15.119:0/2930342405 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf800c9b0 con 0x7efd041601e0 2023-03-29T10:57:04.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.480+0000 7efd00ff9700 1 -- 172.21.15.119:0/2930342405 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf800cd40 con 0x7efd041601e0 2023-03-29T10:57:04.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.480+0000 7efd0a9ff700 1 -- 172.21.15.119:0/2930342405 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf40094a0 con 0x7efd041601e0 2023-03-29T10:57:04.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.480+0000 7efd0a9ff700 1 -- 172.21.15.119:0/2930342405 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd040cd980 con 0x7efd041601e0 2023-03-29T10:57:04.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.480+0000 7efd00ff9700 1 -- 172.21.15.119:0/2930342405 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7efcf8013770 con 0x7efd041601e0 2023-03-29T10:57:04.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.480+0000 7efd00ff9700 1 -- 172.21.15.119:0/2930342405 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7efcf801cdf0 con 0x7efd041601e0 2023-03-29T10:57:04.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.481+0000 7efd0a9ff700 1 -- 172.21.15.119:0/2930342405 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd040d2110 con 0x7efd041601e0 2023-03-29T10:57:04.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.483+0000 7efd00ff9700 1 -- 172.21.15.119:0/2930342405 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7efcf803bb10 con 0x7efd041601e0 2023-03-29T10:57:04.529 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:04.528+0000 7fa172b1e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-03-29T10:57:04.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.569+0000 7efd0a9ff700 1 -- 172.21.15.119:0/2930342405 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fb37ec18-5612-46aa-ae6b-682bc8fc784f", "id": 6} v 0) v1 -- 0x7efd040c7ff0 con 0x7efd041601e0 2023-03-29T10:57:04.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.573+0000 7efd00ff9700 1 -- 172.21.15.119:0/2930342405 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fb37ec18-5612-46aa-ae6b-682bc8fc784f", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7efd040c7ff0 con 0x7efd041601e0 2023-03-29T10:57:04.574 INFO:teuthology.orchestra.run.smithi119.stdout:6 2023-03-29T10:57:04.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.574+0000 7efd0a9ff700 1 -- 172.21.15.119:0/2930342405 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd041601e0 msgr2=0x7efd040b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:04.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.574+0000 7efd0a9ff700 1 --2- 172.21.15.119:0/2930342405 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd041601e0 0x7efd040b3960 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efcf800ebe0 tx=0x7efcf800eef0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.574+0000 7efd0a9ff700 1 -- 172.21.15.119:0/2930342405 shutdown_connections 2023-03-29T10:57:04.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.574+0000 7efd0a9ff700 1 --2- 172.21.15.119:0/2930342405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd040b87a0 0x7efd040c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.574+0000 7efd0a9ff700 1 --2- 172.21.15.119:0/2930342405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd04160b70 0x7efd040b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.574+0000 7efd0a9ff700 1 --2- 172.21.15.119:0/2930342405 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd041601e0 0x7efd040b3960 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.574+0000 7efd0a9ff700 1 -- 172.21.15.119:0/2930342405 >> 172.21.15.119:0/2930342405 conn(0x7efd04071b10 msgr2=0x7efd0406bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:04.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.574+0000 7efd0a9ff700 1 -- 172.21.15.119:0/2930342405 shutdown_connections 2023-03-29T10:57:04.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.574+0000 7efd0a9ff700 1 -- 172.21.15.119:0/2930342405 wait complete. 2023-03-29T10:57:04.591 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd new 859085a7-fa35-4b0b-8583-f6e43ebfa7ce 7 2023-03-29T10:57:04.622 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:04.622+0000 7fe1bee6a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-03-29T10:57:04.702 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:04.701+0000 7fa172b1e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-03-29T10:57:04.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.770+0000 7f70e57ff700 1 Processor -- start 2023-03-29T10:57:04.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.770+0000 7f70e57ff700 1 -- start start 2023-03-29T10:57:04.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.771+0000 7f70e57ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70e00cfbf0 0x7f70e01608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:04.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.771+0000 7f70e57ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f70e00d1100 0x7f70e00d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:04.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.771+0000 7f70e57ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e00d0020 con 0x7f70e00b87a0 2023-03-29T10:57:04.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.771+0000 7f70e57ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e00d02d0 con 0x7f70e00d1100 2023-03-29T10:57:04.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.772+0000 7f70e57ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00d0cb0 con 0x7f70e00cfbf0 2023-03-29T10:57:04.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.772+0000 7f70deffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f70e00b87a0 0x7f70e01601e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59192/0 (socket says 172.21.15.119:59192) 2023-03-29T10:57:04.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.772+0000 7f70d67fc700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70e00cfbf0 0x7f70e01608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:04.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.772+0000 7f70deffd700 1 -- 172.21.15.119:0/2306553812 learned_addr learned my addr 172.21.15.119:0/2306553812 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-03-29T10:57:04.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.772+0000 7f70df7fe700 1 --2- 172.21.15.119:0/2306553812 >> v2:172.21.15.111:3300/0 conn(0x7f70e00d1100 0x7f70e00d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:04.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.772+0000 7f70d67fc700 1 -- 172.21.15.119:0/2306553812 >> v2:172.21.15.111:3300/0 conn(0x7f70e00d1100 msgr2=0x7f70e00d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:04.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.772+0000 7f70d67fc700 1 --2- 172.21.15.119:0/2306553812 >> v2:172.21.15.111:3300/0 conn(0x7f70e00d1100 0x7f70e00d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.772+0000 7f70d67fc700 1 -- 172.21.15.119:0/2306553812 >> v1:172.21.15.119:6789/0 conn(0x7f70e00b87a0 legacy=0x7f70e01601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:04.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.772+0000 7f70d67fc700 1 -- 172.21.15.119:0/2306553812 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e00cdc60 con 0x7f70e00cfbf0 2023-03-29T10:57:04.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.772+0000 7f70de7fc700 1 -- 172.21.15.119:0/2306553812 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775279091 0 0) 0x7f70e00d0020 con 0x7f70e00b87a0 2023-03-29T10:57:04.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.772+0000 7f70d67fc700 1 --2- 172.21.15.119:0/2306553812 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70e00cfbf0 0x7f70e01608f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009840 tx=0x7f70c8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e2b3551d4c7c6e5 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:04.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.773+0000 7f70de7fc700 1 -- 172.21.15.119:0/2306553812 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8011830 con 0x7f70e00cfbf0 2023-03-29T10:57:04.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.773+0000 7f70de7fc700 1 -- 172.21.15.119:0/2306553812 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c8011990 con 0x7f70e00cfbf0 2023-03-29T10:57:04.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.773+0000 7f70de7fc700 1 -- 172.21.15.119:0/2306553812 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8011d20 con 0x7f70e00cfbf0 2023-03-29T10:57:04.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.773+0000 7f70e57ff700 1 -- 172.21.15.119:0/2306553812 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70e00cfbf0 msgr2=0x7f70e01608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:04.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.773+0000 7f70e57ff700 1 --2- 172.21.15.119:0/2306553812 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70e00cfbf0 0x7f70e01608f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009840 tx=0x7f70c8009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:04.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.773+0000 7f70e57ff700 1 -- 172.21.15.119:0/2306553812 shutdown_connections 2023-03-29T10:57:04.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.773+0000 7f70e57ff700 1 --2- 172.21.15.119:0/2306553812 >> v2:172.21.15.111:3300/0 conn(0x7f70e00d1100 0x7f70e00d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.773+0000 7f70e57ff700 1 --2- 172.21.15.119:0/2306553812 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70e00cfbf0 0x7f70e01608f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.773+0000 7f70e57ff700 1 -- 172.21.15.119:0/2306553812 >> 172.21.15.119:0/2306553812 conn(0x7f70e0071b10 msgr2=0x7f70e0071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:04.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.773+0000 7f70e57ff700 1 -- 172.21.15.119:0/2306553812 shutdown_connections 2023-03-29T10:57:04.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.774+0000 7f70e57ff700 1 -- 172.21.15.119:0/2306553812 wait complete. 2023-03-29T10:57:04.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.775+0000 7f70e57ff700 1 Processor -- start 2023-03-29T10:57:04.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.775+0000 7f70e57ff700 1 -- start start 2023-03-29T10:57:04.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.776+0000 7f70e57ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70e00b87a0 0x7f70e00cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:04.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.776+0000 7f70e57ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70e00cfbf0 0x7f70e0153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:04.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.776+0000 7f70e57ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70e00d1100 0x7f70e01e7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:04.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.776+0000 7f70e57ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e01e8e30 con 0x7f70e00d1100 2023-03-29T10:57:04.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.776+0000 7f70e57ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e01e8100 con 0x7f70e00cfbf0 2023-03-29T10:57:04.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.776+0000 7f70e57ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00c9f80 con 0x7f70e00b87a0 2023-03-29T10:57:04.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.776+0000 7f70deffd700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70e00b87a0 0x7f70e00cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:04.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.776+0000 7f70d67fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70e00cfbf0 0x7f70e0153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:04.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.776+0000 7f70deffd700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70e00b87a0 0x7f70e00cb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.119:35452/0 (socket says 172.21.15.119:35452) 2023-03-29T10:57:04.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.776+0000 7f70deffd700 1 -- 172.21.15.119:0/2120006269 learned_addr learned my addr 172.21.15.119:0/2120006269 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-03-29T10:57:04.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.776+0000 7f70df7fe700 1 --2- 172.21.15.119:0/2120006269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70e00d1100 0x7f70e01e7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:04.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.777+0000 7f70d67fc700 1 -- 172.21.15.119:0/2120006269 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70e00b87a0 msgr2=0x7f70e00cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:04.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.777+0000 7f70d67fc700 1 --2- 172.21.15.119:0/2120006269 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70e00b87a0 0x7f70e00cb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.777+0000 7f70d67fc700 1 -- 172.21.15.119:0/2120006269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70e00d1100 msgr2=0x7f70e01e7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:04.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.777+0000 7f70d67fc700 1 --2- 172.21.15.119:0/2120006269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70e00d1100 0x7f70e01e7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.777+0000 7f70d67fc700 1 -- 172.21.15.119:0/2120006269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c80094a0 con 0x7f70e00cfbf0 2023-03-29T10:57:04.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.777+0000 7f70deffd700 1 --2- 172.21.15.119:0/2120006269 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70e00b87a0 0x7f70e00cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:04.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.777+0000 7f70d67fc700 1 --2- 172.21.15.119:0/2120006269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70e00cfbf0 0x7f70e0153f10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f70c8016000 tx=0x7f70c8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:04.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.777+0000 7f70d7fff700 1 -- 172.21.15.119:0/2120006269 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c80036a0 con 0x7f70e00cfbf0 2023-03-29T10:57:04.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.777+0000 7f70d7fff700 1 -- 172.21.15.119:0/2120006269 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c801f070 con 0x7f70e00cfbf0 2023-03-29T10:57:04.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.777+0000 7f70d7fff700 1 -- 172.21.15.119:0/2120006269 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8003a20 con 0x7f70e00cfbf0 2023-03-29T10:57:04.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.777+0000 7f70e57ff700 1 -- 172.21.15.119:0/2120006269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e00cd980 con 0x7f70e00cfbf0 2023-03-29T10:57:04.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.777+0000 7f70e57ff700 1 -- 172.21.15.119:0/2120006269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e00cd6a0 con 0x7f70e00cfbf0 2023-03-29T10:57:04.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.778+0000 7f70d7fff700 1 -- 172.21.15.119:0/2120006269 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f70c8026480 con 0x7f70e00cfbf0 2023-03-29T10:57:04.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.778+0000 7f70d7fff700 1 -- 172.21.15.119:0/2120006269 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f70c8021070 con 0x7f70e00cfbf0 2023-03-29T10:57:04.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.779+0000 7f70e57ff700 1 -- 172.21.15.119:0/2120006269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e0160e30 con 0x7f70e00cfbf0 2023-03-29T10:57:04.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.781+0000 7f70d7fff700 1 -- 172.21.15.119:0/2120006269 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f70c802b020 con 0x7f70e00cfbf0 2023-03-29T10:57:04.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.865+0000 7f70e57ff700 1 -- 172.21.15.119:0/2120006269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "859085a7-fa35-4b0b-8583-f6e43ebfa7ce", "id": 7} v 0) v1 -- 0x7f70e00d0020 con 0x7f70e00cfbf0 2023-03-29T10:57:04.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.870+0000 7f70d7fff700 1 -- 172.21.15.119:0/2120006269 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "859085a7-fa35-4b0b-8583-f6e43ebfa7ce", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f70c8015020 con 0x7f70e00cfbf0 2023-03-29T10:57:04.874 INFO:teuthology.orchestra.run.smithi119.stdout:7 2023-03-29T10:57:04.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.870+0000 7f70e57ff700 1 -- 172.21.15.119:0/2120006269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70e00cfbf0 msgr2=0x7f70e0153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:04.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.870+0000 7f70e57ff700 1 --2- 172.21.15.119:0/2120006269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70e00cfbf0 0x7f70e0153f10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f70c8016000 tx=0x7f70c8004750 comp rx=0 tx=0).stop 2023-03-29T10:57:04.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.871+0000 7f70e57ff700 1 -- 172.21.15.119:0/2120006269 shutdown_connections 2023-03-29T10:57:04.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.871+0000 7f70e57ff700 1 --2- 172.21.15.119:0/2120006269 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70e00d1100 0x7f70e01e7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.871+0000 7f70e57ff700 1 --2- 172.21.15.119:0/2120006269 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70e00b87a0 0x7f70e00cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.871+0000 7f70e57ff700 1 --2- 172.21.15.119:0/2120006269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70e00cfbf0 0x7f70e0153f10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:04.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.871+0000 7f70e57ff700 1 -- 172.21.15.119:0/2120006269 >> 172.21.15.119:0/2120006269 conn(0x7f70e0071b10 msgr2=0x7f70e00b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:04.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.871+0000 7f70e57ff700 1 -- 172.21.15.119:0/2120006269 shutdown_connections 2023-03-29T10:57:04.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-03-29T10:57:04.871+0000 7f70e57ff700 1 -- 172.21.15.119:0/2120006269 wait complete. 2023-03-29T10:57:04.887 INFO:tasks.ceph.osd.0:Restarting daemon 2023-03-29T10:57:04.888 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-03-29T10:57:04.891 INFO:tasks.ceph.osd.0:Started 2023-03-29T10:57:04.891 INFO:tasks.ceph.osd.1:Restarting daemon 2023-03-29T10:57:04.891 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-03-29T10:57:04.895 INFO:tasks.ceph.osd.1:Started 2023-03-29T10:57:04.895 INFO:tasks.ceph.osd.2:Restarting daemon 2023-03-29T10:57:04.895 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-03-29T10:57:04.899 INFO:tasks.ceph.osd.2:Started 2023-03-29T10:57:04.899 INFO:tasks.ceph.osd.3:Restarting daemon 2023-03-29T10:57:04.899 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-03-29T10:57:04.903 INFO:tasks.ceph.osd.3:Started 2023-03-29T10:57:04.903 INFO:tasks.ceph.osd.4:Restarting daemon 2023-03-29T10:57:04.903 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-03-29T10:57:04.907 INFO:tasks.ceph.osd.4:Started 2023-03-29T10:57:04.907 INFO:tasks.ceph.osd.5:Restarting daemon 2023-03-29T10:57:04.908 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-03-29T10:57:04.912 INFO:tasks.ceph.osd.5:Started 2023-03-29T10:57:04.912 INFO:tasks.ceph.osd.6:Restarting daemon 2023-03-29T10:57:04.912 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-03-29T10:57:04.916 INFO:tasks.ceph.osd.6:Started 2023-03-29T10:57:04.916 INFO:tasks.ceph.osd.7:Restarting daemon 2023-03-29T10:57:04.916 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-03-29T10:57:04.920 INFO:tasks.ceph.osd.7:Started 2023-03-29T10:57:04.920 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:57:05.124 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:05.123+0000 7fe1bee6a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-03-29T10:57:05.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.158+0000 7f26fbbff700 1 Processor -- start 2023-03-29T10:57:05.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.158+0000 7f26fbbff700 1 -- start start 2023-03-29T10:57:05.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.159+0000 7f26fbbff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f26f40cf950 0x7f26f40cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:05.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.159+0000 7f26fbbff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f26f40d3a30 0x7f26f40b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:05.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.159+0000 7f26fbbff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f40cdc80 con 0x7f26f415c950 2023-03-29T10:57:05.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.159+0000 7f26fbbff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26f40d0260 con 0x7f26f40d3a30 2023-03-29T10:57:05.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.159+0000 7f26fbbff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f40cd6c0 con 0x7f26f40cf950 2023-03-29T10:57:05.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.160+0000 7f26f999b700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f26f40cf950 0x7f26f40cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:05.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.160+0000 7f26fa19c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f26f40d3a30 0x7f26f40b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:05.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.160+0000 7f26f999b700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f26f40cf950 0x7f26f40cfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:60534/0 (socket says 172.21.15.111:60534) 2023-03-29T10:57:05.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.160+0000 7f26f999b700 1 -- 172.21.15.111:0/3993111904 learned_addr learned my addr 172.21.15.111:0/3993111904 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:05.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.160+0000 7f26f999b700 1 -- 172.21.15.111:0/3993111904 >> v2:172.21.15.119:3300/0 conn(0x7f26f40d3a30 msgr2=0x7f26f40b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:05.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.160+0000 7f26f999b700 1 --2- 172.21.15.111:0/3993111904 >> v2:172.21.15.119:3300/0 conn(0x7f26f40d3a30 0x7f26f40b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:05.161 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.160+0000 7f26f999b700 1 -- 172.21.15.111:0/3993111904 >> v1:172.21.15.119:6789/0 conn(0x7f26f415c950 legacy=0x7f26f40cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:05.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.160+0000 7f26f999b700 1 -- 172.21.15.111:0/3993111904 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f40cd9a0 con 0x7f26f40cf950 2023-03-29T10:57:05.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.160+0000 7f26f999b700 1 --2- 172.21.15.111:0/3993111904 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f26f40cf950 0x7f26f40cfd20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f26f00098f0 tx=0x7f26f0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddeb3f57876d89ea server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:05.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.161+0000 7f26f8999700 1 -- 172.21.15.111:0/3993111904 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0011830 con 0x7f26f40cf950 2023-03-29T10:57:05.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.161+0000 7f26f8999700 1 -- 172.21.15.111:0/3993111904 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f0011990 con 0x7f26f40cf950 2023-03-29T10:57:05.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.161+0000 7f26f8999700 1 -- 172.21.15.111:0/3993111904 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0011d20 con 0x7f26f40cf950 2023-03-29T10:57:05.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.161+0000 7f26fbbff700 1 -- 172.21.15.111:0/3993111904 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f26f40cf950 msgr2=0x7f26f40cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:05.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.161+0000 7f26fbbff700 1 --2- 172.21.15.111:0/3993111904 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f26f40cf950 0x7f26f40cfd20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f26f00098f0 tx=0x7f26f0009c00 comp rx=0 tx=0).stop 2023-03-29T10:57:05.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.161+0000 7f26fbbff700 1 -- 172.21.15.111:0/3993111904 shutdown_connections 2023-03-29T10:57:05.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.161+0000 7f26fbbff700 1 --2- 172.21.15.111:0/3993111904 >> v2:172.21.15.119:3300/0 conn(0x7f26f40d3a30 0x7f26f40b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:05.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.161+0000 7f26fbbff700 1 --2- 172.21.15.111:0/3993111904 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f26f40cf950 0x7f26f40cfd20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:05.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.161+0000 7f26fbbff700 1 -- 172.21.15.111:0/3993111904 >> 172.21.15.111:0/3993111904 conn(0x7f26f4071b10 msgr2=0x7f26f4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:05.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.161+0000 7f26fbbff700 1 -- 172.21.15.111:0/3993111904 shutdown_connections 2023-03-29T10:57:05.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.161+0000 7f26fbbff700 1 -- 172.21.15.111:0/3993111904 wait complete. 2023-03-29T10:57:05.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.163+0000 7f26fbbff700 1 Processor -- start 2023-03-29T10:57:05.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.163+0000 7f26fbbff700 1 -- start start 2023-03-29T10:57:05.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.164+0000 7f26fbbff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26f40cf950 0x7f26f415a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:05.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.164+0000 7f26fbbff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f26f40d3a30 0x7f26f40cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:05.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.164+0000 7f26fbbff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26f415c950 0x7f26f41f2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:05.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.164+0000 7f26fbbff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f40cc460 con 0x7f26f415c950 2023-03-29T10:57:05.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.164+0000 7f26fbbff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f40c8be0 con 0x7f26f40cf950 2023-03-29T10:57:05.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.164+0000 7f26fbbff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f40c7ff0 con 0x7f26f40d3a30 2023-03-29T10:57:05.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.164+0000 7f26f999b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26f40cf950 0x7f26f415a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:05.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.164+0000 7f26f919a700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f26f40d3a30 0x7f26f40cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:05.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.164+0000 7f26fa19c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26f415c950 0x7f26f41f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:05.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.164+0000 7f26f999b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26f40cf950 0x7f26f415a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:60538/0 (socket says 172.21.15.111:60538) 2023-03-29T10:57:05.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.164+0000 7f26f999b700 1 -- 172.21.15.111:0/1059794637 learned_addr learned my addr 172.21.15.111:0/1059794637 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:05.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.165+0000 7f26f919a700 1 -- 172.21.15.111:0/1059794637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26f40cf950 msgr2=0x7f26f415a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:05.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.165+0000 7f26f919a700 1 --2- 172.21.15.111:0/1059794637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26f40cf950 0x7f26f415a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:05.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.165+0000 7f26f919a700 1 -- 172.21.15.111:0/1059794637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26f415c950 msgr2=0x7f26f41f2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:05.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.165+0000 7f26f919a700 1 --2- 172.21.15.111:0/1059794637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26f415c950 0x7f26f41f2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:05.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.165+0000 7f26f919a700 1 -- 172.21.15.111:0/1059794637 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f0009550 con 0x7f26f40d3a30 2023-03-29T10:57:05.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.165+0000 7f26f999b700 1 --2- 172.21.15.111:0/1059794637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26f40cf950 0x7f26f415a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:05.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.165+0000 7f26f919a700 1 --2- 172.21.15.111:0/1059794637 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f26f40d3a30 0x7f26f40cccb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f26e4000f60 tx=0x7f26e400c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:05.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.165+0000 7f26ea7fc700 1 -- 172.21.15.111:0/1059794637 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e40052d0 con 0x7f26f40d3a30 2023-03-29T10:57:05.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.165+0000 7f26ea7fc700 1 -- 172.21.15.111:0/1059794637 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e4005430 con 0x7f26f40d3a30 2023-03-29T10:57:05.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.165+0000 7f26fbbff700 1 -- 172.21.15.111:0/1059794637 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f40d19a0 con 0x7f26f40d3a30 2023-03-29T10:57:05.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.165+0000 7f26ea7fc700 1 -- 172.21.15.111:0/1059794637 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e40038c0 con 0x7f26f40d3a30 2023-03-29T10:57:05.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.165+0000 7f26fbbff700 1 -- 172.21.15.111:0/1059794637 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f40d16c0 con 0x7f26f40d3a30 2023-03-29T10:57:05.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.165+0000 7f26ea7fc700 1 -- 172.21.15.111:0/1059794637 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f26e4003d20 con 0x7f26f40d3a30 2023-03-29T10:57:05.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.166+0000 7f26ea7fc700 1 -- 172.21.15.111:0/1059794637 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f26e4012af0 con 0x7f26f40d3a30 2023-03-29T10:57:05.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.166+0000 7f26fbbff700 1 -- 172.21.15.111:0/1059794637 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f40b62d0 con 0x7f26f40d3a30 2023-03-29T10:57:05.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.169+0000 7f26ea7fc700 1 -- 172.21.15.111:0/1059794637 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f26e4017070 con 0x7f26f40d3a30 2023-03-29T10:57:05.188 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:05.187+0000 7fa172b1e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-03-29T10:57:05.221 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:05.221+0000 7fe1bee6a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-03-29T10:57:05.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.242+0000 7f26fbbff700 1 -- 172.21.15.111:0/1059794637 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26f40c83c0 con 0x7f26f40d3a30 2023-03-29T10:57:05.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.243+0000 7f26ea7fc700 1 -- 172.21.15.111:0/1059794637 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f26e401c440 con 0x7f26f40d3a30 2023-03-29T10:57:05.244 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:05.244 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":10,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:57:04.868340+0000","last_up_change":"0.000000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:57:05.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.244+0000 7f26fbbff700 1 -- 172.21.15.111:0/1059794637 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f26f40d3a30 msgr2=0x7f26f40cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:05.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.244+0000 7f26fbbff700 1 --2- 172.21.15.111:0/1059794637 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f26f40d3a30 0x7f26f40cccb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f26e4000f60 tx=0x7f26e400c6e0 comp rx=0 tx=0).stop 2023-03-29T10:57:05.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.244+0000 7f26fbbff700 1 -- 172.21.15.111:0/1059794637 shutdown_connections 2023-03-29T10:57:05.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.244+0000 7f26fbbff700 1 --2- 172.21.15.111:0/1059794637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f26f415c950 0x7f26f41f2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:05.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.244+0000 7f26fbbff700 1 --2- 172.21.15.111:0/1059794637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26f40cf950 0x7f26f415a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:05.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.244+0000 7f26fbbff700 1 --2- 172.21.15.111:0/1059794637 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f26f40d3a30 0x7f26f40cccb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:05.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.244+0000 7f26fbbff700 1 -- 172.21.15.111:0/1059794637 >> 172.21.15.111:0/1059794637 conn(0x7f26f4071b10 msgr2=0x7f26f40b8670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:05.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.244+0000 7f26fbbff700 1 -- 172.21.15.111:0/1059794637 shutdown_connections 2023-03-29T10:57:05.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:05.244+0000 7f26fbbff700 1 -- 172.21.15.111:0/1059794637 wait complete. 2023-03-29T10:57:05.262 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-03-29T10:57:05.262 INFO:tasks.ceph:Waiting for OSDs to come up 2023-03-29T10:57:05.281 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:05.280+0000 7fa172b1e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-03-29T10:57:05.400 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:05.399+0000 7fe1bee6a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-03-29T10:57:05.457 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:05.456+0000 7fa172b1e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-03-29T10:57:05.540 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:05.539+0000 7fe1bee6a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-03-29T10:57:05.595 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-03-29T10:57:05.594+0000 7fd237865640 -1 Falling back to public interface 2023-03-29T10:57:05.598 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:05.597+0000 7fa172b1e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-03-29T10:57:05.600 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-03-29T10:57:05.599+0000 7f22c23a3640 -1 Falling back to public interface 2023-03-29T10:57:05.600 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-03-29T10:57:05.599+0000 7fabc169a640 -1 Falling back to public interface 2023-03-29T10:57:05.603 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-03-29T10:57:05.602+0000 7f047b0f0640 -1 Falling back to public interface 2023-03-29T10:57:05.610 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T10:57:05.610+0000 7f7a79a7a640 -1 Falling back to public interface 2023-03-29T10:57:05.618 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-03-29T10:57:05.618+0000 7f0c40fb3640 -1 Falling back to public interface 2023-03-29T10:57:05.624 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-03-29T10:57:05.624+0000 7f93de8cf640 -1 Falling back to public interface 2023-03-29T10:57:05.649 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-03-29T10:57:05.649+0000 7f9665f6f640 -1 Falling back to public interface 2023-03-29T10:57:05.718 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:05.718+0000 7fe1bee6a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-03-29T10:57:05.773 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:05.772+0000 7fa172b1e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-03-29T10:57:05.971 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:05.970+0000 7fe1bee6a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-03-29T10:57:06.024 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:06.024+0000 7fa172b1e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-03-29T10:57:06.067 DEBUG:teuthology.orchestra.run.smithi111:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-03-29T10:57:06.071 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:06.070+0000 7fe1bee6a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-03-29T10:57:06.122 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:06.121+0000 7fa172b1e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-03-29T10:57:06.169 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:06.169+0000 7fe1bee6a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-03-29T10:57:06.218 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:06.217+0000 7fa172b1e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-03-29T10:57:06.234 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.233+0000 7f5a009ff700 1 Processor -- start 2023-03-29T10:57:06.234 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.233+0000 7f5a009ff700 1 -- start start 2023-03-29T10:57:06.237 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.235+0000 7f5a009ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59fc0cf570 0x7f59fc0c9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:06.237 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.235+0000 7f5a009ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f59fc0c96d0 0x7f59fc15e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:06.237 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.235+0000 7f5a009ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f59fc15e5e0 0x7f59fc0cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:06.237 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.235+0000 7f5a009ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59fc0cf940 con 0x7f59fc15e5e0 2023-03-29T10:57:06.237 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.235+0000 7f5a009ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59fc0c9b20 con 0x7f59fc0c96d0 2023-03-29T10:57:06.237 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.235+0000 7f5a009ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc15b910 con 0x7f59fc0cf570 2023-03-29T10:57:06.237 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.235+0000 7f59fa59c700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59fc0cf570 0x7f59fc0c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:06.237 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.235+0000 7f59f3fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f59fc0c96d0 0x7f59fc15e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:06.238 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.235+0000 7f59fa59c700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59fc0cf570 0x7f59fc0c9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:60550/0 (socket says 172.21.15.111:60550) 2023-03-29T10:57:06.238 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.235+0000 7f59fa59c700 1 -- 172.21.15.111:0/3265868002 learned_addr learned my addr 172.21.15.111:0/3265868002 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:06.238 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.235+0000 7f59fad9d700 1 --2- 172.21.15.111:0/3265868002 >> v2:172.21.15.111:3300/0 conn(0x7f59fc15e5e0 0x7f59fc0cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:06.238 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.235+0000 7f59fa59c700 1 -- 172.21.15.111:0/3265868002 >> v2:172.21.15.119:3300/0 conn(0x7f59fc0c96d0 msgr2=0x7f59fc15e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:06.238 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.235+0000 7f59fa59c700 1 --2- 172.21.15.111:0/3265868002 >> v2:172.21.15.119:3300/0 conn(0x7f59fc0c96d0 0x7f59fc15e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:06.238 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.235+0000 7f59fa59c700 1 -- 172.21.15.111:0/3265868002 >> v2:172.21.15.111:3300/0 conn(0x7f59fc15e5e0 msgr2=0x7f59fc0cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:06.238 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.235+0000 7f59fa59c700 1 --2- 172.21.15.111:0/3265868002 >> v2:172.21.15.111:3300/0 conn(0x7f59fc15e5e0 0x7f59fc0cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:06.238 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.235+0000 7f59fa59c700 1 -- 172.21.15.111:0/3265868002 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59fc15ee10 con 0x7f59fc0cf570 2023-03-29T10:57:06.238 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.235+0000 7f59f3fff700 1 --2- 172.21.15.111:0/3265868002 >> v2:172.21.15.119:3300/0 conn(0x7f59fc0c96d0 0x7f59fc15e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:06.238 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.236+0000 7f59fa59c700 1 --2- 172.21.15.111:0/3265868002 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59fc0cf570 0x7f59fc0c9190 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0cfb50 tx=0x7f59ec009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15dee7f6e322ca7c server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:06.238 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.236+0000 7f59f9d9b700 1 -- 172.21.15.111:0/3265868002 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec011830 con 0x7f59fc0cf570 2023-03-29T10:57:06.238 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.236+0000 7f59f9d9b700 1 -- 172.21.15.111:0/3265868002 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ec011990 con 0x7f59fc0cf570 2023-03-29T10:57:06.239 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.236+0000 7f59f9d9b700 1 -- 172.21.15.111:0/3265868002 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec011d20 con 0x7f59fc0cf570 2023-03-29T10:57:06.239 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.237+0000 7f5a009ff700 1 -- 172.21.15.111:0/3265868002 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59fc0cf570 msgr2=0x7f59fc0c9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:06.240 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.237+0000 7f5a009ff700 1 --2- 172.21.15.111:0/3265868002 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59fc0cf570 0x7f59fc0c9190 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f59fc0cfb50 tx=0x7f59ec009bd0 comp rx=0 tx=0).stop 2023-03-29T10:57:06.240 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.237+0000 7f5a009ff700 1 -- 172.21.15.111:0/3265868002 shutdown_connections 2023-03-29T10:57:06.240 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.237+0000 7f5a009ff700 1 --2- 172.21.15.111:0/3265868002 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59fc0cf570 0x7f59fc0c9190 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:06.240 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.237+0000 7f5a009ff700 1 --2- 172.21.15.111:0/3265868002 >> v2:172.21.15.111:3300/0 conn(0x7f59fc15e5e0 0x7f59fc0cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:06.240 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.237+0000 7f5a009ff700 1 --2- 172.21.15.111:0/3265868002 >> v2:172.21.15.119:3300/0 conn(0x7f59fc0c96d0 0x7f59fc15e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:06.240 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.237+0000 7f5a009ff700 1 -- 172.21.15.111:0/3265868002 >> 172.21.15.111:0/3265868002 conn(0x7f59fc06a3c0 msgr2=0x7f59fc06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:06.240 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.237+0000 7f5a009ff700 1 -- 172.21.15.111:0/3265868002 shutdown_connections 2023-03-29T10:57:06.240 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.237+0000 7f5a009ff700 1 -- 172.21.15.111:0/3265868002 wait complete. 2023-03-29T10:57:06.240 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.238+0000 7f5a009ff700 1 Processor -- start 2023-03-29T10:57:06.241 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.239+0000 7f5a009ff700 1 -- start start 2023-03-29T10:57:06.243 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.239+0000 7f5a009ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59fc0c96d0 0x7f59fc1e7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:06.243 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.239+0000 7f5a009ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59fc0cf570 0x7f59fc1e4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:06.243 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.239+0000 7f5a009ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59fc15e5e0 0x7f59fc13edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:06.243 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.239+0000 7f5a009ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc1e8cc0 con 0x7f59fc0c96d0 2023-03-29T10:57:06.243 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.240+0000 7f5a009ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc1e8420 con 0x7f59fc0cf570 2023-03-29T10:57:06.243 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.240+0000 7f5a009ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc1e8700 con 0x7f59fc15e5e0 2023-03-29T10:57:06.244 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.240+0000 7f59fa59c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59fc0c96d0 0x7f59fc1e7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:06.244 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.240+0000 7f59fa59c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59fc0c96d0 0x7f59fc1e7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:57094/0 (socket says 172.21.15.111:57094) 2023-03-29T10:57:06.244 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.240+0000 7f59fa59c700 1 -- 172.21.15.111:0/685444383 learned_addr learned my addr 172.21.15.111:0/685444383 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:06.244 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.240+0000 7f59fad9d700 1 --2- 172.21.15.111:0/685444383 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59fc15e5e0 0x7f59fc13edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:06.244 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.240+0000 7f59f3fff700 1 --2- 172.21.15.111:0/685444383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59fc0cf570 0x7f59fc1e4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:06.244 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.240+0000 7f59fa59c700 1 -- 172.21.15.111:0/685444383 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59fc15e5e0 msgr2=0x7f59fc13edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:06.244 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.240+0000 7f59fa59c700 1 --2- 172.21.15.111:0/685444383 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59fc15e5e0 0x7f59fc13edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:06.244 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.240+0000 7f59fa59c700 1 -- 172.21.15.111:0/685444383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59fc0cf570 msgr2=0x7f59fc1e4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:06.244 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.240+0000 7f59fa59c700 1 --2- 172.21.15.111:0/685444383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59fc0cf570 0x7f59fc1e4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:06.245 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.240+0000 7f59fa59c700 1 -- 172.21.15.111:0/685444383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e40094a0 con 0x7f59fc0c96d0 2023-03-29T10:57:06.245 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.240+0000 7f59fad9d700 1 --2- 172.21.15.111:0/685444383 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59fc15e5e0 0x7f59fc13edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:06.245 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.240+0000 7f59fa59c700 1 --2- 172.21.15.111:0/685444383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59fc0c96d0 0x7f59fc1e7d90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009c10 tx=0x7f59ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:06.245 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.241+0000 7f59f2ffd700 1 -- 172.21.15.111:0/685444383 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec004450 con 0x7f59fc0c96d0 2023-03-29T10:57:06.245 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.241+0000 7f59f2ffd700 1 -- 172.21.15.111:0/685444383 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ec01e070 con 0x7f59fc0c96d0 2023-03-29T10:57:06.245 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.241+0000 7f59f2ffd700 1 -- 172.21.15.111:0/685444383 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec0038e0 con 0x7f59fc0c96d0 2023-03-29T10:57:06.245 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.241+0000 7f5a009ff700 1 -- 172.21.15.111:0/685444383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ec009550 con 0x7f59fc0c96d0 2023-03-29T10:57:06.245 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.241+0000 7f5a009ff700 1 -- 172.21.15.111:0/685444383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59fc15ae30 con 0x7f59fc0c96d0 2023-03-29T10:57:06.245 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.241+0000 7f59f2ffd700 1 -- 172.21.15.111:0/685444383 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f59ec003d40 con 0x7f59fc0c96d0 2023-03-29T10:57:06.246 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.241+0000 7f59f2ffd700 1 -- 172.21.15.111:0/685444383 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f59ec02d450 con 0x7f59fc0c96d0 2023-03-29T10:57:06.246 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.242+0000 7f5a009ff700 1 -- 172.21.15.111:0/685444383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59fc15ab50 con 0x7f59fc0c96d0 2023-03-29T10:57:06.246 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.245+0000 7f59f2ffd700 1 -- 172.21.15.111:0/685444383 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f59ec020070 con 0x7f59fc0c96d0 2023-03-29T10:57:06.327 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.326+0000 7f5a009ff700 1 -- 172.21.15.111:0/685444383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59fc15b110 con 0x7f59fc0c96d0 2023-03-29T10:57:06.328 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.327+0000 7f59f2ffd700 1 -- 172.21.15.111:0/685444383 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f59ec015020 con 0x7f59fc0c96d0 2023-03-29T10:57:06.328 INFO:teuthology.misc.health.smithi111.stdout: 2023-03-29T10:57:06.328 INFO:teuthology.misc.health.smithi111.stdout:{"epoch":10,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:57:04.868340+0000","last_up_change":"0.000000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:57:06.330 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.328+0000 7f5a009ff700 1 -- 172.21.15.111:0/685444383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59fc0c96d0 msgr2=0x7f59fc1e7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:06.330 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.328+0000 7f5a009ff700 1 --2- 172.21.15.111:0/685444383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59fc0c96d0 0x7f59fc1e7d90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009c10 tx=0x7f59ec004750 comp rx=0 tx=0).stop 2023-03-29T10:57:06.330 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.328+0000 7f5a009ff700 1 -- 172.21.15.111:0/685444383 shutdown_connections 2023-03-29T10:57:06.330 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.328+0000 7f5a009ff700 1 --2- 172.21.15.111:0/685444383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59fc0c96d0 0x7f59fc1e7d90 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:06.330 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.328+0000 7f5a009ff700 1 --2- 172.21.15.111:0/685444383 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59fc15e5e0 0x7f59fc13edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:06.330 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.328+0000 7f5a009ff700 1 --2- 172.21.15.111:0/685444383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59fc0cf570 0x7f59fc1e4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:06.330 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.328+0000 7f5a009ff700 1 -- 172.21.15.111:0/685444383 >> 172.21.15.111:0/685444383 conn(0x7f59fc06a3c0 msgr2=0x7f59fc15bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:06.330 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.328+0000 7f5a009ff700 1 -- 172.21.15.111:0/685444383 shutdown_connections 2023-03-29T10:57:06.330 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:06.328+0000 7f5a009ff700 1 -- 172.21.15.111:0/685444383 wait complete. 2023-03-29T10:57:06.343 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-03-29T10:57:07.112 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:07.111+0000 7fe1bee6a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-03-29T10:57:07.145 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:07.145+0000 7fa172b1e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-03-29T10:57:07.212 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:07.211+0000 7fe1bee6a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-03-29T10:57:07.246 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:07.245+0000 7fa172b1e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-03-29T10:57:07.307 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:07.307+0000 7fe1bee6a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-03-29T10:57:07.340 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:07.340+0000 7fa172b1e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-03-29T10:57:07.409 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:07.408+0000 7fe1bee6a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-03-29T10:57:07.440 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:07.439+0000 7fa172b1e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-03-29T10:57:07.591 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:07.590+0000 7fe1bee6a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-03-29T10:57:07.623 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:07.622+0000 7fa172b1e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-03-29T10:57:07.815 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:07.814+0000 7fe1bee6a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-03-29T10:57:07.845 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:07.845+0000 7fa172b1e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-03-29T10:57:07.907 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:07.906+0000 7fe1bee6a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-03-29T10:57:07.935 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:07.934+0000 7fa172b1e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-03-29T10:57:08.104 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:08.103+0000 7fe1bee6a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-03-29T10:57:08.148 INFO:tasks.ceph.mgr.x.smithi119.stderr:2023-03-29T10:57:08.148+0000 7fa172b1e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-03-29T10:57:09.129 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:09.128+0000 7fe173bfa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-03-29T10:57:09.129 INFO:tasks.ceph.mgr.y.smithi111.stderr:2023-03-29T10:57:09.129+0000 7fe173bfa700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-03-29T10:57:09.153 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-03-29T10:57:09.152+0000 7fd237865640 -1 osd.4 0 log_to_monitors true 2023-03-29T10:57:09.154 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-03-29T10:57:09.153+0000 7fabc169a640 -1 osd.6 0 log_to_monitors true 2023-03-29T10:57:09.157 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-03-29T10:57:09.156+0000 7f0c40fb3640 -1 osd.5 0 log_to_monitors true 2023-03-29T10:57:09.194 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-03-29T10:57:09.194+0000 7f047b0f0640 -1 osd.1 0 log_to_monitors true 2023-03-29T10:57:09.197 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-03-29T10:57:09.196+0000 7f22c23a3640 -1 osd.3 0 log_to_monitors true 2023-03-29T10:57:09.208 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-03-29T10:57:09.207+0000 7f9665f6f640 -1 osd.7 0 log_to_monitors true 2023-03-29T10:57:09.209 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T10:57:09.209+0000 7f7a79a7a640 -1 osd.0 0 log_to_monitors true 2023-03-29T10:57:09.229 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-03-29T10:57:09.229+0000 7f93de8cf640 -1 osd.2 0 log_to_monitors true 2023-03-29T10:57:13.149 DEBUG:teuthology.orchestra.run.smithi111:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-03-29T10:57:13.232 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-03-29T10:57:13.230+0000 7f0c2fdf7700 -1 osd.5 0 waiting for initial osdmap 2023-03-29T10:57:13.257 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-03-29T10:57:13.256+0000 7f0c275fa700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-29T10:57:13.269 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-03-29T10:57:13.268+0000 7f9654df7700 -1 osd.7 0 waiting for initial osdmap 2023-03-29T10:57:13.274 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T10:57:13.272+0000 7f7a689f7700 -1 osd.0 0 waiting for initial osdmap 2023-03-29T10:57:13.286 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-03-29T10:57:13.285+0000 7f964c5fa700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-29T10:57:13.288 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-03-29T10:57:13.287+0000 7f0469ff7700 -1 osd.1 0 waiting for initial osdmap 2023-03-29T10:57:13.289 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-03-29T10:57:13.289+0000 7f22b11f7700 -1 osd.3 0 waiting for initial osdmap 2023-03-29T10:57:13.294 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T10:57:13.291+0000 7f7a601fa700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-29T10:57:13.296 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-03-29T10:57:13.295+0000 7f93cd7f7700 -1 osd.2 0 waiting for initial osdmap 2023-03-29T10:57:13.305 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-03-29T10:57:13.304+0000 7f22a89fa700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-29T10:57:13.308 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-03-29T10:57:13.308+0000 7f04617fa700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-29T10:57:13.315 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-03-29T10:57:13.313+0000 7fabb05f7700 -1 osd.6 0 waiting for initial osdmap 2023-03-29T10:57:13.319 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-03-29T10:57:13.319+0000 7f93c4ffa700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-29T10:57:13.329 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-03-29T10:57:13.328+0000 7faba7dfa700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-29T10:57:13.387 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.386+0000 7f7bf7dff700 1 Processor -- start 2023-03-29T10:57:13.387 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.386+0000 7f7bf7dff700 1 -- start start 2023-03-29T10:57:13.388 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.387+0000 7f7bf7dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7bf015c830 0x7f7bf00cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:13.388 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.387+0000 7f7bf7dff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7bf00b6200 0x7f7bf00d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:13.388 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.387+0000 7f7bf7dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf015cc00 con 0x7f7bf00b5da0 2023-03-29T10:57:13.388 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.387+0000 7f7bf7dff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf00cd640 con 0x7f7bf00b6200 2023-03-29T10:57:13.389 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.387+0000 7f7bf7dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf00cd8e0 con 0x7f7bf015c830 2023-03-29T10:57:13.389 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.387+0000 7f7bf639c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7bf00b6200 0x7f7bf00d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:13.389 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.388+0000 7f7bf639c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7bf00b6200 0x7f7bf00d2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40930/0 (socket says 172.21.15.111:40930) 2023-03-29T10:57:13.389 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.388+0000 7f7bf639c700 1 -- 172.21.15.111:0/1676981728 learned_addr learned my addr 172.21.15.111:0/1676981728 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:13.389 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.388+0000 7f7bf5b9b700 1 --2- 172.21.15.111:0/1676981728 >> v2:172.21.15.119:3300/0 conn(0x7f7bf015c830 0x7f7bf00cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:13.390 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.388+0000 7f7bf639c700 1 -- 172.21.15.111:0/1676981728 >> v2:172.21.15.119:3300/0 conn(0x7f7bf015c830 msgr2=0x7f7bf00cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:13.390 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.388+0000 7f7bf639c700 1 --2- 172.21.15.111:0/1676981728 >> v2:172.21.15.119:3300/0 conn(0x7f7bf015c830 0x7f7bf00cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:13.390 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.388+0000 7f7bf639c700 1 -- 172.21.15.111:0/1676981728 >> v1:172.21.15.119:6789/0 conn(0x7f7bf00b5da0 legacy=0x7f7bf00d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:13.390 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.388+0000 7f7bf639c700 1 -- 172.21.15.111:0/1676981728 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf00d15f0 con 0x7f7bf00b6200 2023-03-29T10:57:13.391 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.388+0000 7f7bf639c700 1 --2- 172.21.15.111:0/1676981728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bf00b6200 0x7f7bf00d2a50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00d0a40 tx=0x7f7be800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ad7c94cae6fc6e server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:13.391 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.388+0000 7f7bf4b99700 1 -- 172.21.15.111:0/1676981728 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be800e070 con 0x7f7bf00b6200 2023-03-29T10:57:13.391 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.388+0000 7f7bf4b99700 1 -- 172.21.15.111:0/1676981728 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be800b910 con 0x7f7bf00b6200 2023-03-29T10:57:13.391 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.388+0000 7f7bf4b99700 1 -- 172.21.15.111:0/1676981728 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be800bcc0 con 0x7f7bf00b6200 2023-03-29T10:57:13.391 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.389+0000 7f7bf7dff700 1 -- 172.21.15.111:0/1676981728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bf00b6200 msgr2=0x7f7bf00d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:13.391 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.389+0000 7f7bf7dff700 1 --2- 172.21.15.111:0/1676981728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bf00b6200 0x7f7bf00d2a50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00d0a40 tx=0x7f7be800b330 comp rx=0 tx=0).stop 2023-03-29T10:57:13.391 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.389+0000 7f7bf7dff700 1 -- 172.21.15.111:0/1676981728 shutdown_connections 2023-03-29T10:57:13.391 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.389+0000 7f7bf7dff700 1 --2- 172.21.15.111:0/1676981728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bf00b6200 0x7f7bf00d2a50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:13.391 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.389+0000 7f7bf7dff700 1 --2- 172.21.15.111:0/1676981728 >> v2:172.21.15.119:3300/0 conn(0x7f7bf015c830 0x7f7bf00cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:13.392 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.389+0000 7f7bf7dff700 1 -- 172.21.15.111:0/1676981728 >> 172.21.15.111:0/1676981728 conn(0x7f7bf0071af0 msgr2=0x7f7bf0071ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:13.392 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.389+0000 7f7bf7dff700 1 -- 172.21.15.111:0/1676981728 shutdown_connections 2023-03-29T10:57:13.392 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.389+0000 7f7bf7dff700 1 -- 172.21.15.111:0/1676981728 wait complete. 2023-03-29T10:57:13.393 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.391+0000 7f7bf7dff700 1 Processor -- start 2023-03-29T10:57:13.394 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.391+0000 7f7bf7dff700 1 -- start start 2023-03-29T10:57:13.394 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.391+0000 7f7bf7dff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bf00b5da0 0x7f7bf00c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:13.394 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.392+0000 7f7bf7dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bf00b6200 0x7f7bf0158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:13.394 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.392+0000 7f7bf7dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7bf015c830 0x7f7bf01585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:13.394 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.392+0000 7f7bf7dff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf00cb5d0 con 0x7f7bf00b5da0 2023-03-29T10:57:13.394 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.392+0000 7f7bf7dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf015dce0 con 0x7f7bf00b6200 2023-03-29T10:57:13.394 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.392+0000 7f7bf7dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf015dfc0 con 0x7f7bf015c830 2023-03-29T10:57:13.394 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.392+0000 7f7bf5b9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bf00b5da0 0x7f7bf00c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:13.394 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.392+0000 7f7bf5b9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bf00b5da0 0x7f7bf00c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40942/0 (socket says 172.21.15.111:40942) 2023-03-29T10:57:13.395 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.392+0000 7f7bf5b9b700 1 -- 172.21.15.111:0/2839293496 learned_addr learned my addr 172.21.15.111:0/2839293496 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:13.395 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.392+0000 7f7bf539a700 1 --2- 172.21.15.111:0/2839293496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bf00b6200 0x7f7bf0158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:13.395 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.392+0000 7f7bf639c700 1 --2- 172.21.15.111:0/2839293496 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7bf015c830 0x7f7bf01585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:13.395 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.392+0000 7f7bf5b9b700 1 -- 172.21.15.111:0/2839293496 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7bf015c830 msgr2=0x7f7bf01585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:13.395 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.392+0000 7f7bf5b9b700 1 --2- 172.21.15.111:0/2839293496 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7bf015c830 0x7f7bf01585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:13.395 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.392+0000 7f7bf5b9b700 1 -- 172.21.15.111:0/2839293496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bf00b6200 msgr2=0x7f7bf0158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:13.395 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.392+0000 7f7bf5b9b700 1 --2- 172.21.15.111:0/2839293496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bf00b6200 0x7f7bf0158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:13.395 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.392+0000 7f7bf5b9b700 1 -- 172.21.15.111:0/2839293496 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be8009b30 con 0x7f7bf00b5da0 2023-03-29T10:57:13.395 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.392+0000 7f7bf639c700 1 --2- 172.21.15.111:0/2839293496 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7bf015c830 0x7f7bf01585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:57:13.395 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.392+0000 7f7bf5b9b700 1 --2- 172.21.15.111:0/2839293496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bf00b5da0 0x7f7bf00c8b90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00e870 tx=0x7f7bec00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:13.396 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.393+0000 7f7be67fc700 1 -- 172.21.15.111:0/2839293496 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec009590 con 0x7f7bf00b5da0 2023-03-29T10:57:13.396 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.393+0000 7f7be67fc700 1 -- 172.21.15.111:0/2839293496 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bec013680 con 0x7f7bf00b5da0 2023-03-29T10:57:13.396 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.393+0000 7f7bf7dff700 1 -- 172.21.15.111:0/2839293496 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf00b2740 con 0x7f7bf00b5da0 2023-03-29T10:57:13.396 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.393+0000 7f7be67fc700 1 -- 172.21.15.111:0/2839293496 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec0139f0 con 0x7f7bf00b5da0 2023-03-29T10:57:13.396 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.393+0000 7f7bf7dff700 1 -- 172.21.15.111:0/2839293496 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf00d18d0 con 0x7f7bf00b5da0 2023-03-29T10:57:13.397 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.394+0000 7f7bf7dff700 1 -- 172.21.15.111:0/2839293496 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf00cdbc0 con 0x7f7bf00b5da0 2023-03-29T10:57:13.397 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.395+0000 7f7be67fc700 1 -- 172.21.15.111:0/2839293496 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 4) v1 ==== 87553+0+0 (secure 0 0 0) 0x7f7bec013b50 con 0x7f7bf00b5da0 2023-03-29T10:57:13.397 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.395+0000 7f7be67fc700 1 --2- 172.21.15.111:0/2839293496 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7bd4069d50 0x7f7bd406c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:13.397 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.396+0000 7f7be67fc700 1 -- 172.21.15.111:0/2839293496 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f7bec086ea0 con 0x7f7bf00b5da0 2023-03-29T10:57:13.397 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.396+0000 7f7bf539a700 1 --2- 172.21.15.111:0/2839293496 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7bd4069d50 0x7f7bd406c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:13.397 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.396+0000 7f7bf539a700 1 --2- 172.21.15.111:0/2839293496 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7bd4069d50 0x7f7bd406c200 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7bf01f5630 tx=0x7f7be0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:13.401 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.400+0000 7f7be67fc700 1 -- 172.21.15.111:0/2839293496 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f7bec053710 con 0x7f7bf00b5da0 2023-03-29T10:57:13.442 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-03-29T10:57:13.441+0000 7fd2267f7700 -1 osd.4 0 waiting for initial osdmap 2023-03-29T10:57:13.455 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-03-29T10:57:13.454+0000 7fd21dffa700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-03-29T10:57:13.599 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.598+0000 7f7bf7dff700 1 -- 172.21.15.111:0/2839293496 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bf00743d0 con 0x7f7bf00b5da0 2023-03-29T10:57:13.599 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.599+0000 7f7be67fc700 1 -- 172.21.15.111:0/2839293496 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f7bec021090 con 0x7f7bf00b5da0 2023-03-29T10:57:13.600 INFO:teuthology.misc.health.smithi111.stdout: 2023-03-29T10:57:13.600 INFO:teuthology.misc.health.smithi111.stdout:{"epoch":12,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:57:11.126944+0000","last_up_change":"0.000000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:57:13.602 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.601+0000 7f7bf7dff700 1 -- 172.21.15.111:0/2839293496 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7bd4069d50 msgr2=0x7f7bd406c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:13.602 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.601+0000 7f7bf7dff700 1 --2- 172.21.15.111:0/2839293496 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7bd4069d50 0x7f7bd406c200 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7bf01f5630 tx=0x7f7be0009040 comp rx=0 tx=0).stop 2023-03-29T10:57:13.602 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.601+0000 7f7bf7dff700 1 -- 172.21.15.111:0/2839293496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bf00b5da0 msgr2=0x7f7bf00c8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:13.602 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.601+0000 7f7bf7dff700 1 --2- 172.21.15.111:0/2839293496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bf00b5da0 0x7f7bf00c8b90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00e870 tx=0x7f7bec00eb80 comp rx=0 tx=0).stop 2023-03-29T10:57:13.602 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.601+0000 7f7bf7dff700 1 -- 172.21.15.111:0/2839293496 shutdown_connections 2023-03-29T10:57:13.603 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.601+0000 7f7bf7dff700 1 --2- 172.21.15.111:0/2839293496 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7bd4069d50 0x7f7bd406c200 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:13.603 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.601+0000 7f7bf7dff700 1 --2- 172.21.15.111:0/2839293496 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7bf00b5da0 0x7f7bf00c8b90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:13.603 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.601+0000 7f7bf7dff700 1 --2- 172.21.15.111:0/2839293496 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7bf015c830 0x7f7bf01585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:13.603 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.601+0000 7f7bf7dff700 1 --2- 172.21.15.111:0/2839293496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bf00b6200 0x7f7bf0158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:13.603 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.601+0000 7f7bf7dff700 1 -- 172.21.15.111:0/2839293496 >> 172.21.15.111:0/2839293496 conn(0x7f7bf0071af0 msgr2=0x7f7bf00b86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:13.603 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.602+0000 7f7bf7dff700 1 -- 172.21.15.111:0/2839293496 shutdown_connections 2023-03-29T10:57:13.603 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:13.602+0000 7f7bf7dff700 1 -- 172.21.15.111:0/2839293496 wait complete. 2023-03-29T10:57:13.617 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-03-29T10:57:20.423 DEBUG:teuthology.orchestra.run.smithi111:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-03-29T10:57:20.587 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.586+0000 7f8d017ff700 1 Processor -- start 2023-03-29T10:57:20.587 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.586+0000 7f8d017ff700 1 -- start start 2023-03-29T10:57:20.588 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.587+0000 7f8d017ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8cfc0cf570 0x7f8cfc0c9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:20.588 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.587+0000 7f8d017ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cfc0cf940 con 0x7f8cfc15e7b0 2023-03-29T10:57:20.588 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.587+0000 7f8d017ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cfc0c9b20 con 0x7f8cfc0c96d0 2023-03-29T10:57:20.588 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.587+0000 7f8d017ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cfc15ebd0 con 0x7f8cfc0cf570 2023-03-29T10:57:20.588 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.587+0000 7f8cfaffd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8cfc0cf570 0x7f8cfc0c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:20.588 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.588+0000 7f8cfaffd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8cfc0cf570 0x7f8cfc0c9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50670/0 (socket says 172.21.15.111:50670) 2023-03-29T10:57:20.588 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.588+0000 7f8cfaffd700 1 -- 172.21.15.111:0/1481010734 learned_addr learned my addr 172.21.15.111:0/1481010734 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:20.589 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.588+0000 7f8cfb7fe700 1 --1- 172.21.15.111:0/1481010734 >> v1:172.21.15.111:6789/0 conn(0x7f8cfc15e7b0 0x7f8cfc06cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53538/0 (socket says 172.21.15.111:53538) 2023-03-29T10:57:20.590 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.588+0000 7f8cfaffd700 1 -- 172.21.15.111:0/1481010734 >> v1:172.21.15.119:6789/0 conn(0x7f8cfc0c96d0 legacy=0x7f8cfc15e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:20.590 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.588+0000 7f8cfaffd700 1 -- 172.21.15.111:0/1481010734 >> v1:172.21.15.111:6789/0 conn(0x7f8cfc15e7b0 legacy=0x7f8cfc06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:20.590 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.588+0000 7f8cfaffd700 1 -- 172.21.15.111:0/1481010734 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc15b910 con 0x7f8cfc0cf570 2023-03-29T10:57:20.590 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.588+0000 7f8cf9ffb700 1 -- 172.21.15.111:0/1481010734 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449522421 0 0) 0x7f8cfc0cf940 con 0x7f8cfc15e7b0 2023-03-29T10:57:20.590 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.588+0000 7f8cfaffd700 1 --2- 172.21.15.111:0/1481010734 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cfc0cf570 0x7f8cfc0c9190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0098f0 tx=0x7f8cec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae7fef17e893ab server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:20.590 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.588+0000 7f8cf9ffb700 1 -- 172.21.15.111:0/1481010734 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec00b8a0 con 0x7f8cfc0cf570 2023-03-29T10:57:20.590 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.589+0000 7f8cf9ffb700 1 -- 172.21.15.111:0/1481010734 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cec00ba00 con 0x7f8cfc0cf570 2023-03-29T10:57:20.590 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.589+0000 7f8cf9ffb700 1 -- 172.21.15.111:0/1481010734 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec00bd90 con 0x7f8cfc0cf570 2023-03-29T10:57:20.590 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.589+0000 7f8d017ff700 1 -- 172.21.15.111:0/1481010734 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cfc0cf570 msgr2=0x7f8cfc0c9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:20.590 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.589+0000 7f8d017ff700 1 --2- 172.21.15.111:0/1481010734 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cfc0cf570 0x7f8cfc0c9190 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0098f0 tx=0x7f8cec009c00 comp rx=0 tx=0).stop 2023-03-29T10:57:20.590 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.589+0000 7f8d017ff700 1 -- 172.21.15.111:0/1481010734 shutdown_connections 2023-03-29T10:57:20.590 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.589+0000 7f8d017ff700 1 --2- 172.21.15.111:0/1481010734 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cfc0cf570 0x7f8cfc0c9190 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:20.590 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.589+0000 7f8d017ff700 1 -- 172.21.15.111:0/1481010734 >> 172.21.15.111:0/1481010734 conn(0x7f8cfc06a3c0 msgr2=0x7f8cfc06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:20.591 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.589+0000 7f8d017ff700 1 -- 172.21.15.111:0/1481010734 shutdown_connections 2023-03-29T10:57:20.591 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.589+0000 7f8d017ff700 1 -- 172.21.15.111:0/1481010734 wait complete. 2023-03-29T10:57:20.591 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.591+0000 7f8d017ff700 1 Processor -- start 2023-03-29T10:57:20.591 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.591+0000 7f8d017ff700 1 -- start start 2023-03-29T10:57:20.594 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.592+0000 7f8d017ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cfc0c96d0 0x7f8cfc14a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:20.594 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.592+0000 7f8d017ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8cfc0cf570 0x7f8cfc1f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:20.594 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.592+0000 7f8d017ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cfc15e7b0 0x7f8cfc0b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:20.594 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.592+0000 7f8d017ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc1f0ed0 con 0x7f8cfc15e7b0 2023-03-29T10:57:20.594 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.592+0000 7f8d017ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc1f05a0 con 0x7f8cfc0c96d0 2023-03-29T10:57:20.594 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.592+0000 7f8d017ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc1f0880 con 0x7f8cfc0cf570 2023-03-29T10:57:20.594 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.592+0000 7f8cfb7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cfc15e7b0 0x7f8cfc0b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:20.594 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.592+0000 7f8cfb7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cfc15e7b0 0x7f8cfc0b07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50680/0 (socket says 172.21.15.111:50680) 2023-03-29T10:57:20.594 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.592+0000 7f8cfb7fe700 1 -- 172.21.15.111:0/2801909361 learned_addr learned my addr 172.21.15.111:0/2801909361 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:20.594 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.593+0000 7f8cfaffd700 1 --2- 172.21.15.111:0/2801909361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cfc0c96d0 0x7f8cfc14a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:20.594 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.593+0000 7f8cfa7fc700 1 --2- 172.21.15.111:0/2801909361 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8cfc0cf570 0x7f8cfc1f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:20.594 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.593+0000 7f8cfb7fe700 1 -- 172.21.15.111:0/2801909361 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8cfc0cf570 msgr2=0x7f8cfc1f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:20.594 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.593+0000 7f8cfb7fe700 1 --2- 172.21.15.111:0/2801909361 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8cfc0cf570 0x7f8cfc1f4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:20.595 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.593+0000 7f8cfb7fe700 1 -- 172.21.15.111:0/2801909361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cfc0c96d0 msgr2=0x7f8cfc14a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:20.595 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.593+0000 7f8cfb7fe700 1 --2- 172.21.15.111:0/2801909361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cfc0c96d0 0x7f8cfc14a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:20.595 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.593+0000 7f8cfb7fe700 1 -- 172.21.15.111:0/2801909361 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec009550 con 0x7f8cfc15e7b0 2023-03-29T10:57:20.595 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.593+0000 7f8cfaffd700 1 --2- 172.21.15.111:0/2801909361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cfc0c96d0 0x7f8cfc14a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:57:20.595 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.593+0000 7f8cfb7fe700 1 --2- 172.21.15.111:0/2801909361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cfc15e7b0 0x7f8cfc0b07d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4002230 tx=0x7f8cf400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:20.595 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.593+0000 7f8ce37fe700 1 -- 172.21.15.111:0/2801909361 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf400ac20 con 0x7f8cfc15e7b0 2023-03-29T10:57:20.595 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.593+0000 7f8ce37fe700 1 -- 172.21.15.111:0/2801909361 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf400ada0 con 0x7f8cfc15e7b0 2023-03-29T10:57:20.595 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.593+0000 7f8d017ff700 1 -- 172.21.15.111:0/2801909361 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cfc15ab50 con 0x7f8cfc15e7b0 2023-03-29T10:57:20.595 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.593+0000 7f8ce37fe700 1 -- 172.21.15.111:0/2801909361 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4007310 con 0x7f8cfc15e7b0 2023-03-29T10:57:20.595 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.593+0000 7f8d017ff700 1 -- 172.21.15.111:0/2801909361 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc15ee10 con 0x7f8cfc15e7b0 2023-03-29T10:57:20.595 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.594+0000 7f8d017ff700 1 -- 172.21.15.111:0/2801909361 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cfc15ae30 con 0x7f8cfc15e7b0 2023-03-29T10:57:20.596 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.595+0000 7f8ce37fe700 1 -- 172.21.15.111:0/2801909361 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f8cf4007470 con 0x7f8cfc15e7b0 2023-03-29T10:57:20.596 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.596+0000 7f8ce37fe700 1 --2- 172.21.15.111:0/2801909361 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8ce8069de0 0x7f8ce806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:20.596 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.596+0000 7f8ce37fe700 1 -- 172.21.15.111:0/2801909361 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8cf4087a50 con 0x7f8cfc15e7b0 2023-03-29T10:57:20.597 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.596+0000 7f8cfaffd700 1 --2- 172.21.15.111:0/2801909361 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8ce8069de0 0x7f8ce806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:20.597 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.597+0000 7f8cfaffd700 1 --2- 172.21.15.111:0/2801909361 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8ce8069de0 0x7f8ce806c290 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc14a490 tx=0x7f8cec008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:20.602 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.601+0000 7f8ce37fe700 1 -- 172.21.15.111:0/2801909361 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f8cf4053710 con 0x7f8cfc15e7b0 2023-03-29T10:57:20.794 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.794+0000 7f8d017ff700 1 -- 172.21.15.111:0/2801909361 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8cfc0743d0 con 0x7f8cfc15e7b0 2023-03-29T10:57:20.795 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.795+0000 7f8ce37fe700 1 -- 172.21.15.111:0/2801909361 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12526 (secure 0 0 0) 0x7f8cf4018020 con 0x7f8cfc15e7b0 2023-03-29T10:57:20.796 INFO:teuthology.misc.health.smithi111.stdout: 2023-03-29T10:57:20.796 INFO:teuthology.misc.health.smithi111.stdout:{"epoch":17,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:57:18.287456+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:57:20.798 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.797+0000 7f8d017ff700 1 -- 172.21.15.111:0/2801909361 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8ce8069de0 msgr2=0x7f8ce806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:20.798 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.797+0000 7f8d017ff700 1 --2- 172.21.15.111:0/2801909361 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8ce8069de0 0x7f8ce806c290 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc14a490 tx=0x7f8cec008030 comp rx=0 tx=0).stop 2023-03-29T10:57:20.798 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.797+0000 7f8d017ff700 1 -- 172.21.15.111:0/2801909361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cfc15e7b0 msgr2=0x7f8cfc0b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:20.798 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.797+0000 7f8d017ff700 1 --2- 172.21.15.111:0/2801909361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cfc15e7b0 0x7f8cfc0b07d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4002230 tx=0x7f8cf400a060 comp rx=0 tx=0).stop 2023-03-29T10:57:20.798 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.797+0000 7f8d017ff700 1 -- 172.21.15.111:0/2801909361 shutdown_connections 2023-03-29T10:57:20.798 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.797+0000 7f8d017ff700 1 --2- 172.21.15.111:0/2801909361 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8ce8069de0 0x7f8ce806c290 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:20.798 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.797+0000 7f8d017ff700 1 --2- 172.21.15.111:0/2801909361 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8cfc15e7b0 0x7f8cfc0b07d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:20.798 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.797+0000 7f8d017ff700 1 --2- 172.21.15.111:0/2801909361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cfc0c96d0 0x7f8cfc14a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:20.798 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.797+0000 7f8d017ff700 1 --2- 172.21.15.111:0/2801909361 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8cfc0cf570 0x7f8cfc1f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:20.799 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.797+0000 7f8d017ff700 1 -- 172.21.15.111:0/2801909361 >> 172.21.15.111:0/2801909361 conn(0x7f8cfc06a3c0 msgr2=0x7f8cfc0b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:20.799 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.797+0000 7f8d017ff700 1 -- 172.21.15.111:0/2801909361 shutdown_connections 2023-03-29T10:57:20.799 INFO:teuthology.misc.health.smithi111.stderr:2023-03-29T10:57:20.797+0000 7f8d017ff700 1 -- 172.21.15.111:0/2801909361 wait complete. 2023-03-29T10:57:20.812 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-03-29T10:57:20.813 INFO:tasks.ceph:Creating RBD pool 2023-03-29T10:57:20.813 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd pool create rbd 8 2023-03-29T10:57:20.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.990+0000 7fa9791ff700 1 Processor -- start 2023-03-29T10:57:20.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.990+0000 7fa9791ff700 1 -- start start 2023-03-29T10:57:20.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.991+0000 7fa9791ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9740cfbf0 0x7fa9741608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:20.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.991+0000 7fa9791ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa9740d1100 0x7fa9740d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:20.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.991+0000 7fa9791ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9740d0020 con 0x7fa9740b87a0 2023-03-29T10:57:20.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.991+0000 7fa9791ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9740d02d0 con 0x7fa9740d1100 2023-03-29T10:57:20.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.991+0000 7fa9791ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9740d0cb0 con 0x7fa9740cfbf0 2023-03-29T10:57:20.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.992+0000 7fa972d9d700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa9740b87a0 0x7fa9741601e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53554/0 (socket says 172.21.15.111:53554) 2023-03-29T10:57:20.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.992+0000 7fa972d9d700 1 -- 172.21.15.111:0/2231728931 learned_addr learned my addr 172.21.15.111:0/2231728931 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:20.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.992+0000 7fa97359e700 1 --2- 172.21.15.111:0/2231728931 >> v2:172.21.15.119:3300/0 conn(0x7fa9740d1100 0x7fa9740d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:20.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.992+0000 7fa97259c700 1 --2- 172.21.15.111:0/2231728931 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9740cfbf0 0x7fa9741608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:20.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.992+0000 7fa971d9b700 1 -- 172.21.15.111:0/2231728931 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2397805769 0 0) 0x7fa9740d0020 con 0x7fa9740b87a0 2023-03-29T10:57:20.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.992+0000 7fa971d9b700 1 -- 172.21.15.111:0/2231728931 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa95c0035b0 con 0x7fa9740b87a0 2023-03-29T10:57:20.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.992+0000 7fa971d9b700 1 -- 172.21.15.111:0/2231728931 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3496100699 0 0) 0x7fa95c0035b0 con 0x7fa9740b87a0 2023-03-29T10:57:20.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.992+0000 7fa97359e700 1 -- 172.21.15.111:0/2231728931 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9740cfbf0 msgr2=0x7fa9741608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:20.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.992+0000 7fa97359e700 1 --2- 172.21.15.111:0/2231728931 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9740cfbf0 0x7fa9741608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:20.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.992+0000 7fa97359e700 1 -- 172.21.15.111:0/2231728931 >> v1:172.21.15.111:6789/0 conn(0x7fa9740b87a0 legacy=0x7fa9741601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:20.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.992+0000 7fa97359e700 1 -- 172.21.15.111:0/2231728931 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9740cdc60 con 0x7fa9740d1100 2023-03-29T10:57:20.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.993+0000 7fa97359e700 1 --2- 172.21.15.111:0/2231728931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9740d1100 0x7fa9740d1580 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009d90 tx=0x7fa96c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aae40eb21980611 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:20.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.993+0000 7fa971d9b700 1 -- 172.21.15.111:0/2231728931 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c00e070 con 0x7fa9740d1100 2023-03-29T10:57:20.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.993+0000 7fa971d9b700 1 -- 172.21.15.111:0/2231728931 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96c003a20 con 0x7fa9740d1100 2023-03-29T10:57:20.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.993+0000 7fa971d9b700 1 -- 172.21.15.111:0/2231728931 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964001670 con 0x7fa9740d1100 2023-03-29T10:57:20.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.993+0000 7fa9791ff700 1 -- 172.21.15.111:0/2231728931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9740d1100 msgr2=0x7fa9740d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:20.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.993+0000 7fa9791ff700 1 --2- 172.21.15.111:0/2231728931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9740d1100 0x7fa9740d1580 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa96c009d90 tx=0x7fa96c00b330 comp rx=0 tx=0).stop 2023-03-29T10:57:20.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.993+0000 7fa9791ff700 1 -- 172.21.15.111:0/2231728931 shutdown_connections 2023-03-29T10:57:20.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.993+0000 7fa9791ff700 1 --2- 172.21.15.111:0/2231728931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9740d1100 0x7fa9740d1580 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:20.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.993+0000 7fa9791ff700 1 --2- 172.21.15.111:0/2231728931 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9740cfbf0 0x7fa9741608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:20.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.993+0000 7fa9791ff700 1 -- 172.21.15.111:0/2231728931 >> 172.21.15.111:0/2231728931 conn(0x7fa974071b10 msgr2=0x7fa974071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:20.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.993+0000 7fa9791ff700 1 -- 172.21.15.111:0/2231728931 shutdown_connections 2023-03-29T10:57:20.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.994+0000 7fa9791ff700 1 -- 172.21.15.111:0/2231728931 wait complete. 2023-03-29T10:57:20.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.995+0000 7fa9791ff700 1 Processor -- start 2023-03-29T10:57:20.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.995+0000 7fa9791ff700 1 -- start start 2023-03-29T10:57:20.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.996+0000 7fa9791ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9740b87a0 0x7fa9740c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:20.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.996+0000 7fa9791ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9740cfbf0 0x7fa9741f17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:20.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.996+0000 7fa9791ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9740d1100 0x7fa974151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:20.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.996+0000 7fa9791ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97415fc10 con 0x7fa9740cfbf0 2023-03-29T10:57:20.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.996+0000 7fa9791ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9740caf20 con 0x7fa9740b87a0 2023-03-29T10:57:20.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.996+0000 7fa9791ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97415cea0 con 0x7fa9740d1100 2023-03-29T10:57:20.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.997+0000 7fa97259c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9740cfbf0 0x7fa9741f17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:20.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.997+0000 7fa97259c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9740cfbf0 0x7fa9741f17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50694/0 (socket says 172.21.15.111:50694) 2023-03-29T10:57:20.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.997+0000 7fa97259c700 1 -- 172.21.15.111:0/964266194 learned_addr learned my addr 172.21.15.111:0/964266194 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:20.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.997+0000 7fa972d9d700 1 --2- 172.21.15.111:0/964266194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9740b87a0 0x7fa9740c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:20.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.997+0000 7fa97359e700 1 --2- 172.21.15.111:0/964266194 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9740d1100 0x7fa974151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:20.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.997+0000 7fa97259c700 1 -- 172.21.15.111:0/964266194 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9740d1100 msgr2=0x7fa974151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:20.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.997+0000 7fa97259c700 1 --2- 172.21.15.111:0/964266194 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9740d1100 0x7fa974151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:20.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.997+0000 7fa97259c700 1 -- 172.21.15.111:0/964266194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9740b87a0 msgr2=0x7fa9740c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:20.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.997+0000 7fa97259c700 1 --2- 172.21.15.111:0/964266194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9740b87a0 0x7fa9740c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:20.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.997+0000 7fa97259c700 1 -- 172.21.15.111:0/964266194 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96c009a70 con 0x7fa9740cfbf0 2023-03-29T10:57:21.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.997+0000 7fa97259c700 1 --2- 172.21.15.111:0/964266194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9740cfbf0 0x7fa9741f17e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa96800e9e0 tx=0x7fa96800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:21.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.997+0000 7fa95b7fe700 1 -- 172.21.15.111:0/964266194 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa968009710 con 0x7fa9740cfbf0 2023-03-29T10:57:21.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.998+0000 7fa95b7fe700 1 -- 172.21.15.111:0/964266194 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96800ee80 con 0x7fa9740cfbf0 2023-03-29T10:57:21.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.998+0000 7fa95b7fe700 1 -- 172.21.15.111:0/964266194 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96800ca50 con 0x7fa9740cfbf0 2023-03-29T10:57:21.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.998+0000 7fa9791ff700 1 -- 172.21.15.111:0/964266194 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9740cd980 con 0x7fa9740cfbf0 2023-03-29T10:57:21.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.998+0000 7fa9791ff700 1 -- 172.21.15.111:0/964266194 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9740cd6a0 con 0x7fa9740cfbf0 2023-03-29T10:57:21.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:20.999+0000 7fa9791ff700 1 -- 172.21.15.111:0/964266194 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa974160e30 con 0x7fa9740cfbf0 2023-03-29T10:57:21.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.000+0000 7fa95b7fe700 1 -- 172.21.15.111:0/964266194 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fa96800cbb0 con 0x7fa9740cfbf0 2023-03-29T10:57:21.004 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.001+0000 7fa95b7fe700 1 --2- 172.21.15.111:0/964266194 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa960069de0 0x7fa96006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:21.004 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.001+0000 7fa95b7fe700 1 -- 172.21.15.111:0/964266194 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa968087b70 con 0x7fa9740cfbf0 2023-03-29T10:57:21.004 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.001+0000 7fa972d9d700 1 --2- 172.21.15.111:0/964266194 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa960069de0 0x7fa96006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:21.004 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.001+0000 7fa972d9d700 1 --2- 172.21.15.111:0/964266194 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa960069de0 0x7fa96006c290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa9640011e0 tx=0x7fa964001da0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:21.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.004+0000 7fa95b7fe700 1 -- 172.21.15.111:0/964266194 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fa968057260 con 0x7fa9740cfbf0 2023-03-29T10:57:21.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.201+0000 7fa9791ff700 1 -- 172.21.15.111:0/964266194 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fa97415cfe0 con 0x7fa9740cfbf0 2023-03-29T10:57:21.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.306+0000 7fa95b7fe700 1 -- 172.21.15.111:0/964266194 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v18) v1 ==== 109+0+0 (secure 0 0 0) 0x7fa97415cfe0 con 0x7fa9740cfbf0 2023-03-29T10:57:21.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.309+0000 7fa9791ff700 1 -- 172.21.15.111:0/964266194 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa960069de0 msgr2=0x7fa96006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:21.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.309+0000 7fa9791ff700 1 --2- 172.21.15.111:0/964266194 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa960069de0 0x7fa96006c290 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa9640011e0 tx=0x7fa964001da0 comp rx=0 tx=0).stop 2023-03-29T10:57:21.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.309+0000 7fa9791ff700 1 -- 172.21.15.111:0/964266194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9740cfbf0 msgr2=0x7fa9741f17e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:21.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.309+0000 7fa9791ff700 1 --2- 172.21.15.111:0/964266194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9740cfbf0 0x7fa9741f17e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa96800e9e0 tx=0x7fa96800ecf0 comp rx=0 tx=0).stop 2023-03-29T10:57:21.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.310+0000 7fa9791ff700 1 -- 172.21.15.111:0/964266194 shutdown_connections 2023-03-29T10:57:21.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.310+0000 7fa9791ff700 1 --2- 172.21.15.111:0/964266194 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa960069de0 0x7fa96006c290 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:21.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.310+0000 7fa9791ff700 1 --2- 172.21.15.111:0/964266194 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa9740d1100 0x7fa974151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:21.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.310+0000 7fa9791ff700 1 --2- 172.21.15.111:0/964266194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9740b87a0 0x7fa9740c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:21.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.310+0000 7fa9791ff700 1 --2- 172.21.15.111:0/964266194 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa9740cfbf0 0x7fa9741f17e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:21.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.310+0000 7fa9791ff700 1 -- 172.21.15.111:0/964266194 >> 172.21.15.111:0/964266194 conn(0x7fa974071b10 msgr2=0x7fa9740cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:21.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.310+0000 7fa9791ff700 1 -- 172.21.15.111:0/964266194 shutdown_connections 2023-03-29T10:57:21.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.310+0000 7fa9791ff700 1 -- 172.21.15.111:0/964266194 wait complete. 2023-03-29T10:57:21.312 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'rbd' created 2023-03-29T10:57:21.328 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-03-29T10:57:21.381 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T10:57:21.384 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T10:57:21.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.513+0000 7f0d091ff700 1 Processor -- start 2023-03-29T10:57:21.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.513+0000 7f0d091ff700 1 -- start start 2023-03-29T10:57:21.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.514+0000 7f0d091ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0d040b87a0 0x7f0d040cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:21.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.514+0000 7f0d091ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d040cdc60 con 0x7f0d041601e0 2023-03-29T10:57:21.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.514+0000 7f0d091ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d040cd6a0 con 0x7f0d04160d40 2023-03-29T10:57:21.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.514+0000 7f0d091ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d040d02d0 con 0x7f0d040b87a0 2023-03-29T10:57:21.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.514+0000 7f0d02d9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0d040b87a0 0x7f0d040cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:21.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.515+0000 7f0d02d9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0d040b87a0 0x7f0d040cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50708/0 (socket says 172.21.15.111:50708) 2023-03-29T10:57:21.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.515+0000 7f0d02d9d700 1 -- 172.21.15.111:0/2998540414 learned_addr learned my addr 172.21.15.111:0/2998540414 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:21.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.515+0000 7f0d0259c700 1 --1- 172.21.15.111:0/2998540414 >> v1:172.21.15.111:6789/0 conn(0x7f0d041601e0 0x7f0d04160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53560/0 (socket says 172.21.15.111:53560) 2023-03-29T10:57:21.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.515+0000 7f0d02d9d700 1 -- 172.21.15.111:0/2998540414 >> v1:172.21.15.119:6789/0 conn(0x7f0d04160d40 legacy=0x7f0d040d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:21.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.515+0000 7f0d02d9d700 1 -- 172.21.15.111:0/2998540414 >> v1:172.21.15.111:6789/0 conn(0x7f0d041601e0 legacy=0x7f0d04160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:21.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.515+0000 7f0d02d9d700 1 -- 172.21.15.111:0/2998540414 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d040cd980 con 0x7f0d040b87a0 2023-03-29T10:57:21.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.515+0000 7f0d01d9b700 1 -- 172.21.15.111:0/2998540414 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999364064 0 0) 0x7f0d040cdc60 con 0x7f0d041601e0 2023-03-29T10:57:21.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.515+0000 7f0d02d9d700 1 --2- 172.21.15.111:0/2998540414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d040b87a0 0x7f0d040cfbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40098f0 tx=0x7f0cf4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ac095b0a859672 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:21.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.515+0000 7f0d01d9b700 1 -- 172.21.15.111:0/2998540414 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf400b8a0 con 0x7f0d040b87a0 2023-03-29T10:57:21.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.515+0000 7f0d01d9b700 1 -- 172.21.15.111:0/2998540414 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf400ba00 con 0x7f0d040b87a0 2023-03-29T10:57:21.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.515+0000 7f0d01d9b700 1 -- 172.21.15.111:0/2998540414 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf400bd90 con 0x7f0d040b87a0 2023-03-29T10:57:21.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.516+0000 7f0d091ff700 1 -- 172.21.15.111:0/2998540414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d040b87a0 msgr2=0x7f0d040cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:21.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.516+0000 7f0d091ff700 1 --2- 172.21.15.111:0/2998540414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d040b87a0 0x7f0d040cfbf0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40098f0 tx=0x7f0cf4009c00 comp rx=0 tx=0).stop 2023-03-29T10:57:21.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.516+0000 7f0d091ff700 1 -- 172.21.15.111:0/2998540414 shutdown_connections 2023-03-29T10:57:21.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.516+0000 7f0d091ff700 1 --2- 172.21.15.111:0/2998540414 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d040b87a0 0x7f0d040cfbf0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:21.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.516+0000 7f0d091ff700 1 -- 172.21.15.111:0/2998540414 >> 172.21.15.111:0/2998540414 conn(0x7f0d04071b10 msgr2=0x7f0d04071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:21.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.516+0000 7f0d091ff700 1 -- 172.21.15.111:0/2998540414 shutdown_connections 2023-03-29T10:57:21.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.516+0000 7f0d091ff700 1 -- 172.21.15.111:0/2998540414 wait complete. 2023-03-29T10:57:21.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.518+0000 7f0d091ff700 1 Processor -- start 2023-03-29T10:57:21.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.519+0000 7f0d091ff700 1 -- start start 2023-03-29T10:57:21.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.520+0000 7f0d091ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d040b87a0 0x7f0d04159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:21.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.520+0000 7f0d091ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d041601e0 0x7f0d040ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:21.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.520+0000 7f0d091ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0d04160d40 0x7f0d0415ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:21.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.520+0000 7f0d091ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d040cb7f0 con 0x7f0d041601e0 2023-03-29T10:57:21.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.520+0000 7f0d091ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0415a300 con 0x7f0d040b87a0 2023-03-29T10:57:21.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.520+0000 7f0d091ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d040cabe0 con 0x7f0d04160d40 2023-03-29T10:57:21.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.520+0000 7f0d0259c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d041601e0 0x7f0d040ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:21.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.520+0000 7f0d0259c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d041601e0 0x7f0d040ca190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50714/0 (socket says 172.21.15.111:50714) 2023-03-29T10:57:21.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.520+0000 7f0d0259c700 1 -- 172.21.15.111:0/743160494 learned_addr learned my addr 172.21.15.111:0/743160494 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:21.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.520+0000 7f0d02d9d700 1 --2- 172.21.15.111:0/743160494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d040b87a0 0x7f0d04159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:21.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.520+0000 7f0d0359e700 1 --2- 172.21.15.111:0/743160494 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0d04160d40 0x7f0d0415ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:21.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.521+0000 7f0d0259c700 1 -- 172.21.15.111:0/743160494 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0d04160d40 msgr2=0x7f0d0415ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:21.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.521+0000 7f0d0259c700 1 --2- 172.21.15.111:0/743160494 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0d04160d40 0x7f0d0415ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:21.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.521+0000 7f0d0259c700 1 -- 172.21.15.111:0/743160494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d040b87a0 msgr2=0x7f0d04159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:21.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.521+0000 7f0d0259c700 1 --2- 172.21.15.111:0/743160494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d040b87a0 0x7f0d04159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:21.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.521+0000 7f0d0259c700 1 -- 172.21.15.111:0/743160494 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf4009550 con 0x7f0d041601e0 2023-03-29T10:57:21.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.521+0000 7f0d0359e700 1 --2- 172.21.15.111:0/743160494 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0d04160d40 0x7f0d0415ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:57:21.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.521+0000 7f0d0259c700 1 --2- 172.21.15.111:0/743160494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d041601e0 0x7f0d040ca190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800c520 tx=0x7f0cf800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:21.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.521+0000 7f0ceb7fe700 1 -- 172.21.15.111:0/743160494 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8004c80 con 0x7f0d041601e0 2023-03-29T10:57:21.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.521+0000 7f0ceb7fe700 1 -- 172.21.15.111:0/743160494 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf8004de0 con 0x7f0d041601e0 2023-03-29T10:57:21.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.521+0000 7f0d091ff700 1 -- 172.21.15.111:0/743160494 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d040d19a0 con 0x7f0d041601e0 2023-03-29T10:57:21.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.521+0000 7f0ceb7fe700 1 -- 172.21.15.111:0/743160494 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8007310 con 0x7f0d041601e0 2023-03-29T10:57:21.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.521+0000 7f0d091ff700 1 -- 172.21.15.111:0/743160494 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d040d1810 con 0x7f0d041601e0 2023-03-29T10:57:21.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.522+0000 7f0d091ff700 1 -- 172.21.15.111:0/743160494 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d040d1b30 con 0x7f0d041601e0 2023-03-29T10:57:21.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.523+0000 7f0ceb7fe700 1 -- 172.21.15.111:0/743160494 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f0cf8007470 con 0x7f0d041601e0 2023-03-29T10:57:21.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.524+0000 7f0ceb7fe700 1 --2- 172.21.15.111:0/743160494 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0cf0069de0 0x7f0cf006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:21.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.524+0000 7f0ceb7fe700 1 -- 172.21.15.111:0/743160494 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f0cf8087920 con 0x7f0d041601e0 2023-03-29T10:57:21.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.524+0000 7f0d02d9d700 1 --2- 172.21.15.111:0/743160494 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0cf0069de0 0x7f0cf006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:21.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.525+0000 7f0d02d9d700 1 --2- 172.21.15.111:0/743160494 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0cf0069de0 0x7f0cf006c290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4000c00 tx=0x7f0cf4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:21.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.529+0000 7f0ceb7fe700 1 -- 172.21.15.111:0/743160494 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f0cf8053530 con 0x7f0d041601e0 2023-03-29T10:57:21.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:21.724+0000 7f0d091ff700 1 -- 172.21.15.111:0/743160494 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0d040cb930 con 0x7f0d041601e0 2023-03-29T10:57:22.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.308+0000 7f0ceb7fe700 1 -- 172.21.15.111:0/743160494 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v19) v1 ==== 173+0+0 (secure 0 0 0) 0x7f0cf8056b50 con 0x7f0d041601e0 2023-03-29T10:57:22.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.312+0000 7f0d091ff700 1 -- 172.21.15.111:0/743160494 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0cf0069de0 msgr2=0x7f0cf006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:22.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.312+0000 7f0d091ff700 1 --2- 172.21.15.111:0/743160494 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0cf0069de0 0x7f0cf006c290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4000c00 tx=0x7f0cf4008030 comp rx=0 tx=0).stop 2023-03-29T10:57:22.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.312+0000 7f0d091ff700 1 -- 172.21.15.111:0/743160494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d041601e0 msgr2=0x7f0d040ca190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:22.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.312+0000 7f0d091ff700 1 --2- 172.21.15.111:0/743160494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d041601e0 0x7f0d040ca190 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0cf800c520 tx=0x7f0cf800c830 comp rx=0 tx=0).stop 2023-03-29T10:57:22.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.313+0000 7f0d091ff700 1 -- 172.21.15.111:0/743160494 shutdown_connections 2023-03-29T10:57:22.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.313+0000 7f0d091ff700 1 --2- 172.21.15.111:0/743160494 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0cf0069de0 0x7f0cf006c290 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:22.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.313+0000 7f0d091ff700 1 --2- 172.21.15.111:0/743160494 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0d04160d40 0x7f0d0415ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:22.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.313+0000 7f0d091ff700 1 --2- 172.21.15.111:0/743160494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d040b87a0 0x7f0d04159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:22.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.313+0000 7f0d091ff700 1 --2- 172.21.15.111:0/743160494 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0d041601e0 0x7f0d040ca190 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:22.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.313+0000 7f0d091ff700 1 -- 172.21.15.111:0/743160494 >> 172.21.15.111:0/743160494 conn(0x7f0d04071b10 msgr2=0x7f0d040d1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:22.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.313+0000 7f0d091ff700 1 -- 172.21.15.111:0/743160494 shutdown_connections 2023-03-29T10:57:22.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.313+0000 7f0d091ff700 1 -- 172.21.15.111:0/743160494 wait complete. 2023-03-29T10:57:22.319 INFO:teuthology.orchestra.run.smithi111.stderr:enabled application 'rbd' on pool 'rbd' 2023-03-29T10:57:22.337 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-03-29T10:57:22.338 INFO:tasks.ceph.mds.a:Restarting daemon 2023-03-29T10:57:22.338 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-03-29T10:57:22.381 INFO:tasks.ceph.mds.a:Started 2023-03-29T10:57:22.381 INFO:tasks.ceph.mds.c:Restarting daemon 2023-03-29T10:57:22.382 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-03-29T10:57:22.385 INFO:tasks.ceph.mds.c:Started 2023-03-29T10:57:22.385 INFO:tasks.ceph.mds.b:Restarting daemon 2023-03-29T10:57:22.385 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-03-29T10:57:22.389 INFO:tasks.ceph.mds.b:Started 2023-03-29T10:57:22.389 INFO:tasks.ceph.mds.d:Restarting daemon 2023-03-29T10:57:22.389 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-03-29T10:57:22.392 INFO:tasks.ceph.mds.d:Started 2023-03-29T10:57:22.392 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-03-29T10:57:22.393 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:57:22.525 INFO:tasks.ceph.mds.a.smithi111.stdout:starting mds.a at 2023-03-29T10:57:22.530 INFO:tasks.ceph.mds.b.smithi119.stdout:starting mds.b at 2023-03-29T10:57:22.530 INFO:tasks.ceph.mds.d.smithi119.stdout:starting mds.d at 2023-03-29T10:57:22.531 INFO:tasks.ceph.mds.c.smithi111.stdout:starting mds.c at 2023-03-29T10:57:22.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.593+0000 7f4ad07ff700 1 Processor -- start 2023-03-29T10:57:22.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.593+0000 7f4ad07ff700 1 -- start start 2023-03-29T10:57:22.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.594+0000 7f4ad07ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4ac80cf970 0x7f4ac80cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:22.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.594+0000 7f4ad07ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ac80b70f0 0x7f4ac80b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:22.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.594+0000 7f4ad07ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac80cdc10 con 0x7f4ac80cfd40 2023-03-29T10:57:22.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.594+0000 7f4ad07ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac80d08d0 con 0x7f4ac80cf970 2023-03-29T10:57:22.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.594+0000 7f4ad07ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac80b6c60 con 0x7f4ac80b70f0 2023-03-29T10:57:22.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.595+0000 7f4ace59b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4ac80cf970 0x7f4ac80cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:22.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.595+0000 7f4acdd9a700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f4ac80cfd40 0x7f4ac80d01c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53576/0 (socket says 172.21.15.111:53576) 2023-03-29T10:57:22.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.595+0000 7f4acdd9a700 1 -- 172.21.15.111:0/3264126331 learned_addr learned my addr 172.21.15.111:0/3264126331 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:22.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.595+0000 7f4aced9c700 1 --2- 172.21.15.111:0/3264126331 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ac80b70f0 0x7f4ac80b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:22.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.595+0000 7f4ace59b700 1 -- 172.21.15.111:0/3264126331 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ac80b70f0 msgr2=0x7f4ac80b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:22.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.595+0000 7f4ace59b700 1 --2- 172.21.15.111:0/3264126331 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ac80b70f0 0x7f4ac80b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:22.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.595+0000 7f4ace59b700 1 -- 172.21.15.111:0/3264126331 >> v1:172.21.15.111:6789/0 conn(0x7f4ac80cfd40 legacy=0x7f4ac80d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:22.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.595+0000 7f4ace59b700 1 -- 172.21.15.111:0/3264126331 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac80cd650 con 0x7f4ac80cf970 2023-03-29T10:57:22.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.595+0000 7f4acd599700 1 -- 172.21.15.111:0/3264126331 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71813591 0 0) 0x7f4ac80cdc10 con 0x7f4ac80cfd40 2023-03-29T10:57:22.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.595+0000 7f4ace59b700 1 --2- 172.21.15.111:0/3264126331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac80cf970 0x7f4ac80cf110 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009840 tx=0x7f4ac4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7d235265422b78 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:22.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.596+0000 7f4acd599700 1 -- 172.21.15.111:0/3264126331 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4011830 con 0x7f4ac80cf970 2023-03-29T10:57:22.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.596+0000 7f4acd599700 1 -- 172.21.15.111:0/3264126331 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac4011990 con 0x7f4ac80cf970 2023-03-29T10:57:22.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.596+0000 7f4acd599700 1 -- 172.21.15.111:0/3264126331 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4011d20 con 0x7f4ac80cf970 2023-03-29T10:57:22.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.596+0000 7f4ad07ff700 1 -- 172.21.15.111:0/3264126331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac80cf970 msgr2=0x7f4ac80cf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:22.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.596+0000 7f4ad07ff700 1 --2- 172.21.15.111:0/3264126331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac80cf970 0x7f4ac80cf110 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009840 tx=0x7f4ac4009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:22.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.596+0000 7f4ad07ff700 1 -- 172.21.15.111:0/3264126331 shutdown_connections 2023-03-29T10:57:22.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.596+0000 7f4ad07ff700 1 --2- 172.21.15.111:0/3264126331 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ac80b70f0 0x7f4ac80b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:22.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.596+0000 7f4ad07ff700 1 --2- 172.21.15.111:0/3264126331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac80cf970 0x7f4ac80cf110 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:22.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.596+0000 7f4ad07ff700 1 -- 172.21.15.111:0/3264126331 >> 172.21.15.111:0/3264126331 conn(0x7f4ac8071b10 msgr2=0x7f4ac8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:22.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.596+0000 7f4ad07ff700 1 -- 172.21.15.111:0/3264126331 shutdown_connections 2023-03-29T10:57:22.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.596+0000 7f4ad07ff700 1 -- 172.21.15.111:0/3264126331 wait complete. 2023-03-29T10:57:22.599 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.598+0000 7f4ad07ff700 1 Processor -- start 2023-03-29T10:57:22.599 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.598+0000 7f4ad07ff700 1 -- start start 2023-03-29T10:57:22.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.599+0000 7f4ad07ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ac80b70f0 0x7f4ac81e7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:22.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.599+0000 7f4ad07ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac80cf970 0x7f4ac815d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:22.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.599+0000 7f4ad07ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ac80cfd40 0x7f4ac81547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:22.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.599+0000 7f4ad07ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac8154d00 con 0x7f4ac80cf970 2023-03-29T10:57:22.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.599+0000 7f4ad07ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac81e8530 con 0x7f4ac80cfd40 2023-03-29T10:57:22.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.599+0000 7f4ad07ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac81e7f70 con 0x7f4ac80b70f0 2023-03-29T10:57:22.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.599+0000 7f4acdd9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac80cf970 0x7f4ac815d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:22.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.599+0000 7f4acdd9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac80cf970 0x7f4ac815d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50766/0 (socket says 172.21.15.111:50766) 2023-03-29T10:57:22.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.599+0000 7f4acdd9a700 1 -- 172.21.15.111:0/2095308598 learned_addr learned my addr 172.21.15.111:0/2095308598 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:22.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.600+0000 7f4ace59b700 1 --2- 172.21.15.111:0/2095308598 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ac80b70f0 0x7f4ac81e7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:22.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.600+0000 7f4aced9c700 1 --2- 172.21.15.111:0/2095308598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ac80cfd40 0x7f4ac81547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:22.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.600+0000 7f4acdd9a700 1 -- 172.21.15.111:0/2095308598 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ac80b70f0 msgr2=0x7f4ac81e7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:22.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.600+0000 7f4acdd9a700 1 --2- 172.21.15.111:0/2095308598 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ac80b70f0 0x7f4ac81e7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:22.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.600+0000 7f4acdd9a700 1 -- 172.21.15.111:0/2095308598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ac80cfd40 msgr2=0x7f4ac81547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:22.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.600+0000 7f4acdd9a700 1 --2- 172.21.15.111:0/2095308598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ac80cfd40 0x7f4ac81547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:22.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.600+0000 7f4acdd9a700 1 -- 172.21.15.111:0/2095308598 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac40094a0 con 0x7f4ac80cf970 2023-03-29T10:57:22.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.600+0000 7f4ace59b700 1 --2- 172.21.15.111:0/2095308598 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ac80b70f0 0x7f4ac81e7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:22.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.600+0000 7f4acdd9a700 1 --2- 172.21.15.111:0/2095308598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac80cf970 0x7f4ac815d6c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800c550 tx=0x7f4ab800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:22.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.600+0000 7f4abeffd700 1 -- 172.21.15.111:0/2095308598 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8004c80 con 0x7f4ac80cf970 2023-03-29T10:57:22.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.600+0000 7f4abeffd700 1 -- 172.21.15.111:0/2095308598 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab8004e00 con 0x7f4ac80cf970 2023-03-29T10:57:22.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.600+0000 7f4abeffd700 1 -- 172.21.15.111:0/2095308598 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8007310 con 0x7f4ac80cf970 2023-03-29T10:57:22.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.600+0000 7f4ad07ff700 1 -- 172.21.15.111:0/2095308598 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac80b68f0 con 0x7f4ac80cf970 2023-03-29T10:57:22.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.600+0000 7f4ad07ff700 1 -- 172.21.15.111:0/2095308598 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac80cd930 con 0x7f4ac80cf970 2023-03-29T10:57:22.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.601+0000 7f4ad07ff700 1 -- 172.21.15.111:0/2095308598 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac80cf650 con 0x7f4ac80cf970 2023-03-29T10:57:22.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.602+0000 7f4abeffd700 1 -- 172.21.15.111:0/2095308598 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f4ab8007470 con 0x7f4ac80cf970 2023-03-29T10:57:22.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.603+0000 7f4abeffd700 1 --2- 172.21.15.111:0/2095308598 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4aac069de0 0x7f4aac06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:22.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.603+0000 7f4abeffd700 1 -- 172.21.15.111:0/2095308598 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f4ab80879f0 con 0x7f4ac80cf970 2023-03-29T10:57:22.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.603+0000 7f4ace59b700 1 --2- 172.21.15.111:0/2095308598 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4aac069de0 0x7f4aac06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:22.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.604+0000 7f4ace59b700 1 --2- 172.21.15.111:0/2095308598 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4aac069de0 0x7f4aac06c290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4000c00 tx=0x7f4ac4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:22.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.607+0000 7f4abeffd700 1 -- 172.21.15.111:0/2095308598 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f4ab8053550 con 0x7f4ac80cf970 2023-03-29T10:57:22.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.803+0000 7f4ad07ff700 1 -- 172.21.15.111:0/2095308598 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ac8074430 con 0x7f4ac80cf970 2023-03-29T10:57:22.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.804+0000 7f4abeffd700 1 -- 172.21.15.111:0/2095308598 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14325 (secure 0 0 0) 0x7f4ab805c020 con 0x7f4ac80cf970 2023-03-29T10:57:22.805 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:22.805 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":19,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:57:22.304777+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:57:22.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.806+0000 7f4ad07ff700 1 -- 172.21.15.111:0/2095308598 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4aac069de0 msgr2=0x7f4aac06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:22.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.806+0000 7f4ad07ff700 1 --2- 172.21.15.111:0/2095308598 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4aac069de0 0x7f4aac06c290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4000c00 tx=0x7f4ac4008030 comp rx=0 tx=0).stop 2023-03-29T10:57:22.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.806+0000 7f4ad07ff700 1 -- 172.21.15.111:0/2095308598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac80cf970 msgr2=0x7f4ac815d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:22.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.806+0000 7f4ad07ff700 1 --2- 172.21.15.111:0/2095308598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac80cf970 0x7f4ac815d6c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800c550 tx=0x7f4ab800c910 comp rx=0 tx=0).stop 2023-03-29T10:57:22.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.806+0000 7f4ad07ff700 1 -- 172.21.15.111:0/2095308598 shutdown_connections 2023-03-29T10:57:22.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.806+0000 7f4ad07ff700 1 --2- 172.21.15.111:0/2095308598 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4aac069de0 0x7f4aac06c290 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:22.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.806+0000 7f4ad07ff700 1 --2- 172.21.15.111:0/2095308598 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ac80cfd40 0x7f4ac81547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:22.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.806+0000 7f4ad07ff700 1 --2- 172.21.15.111:0/2095308598 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4ac80b70f0 0x7f4ac81e7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:22.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.806+0000 7f4ad07ff700 1 --2- 172.21.15.111:0/2095308598 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4ac80cf970 0x7f4ac815d6c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:22.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.806+0000 7f4ad07ff700 1 -- 172.21.15.111:0/2095308598 >> 172.21.15.111:0/2095308598 conn(0x7f4ac8071b10 msgr2=0x7f4ac80d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:22.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.807+0000 7f4ad07ff700 1 -- 172.21.15.111:0/2095308598 shutdown_connections 2023-03-29T10:57:22.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:22.807+0000 7f4ad07ff700 1 -- 172.21.15.111:0/2095308598 wait complete. 2023-03-29T10:57:22.824 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-29T10:57:16.183979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-29T10:57:21.239809+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-03-29T10:57:22.825 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-29T10:57:23.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.010+0000 7f7abd3ff700 1 Processor -- start 2023-03-29T10:57:23.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.010+0000 7f7abd3ff700 1 -- start start 2023-03-29T10:57:23.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.011+0000 7f7abd3ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ab80b87a0 0x7f7ab80cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:23.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.011+0000 7f7abd3ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7ab81601e0 0x7f7ab8160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:23.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.011+0000 7f7abd3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab80cdc60 con 0x7f7ab8160b70 2023-03-29T10:57:23.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.011+0000 7f7abd3ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab80d02d0 con 0x7f7ab81601e0 2023-03-29T10:57:23.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.011+0000 7f7abd3ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab80d0cb0 con 0x7f7ab80b87a0 2023-03-29T10:57:23.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.011+0000 7f7ab67fc700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7ab81601e0 0x7f7ab8160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:23.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.011+0000 7f7ab67fc700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f7ab81601e0 0x7f7ab8160630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50782/0 (socket says 172.21.15.111:50782) 2023-03-29T10:57:23.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.011+0000 7f7ab67fc700 1 -- 172.21.15.111:0/1937226686 learned_addr learned my addr 172.21.15.111:0/1937226686 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:23.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.011+0000 7f7ab6ffd700 1 --2- 172.21.15.111:0/1937226686 >> v2:172.21.15.119:3300/0 conn(0x7f7ab80b87a0 0x7f7ab80cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:23.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.012+0000 7f7ab67fc700 1 -- 172.21.15.111:0/1937226686 >> v2:172.21.15.119:3300/0 conn(0x7f7ab80b87a0 msgr2=0x7f7ab80cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:23.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.012+0000 7f7ab67fc700 1 --2- 172.21.15.111:0/1937226686 >> v2:172.21.15.119:3300/0 conn(0x7f7ab80b87a0 0x7f7ab80cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:23.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.012+0000 7f7ab5ffb700 1 -- 172.21.15.111:0/1937226686 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675660352 0 0) 0x7f7ab80cdc60 con 0x7f7ab8160b70 2023-03-29T10:57:23.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.012+0000 7f7ab67fc700 1 -- 172.21.15.111:0/1937226686 >> v1:172.21.15.119:6789/0 conn(0x7f7ab8160b70 legacy=0x7f7ab80d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:23.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.012+0000 7f7ab67fc700 1 -- 172.21.15.111:0/1937226686 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab80cd6a0 con 0x7f7ab81601e0 2023-03-29T10:57:23.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.012+0000 7f7ab67fc700 1 --2- 172.21.15.111:0/1937226686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ab81601e0 0x7f7ab8160630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80b4ac0 tx=0x7f7aac009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28460a157b689753 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:23.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.012+0000 7f7ab5ffb700 1 -- 172.21.15.111:0/1937226686 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac011830 con 0x7f7ab81601e0 2023-03-29T10:57:23.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.012+0000 7f7ab5ffb700 1 -- 172.21.15.111:0/1937226686 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aac011990 con 0x7f7ab81601e0 2023-03-29T10:57:23.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.012+0000 7f7ab5ffb700 1 -- 172.21.15.111:0/1937226686 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac011d20 con 0x7f7ab81601e0 2023-03-29T10:57:23.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.013+0000 7f7abd3ff700 1 -- 172.21.15.111:0/1937226686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ab81601e0 msgr2=0x7f7ab8160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:23.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.013+0000 7f7abd3ff700 1 --2- 172.21.15.111:0/1937226686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ab81601e0 0x7f7ab8160630 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7ab80b4ac0 tx=0x7f7aac009c00 comp rx=0 tx=0).stop 2023-03-29T10:57:23.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.013+0000 7f7abd3ff700 1 -- 172.21.15.111:0/1937226686 shutdown_connections 2023-03-29T10:57:23.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.013+0000 7f7abd3ff700 1 --2- 172.21.15.111:0/1937226686 >> v2:172.21.15.119:3300/0 conn(0x7f7ab80b87a0 0x7f7ab80cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:23.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.013+0000 7f7abd3ff700 1 --2- 172.21.15.111:0/1937226686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ab81601e0 0x7f7ab8160630 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:23.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.013+0000 7f7abd3ff700 1 -- 172.21.15.111:0/1937226686 >> 172.21.15.111:0/1937226686 conn(0x7f7ab8071b10 msgr2=0x7f7ab8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:23.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.013+0000 7f7abd3ff700 1 -- 172.21.15.111:0/1937226686 shutdown_connections 2023-03-29T10:57:23.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.013+0000 7f7abd3ff700 1 -- 172.21.15.111:0/1937226686 wait complete. 2023-03-29T10:57:23.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.015+0000 7f7abd3ff700 1 Processor -- start 2023-03-29T10:57:23.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.015+0000 7f7abd3ff700 1 -- start start 2023-03-29T10:57:23.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.016+0000 7f7abd3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab80b87a0 0x7f7ab815d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:23.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.016+0000 7f7abd3ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ab81601e0 0x7f7ab81f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:23.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.016+0000 7f7abd3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7ab8160b70 0x7f7ab80b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:23.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.016+0000 7f7abd3ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab815e280 con 0x7f7ab81601e0 2023-03-29T10:57:23.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.016+0000 7f7abd3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab815e560 con 0x7f7ab80b87a0 2023-03-29T10:57:23.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.016+0000 7f7abd3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab815e840 con 0x7f7ab8160b70 2023-03-29T10:57:23.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.017+0000 7f7ab67fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ab81601e0 0x7f7ab81f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:23.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.017+0000 7f7ab67fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ab81601e0 0x7f7ab81f1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50796/0 (socket says 172.21.15.111:50796) 2023-03-29T10:57:23.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.017+0000 7f7ab67fc700 1 -- 172.21.15.111:0/1420285461 learned_addr learned my addr 172.21.15.111:0/1420285461 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:23.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.017+0000 7f7ab6ffd700 1 --2- 172.21.15.111:0/1420285461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab80b87a0 0x7f7ab815d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:23.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.017+0000 7f7ab77fe700 1 --2- 172.21.15.111:0/1420285461 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7ab8160b70 0x7f7ab80b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:23.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.017+0000 7f7ab67fc700 1 -- 172.21.15.111:0/1420285461 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7ab8160b70 msgr2=0x7f7ab80b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:23.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.017+0000 7f7ab67fc700 1 --2- 172.21.15.111:0/1420285461 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7ab8160b70 0x7f7ab80b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:23.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.017+0000 7f7ab67fc700 1 -- 172.21.15.111:0/1420285461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab80b87a0 msgr2=0x7f7ab815d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:23.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.017+0000 7f7ab67fc700 1 --2- 172.21.15.111:0/1420285461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab80b87a0 0x7f7ab815d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:23.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.017+0000 7f7ab67fc700 1 -- 172.21.15.111:0/1420285461 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac009550 con 0x7f7ab81601e0 2023-03-29T10:57:23.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.017+0000 7f7ab67fc700 1 --2- 172.21.15.111:0/1420285461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ab81601e0 0x7f7ab81f1650 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009870 tx=0x7f7aac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:23.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.018+0000 7f7a9f7fe700 1 -- 172.21.15.111:0/1420285461 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac004380 con 0x7f7ab81601e0 2023-03-29T10:57:23.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.018+0000 7f7a9f7fe700 1 -- 172.21.15.111:0/1420285461 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aac01f070 con 0x7f7ab81601e0 2023-03-29T10:57:23.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.018+0000 7f7abd3ff700 1 -- 172.21.15.111:0/1420285461 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab8160f90 con 0x7f7ab81601e0 2023-03-29T10:57:23.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.018+0000 7f7a9f7fe700 1 -- 172.21.15.111:0/1420285461 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac0036a0 con 0x7f7ab81601e0 2023-03-29T10:57:23.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.018+0000 7f7abd3ff700 1 -- 172.21.15.111:0/1420285461 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab80cd980 con 0x7f7ab81601e0 2023-03-29T10:57:23.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.019+0000 7f7abd3ff700 1 -- 172.21.15.111:0/1420285461 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab80d2110 con 0x7f7ab81601e0 2023-03-29T10:57:23.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.020+0000 7f7a9f7fe700 1 -- 172.21.15.111:0/1420285461 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f7aac021030 con 0x7f7ab81601e0 2023-03-29T10:57:23.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.020+0000 7f7a9f7fe700 1 --2- 172.21.15.111:0/1420285461 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7aa4069de0 0x7f7aa406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:23.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.021+0000 7f7a9f7fe700 1 -- 172.21.15.111:0/1420285461 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f7aac088ff0 con 0x7f7ab81601e0 2023-03-29T10:57:23.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.021+0000 7f7ab6ffd700 1 --2- 172.21.15.111:0/1420285461 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7aa4069de0 0x7f7aa406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:23.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.021+0000 7f7ab6ffd700 1 --2- 172.21.15.111:0/1420285461 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7aa4069de0 0x7f7aa406c290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009be0 tx=0x7f7aa8009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:23.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.025+0000 7f7a9f7fe700 1 -- 172.21.15.111:0/1420285461 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f7aac0bb160 con 0x7f7ab81601e0 2023-03-29T10:57:23.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.217+0000 7f7abd3ff700 1 -- 172.21.15.111:0/1420285461 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ab8074430 con 0x7f7ab81601e0 2023-03-29T10:57:23.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.217+0000 7f7a9f7fe700 1 -- 172.21.15.111:0/1420285461 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7aac0150a0 con 0x7f7ab81601e0 2023-03-29T10:57:23.218 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-03-29T10:57:23.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.219+0000 7f7abd3ff700 1 -- 172.21.15.111:0/1420285461 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7aa4069de0 msgr2=0x7f7aa406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:23.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.219+0000 7f7abd3ff700 1 --2- 172.21.15.111:0/1420285461 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7aa4069de0 0x7f7aa406c290 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009be0 tx=0x7f7aa8009210 comp rx=0 tx=0).stop 2023-03-29T10:57:23.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.219+0000 7f7abd3ff700 1 -- 172.21.15.111:0/1420285461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ab81601e0 msgr2=0x7f7ab81f1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:23.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.219+0000 7f7abd3ff700 1 --2- 172.21.15.111:0/1420285461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ab81601e0 0x7f7ab81f1650 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7aac009870 tx=0x7f7aac004750 comp rx=0 tx=0).stop 2023-03-29T10:57:23.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.219+0000 7f7abd3ff700 1 -- 172.21.15.111:0/1420285461 shutdown_connections 2023-03-29T10:57:23.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.219+0000 7f7abd3ff700 1 --2- 172.21.15.111:0/1420285461 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7aa4069de0 0x7f7aa406c290 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:23.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.219+0000 7f7abd3ff700 1 --2- 172.21.15.111:0/1420285461 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7ab8160b70 0x7f7ab80b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:23.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.219+0000 7f7abd3ff700 1 --2- 172.21.15.111:0/1420285461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ab80b87a0 0x7f7ab815d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:23.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.219+0000 7f7abd3ff700 1 --2- 172.21.15.111:0/1420285461 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7ab81601e0 0x7f7ab81f1650 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:23.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.219+0000 7f7abd3ff700 1 -- 172.21.15.111:0/1420285461 >> 172.21.15.111:0/1420285461 conn(0x7f7ab8071b10 msgr2=0x7f7ab815f300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:23.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.219+0000 7f7abd3ff700 1 -- 172.21.15.111:0/1420285461 shutdown_connections 2023-03-29T10:57:23.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.219+0000 7f7abd3ff700 1 -- 172.21.15.111:0/1420285461 wait complete. 2023-03-29T10:57:23.237 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-29T10:57:23.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.423+0000 7f41929ff700 1 Processor -- start 2023-03-29T10:57:23.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.423+0000 7f41929ff700 1 -- start start 2023-03-29T10:57:23.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.425+0000 7f41929ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f418c0b87a0 0x7f418c0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:23.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.425+0000 7f41929ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f418c0cdc60 con 0x7f418c160d40 2023-03-29T10:57:23.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.425+0000 7f41929ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f418c0cd6a0 con 0x7f418c1601e0 2023-03-29T10:57:23.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.425+0000 7f41929ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418c0d02d0 con 0x7f418c0b87a0 2023-03-29T10:57:23.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.425+0000 7f4190f9c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f418c160d40 0x7f418c0d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53582/0 (socket says 172.21.15.111:53582) 2023-03-29T10:57:23.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.425+0000 7f4190f9c700 1 -- 172.21.15.111:0/2135844184 learned_addr learned my addr 172.21.15.111:0/2135844184 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:23.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.425+0000 7f418bfff700 1 --2- 172.21.15.111:0/2135844184 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f418c0b87a0 0x7f418c0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:23.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.425+0000 7f418b7fe700 1 -- 172.21.15.111:0/2135844184 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 172736201 0 0) 0x7f418c0cdc60 con 0x7f418c160d40 2023-03-29T10:57:23.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.425+0000 7f418b7fe700 1 -- 172.21.15.111:0/2135844184 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41780035b0 con 0x7f418c160d40 2023-03-29T10:57:23.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.425+0000 7f418b7fe700 1 -- 172.21.15.111:0/2135844184 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188331373 0 0) 0x7f418c0cd6a0 con 0x7f418c1601e0 2023-03-29T10:57:23.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.425+0000 7f418b7fe700 1 -- 172.21.15.111:0/2135844184 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f418c0cdc60 con 0x7f418c1601e0 2023-03-29T10:57:23.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.426+0000 7f418b7fe700 1 -- 172.21.15.111:0/2135844184 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3651800913 0 0) 0x7f41780035b0 con 0x7f418c160d40 2023-03-29T10:57:23.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.426+0000 7f418bfff700 1 -- 172.21.15.111:0/2135844184 >> v1:172.21.15.119:6789/0 conn(0x7f418c1601e0 legacy=0x7f418c160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:23.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.426+0000 7f418bfff700 1 -- 172.21.15.111:0/2135844184 >> v1:172.21.15.111:6789/0 conn(0x7f418c160d40 legacy=0x7f418c0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:23.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.426+0000 7f418bfff700 1 -- 172.21.15.111:0/2135844184 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418c0cd980 con 0x7f418c0b87a0 2023-03-29T10:57:23.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.426+0000 7f418bfff700 1 --2- 172.21.15.111:0/2135844184 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f418c0b87a0 0x7f418c0cfbf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f417c009870 tx=0x7f417c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b131595b790069e5 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:23.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.426+0000 7f418b7fe700 1 -- 172.21.15.111:0/2135844184 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41840019f0 con 0x7f418c0b87a0 2023-03-29T10:57:23.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.426+0000 7f418b7fe700 1 -- 172.21.15.111:0/2135844184 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417c004030 con 0x7f418c0b87a0 2023-03-29T10:57:23.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.426+0000 7f418b7fe700 1 -- 172.21.15.111:0/2135844184 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417c0043c0 con 0x7f418c0b87a0 2023-03-29T10:57:23.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.427+0000 7f41929ff700 1 -- 172.21.15.111:0/2135844184 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f418c0b87a0 msgr2=0x7f418c0cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:23.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.427+0000 7f41929ff700 1 --2- 172.21.15.111:0/2135844184 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f418c0b87a0 0x7f418c0cfbf0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f417c009870 tx=0x7f417c009b80 comp rx=0 tx=0).stop 2023-03-29T10:57:23.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.427+0000 7f41929ff700 1 -- 172.21.15.111:0/2135844184 shutdown_connections 2023-03-29T10:57:23.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.427+0000 7f41929ff700 1 --2- 172.21.15.111:0/2135844184 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f418c0b87a0 0x7f418c0cfbf0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:23.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.427+0000 7f41929ff700 1 -- 172.21.15.111:0/2135844184 >> 172.21.15.111:0/2135844184 conn(0x7f418c071b10 msgr2=0x7f418c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:23.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.427+0000 7f41929ff700 1 -- 172.21.15.111:0/2135844184 shutdown_connections 2023-03-29T10:57:23.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.427+0000 7f41929ff700 1 -- 172.21.15.111:0/2135844184 wait complete. 2023-03-29T10:57:23.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.428+0000 7f41929ff700 1 Processor -- start 2023-03-29T10:57:23.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.429+0000 7f41929ff700 1 -- start start 2023-03-29T10:57:23.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.429+0000 7f41929ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f418c0b87a0 0x7f418c153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:23.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.429+0000 7f41929ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f418c1601e0 0x7f418c1e5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:23.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.430+0000 7f41929ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f418c160d40 0x7f418c154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:23.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.430+0000 7f41929ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418c154000 con 0x7f418c160d40 2023-03-29T10:57:23.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.430+0000 7f41929ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418c0cc010 con 0x7f418c0b87a0 2023-03-29T10:57:23.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.430+0000 7f41929ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418c0cb440 con 0x7f418c1601e0 2023-03-29T10:57:23.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.430+0000 7f4190f9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f418c160d40 0x7f418c154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:23.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.430+0000 7f4190f9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f418c160d40 0x7f418c154170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50806/0 (socket says 172.21.15.111:50806) 2023-03-29T10:57:23.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.430+0000 7f4190f9c700 1 -- 172.21.15.111:0/496569789 learned_addr learned my addr 172.21.15.111:0/496569789 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:23.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.430+0000 7f418bfff700 1 --2- 172.21.15.111:0/496569789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f418c0b87a0 0x7f418c153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:23.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.430+0000 7f4183fff700 1 --2- 172.21.15.111:0/496569789 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f418c1601e0 0x7f418c1e5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:23.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.430+0000 7f4190f9c700 1 -- 172.21.15.111:0/496569789 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f418c1601e0 msgr2=0x7f418c1e5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:23.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.430+0000 7f4190f9c700 1 --2- 172.21.15.111:0/496569789 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f418c1601e0 0x7f418c1e5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:23.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.430+0000 7f4190f9c700 1 -- 172.21.15.111:0/496569789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f418c0b87a0 msgr2=0x7f418c153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:23.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.430+0000 7f4190f9c700 1 --2- 172.21.15.111:0/496569789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f418c0b87a0 0x7f418c153ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:23.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.430+0000 7f4190f9c700 1 -- 172.21.15.111:0/496569789 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417c0094a0 con 0x7f418c160d40 2023-03-29T10:57:23.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.430+0000 7f4190f9c700 1 --2- 172.21.15.111:0/496569789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f418c160d40 0x7f418c154170 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f418400cc00 tx=0x7f418400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:23.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.431+0000 7f41897fa700 1 -- 172.21.15.111:0/496569789 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418400aa80 con 0x7f418c160d40 2023-03-29T10:57:23.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.431+0000 7f41897fa700 1 -- 172.21.15.111:0/496569789 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418400abe0 con 0x7f418c160d40 2023-03-29T10:57:23.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.431+0000 7f41897fa700 1 -- 172.21.15.111:0/496569789 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41840070d0 con 0x7f418c160d40 2023-03-29T10:57:23.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.431+0000 7f41929ff700 1 -- 172.21.15.111:0/496569789 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f418c0d19a0 con 0x7f418c160d40 2023-03-29T10:57:23.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.431+0000 7f41929ff700 1 -- 172.21.15.111:0/496569789 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f418c0d1810 con 0x7f418c160d40 2023-03-29T10:57:23.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.432+0000 7f41929ff700 1 -- 172.21.15.111:0/496569789 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f418c0d1b30 con 0x7f418c160d40 2023-03-29T10:57:23.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.433+0000 7f41897fa700 1 -- 172.21.15.111:0/496569789 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f4184007230 con 0x7f418c160d40 2023-03-29T10:57:23.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.434+0000 7f41897fa700 1 --2- 172.21.15.111:0/496569789 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f416c069de0 0x7f416c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:23.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.434+0000 7f41897fa700 1 -- 172.21.15.111:0/496569789 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f41840876f0 con 0x7f418c160d40 2023-03-29T10:57:23.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.434+0000 7f418bfff700 1 --2- 172.21.15.111:0/496569789 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f416c069de0 0x7f416c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:23.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.434+0000 7f418bfff700 1 --2- 172.21.15.111:0/496569789 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f416c069de0 0x7f416c06c290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f417c0090b0 tx=0x7f417c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:23.438 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.438+0000 7f41897fa700 1 -- 172.21.15.111:0/496569789 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f41840532f0 con 0x7f418c160d40 2023-03-29T10:57:23.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.633+0000 7f41929ff700 1 -- 172.21.15.111:0/496569789 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f418c0cc180 con 0x7f418c160d40 2023-03-29T10:57:23.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.634+0000 7f41897fa700 1 -- 172.21.15.111:0/496569789 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4184018020 con 0x7f418c160d40 2023-03-29T10:57:23.635 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 8 2023-03-29T10:57:23.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.636+0000 7f41929ff700 1 -- 172.21.15.111:0/496569789 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f416c069de0 msgr2=0x7f416c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:23.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.636+0000 7f41929ff700 1 --2- 172.21.15.111:0/496569789 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f416c069de0 0x7f416c06c290 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f417c0090b0 tx=0x7f417c008030 comp rx=0 tx=0).stop 2023-03-29T10:57:23.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.636+0000 7f41929ff700 1 -- 172.21.15.111:0/496569789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f418c160d40 msgr2=0x7f418c154170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:23.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.636+0000 7f41929ff700 1 --2- 172.21.15.111:0/496569789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f418c160d40 0x7f418c154170 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f418400cc00 tx=0x7f418400cf10 comp rx=0 tx=0).stop 2023-03-29T10:57:23.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.636+0000 7f41929ff700 1 -- 172.21.15.111:0/496569789 shutdown_connections 2023-03-29T10:57:23.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.636+0000 7f41929ff700 1 --2- 172.21.15.111:0/496569789 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f416c069de0 0x7f416c06c290 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:23.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.636+0000 7f41929ff700 1 --2- 172.21.15.111:0/496569789 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f418c160d40 0x7f418c154170 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:23.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.636+0000 7f41929ff700 1 --2- 172.21.15.111:0/496569789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f418c0b87a0 0x7f418c153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:23.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.636+0000 7f41929ff700 1 --2- 172.21.15.111:0/496569789 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f418c1601e0 0x7f418c1e5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:23.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.636+0000 7f41929ff700 1 -- 172.21.15.111:0/496569789 >> 172.21.15.111:0/496569789 conn(0x7f418c071b10 msgr2=0x7f418c15c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:23.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.637+0000 7f41929ff700 1 -- 172.21.15.111:0/496569789 shutdown_connections 2023-03-29T10:57:23.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.637+0000 7f41929ff700 1 -- 172.21.15.111:0/496569789 wait complete. 2023-03-29T10:57:23.654 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-29T10:57:23.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.837+0000 7fe0b69ff700 1 Processor -- start 2023-03-29T10:57:23.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.837+0000 7fe0b69ff700 1 -- start start 2023-03-29T10:57:23.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.838+0000 7fe0b69ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe0b00cf970 0x7fe0b00cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:23.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.838+0000 7fe0b69ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b00cdc10 con 0x7fe0b00cfd40 2023-03-29T10:57:23.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.839+0000 7fe0b69ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b00b6c60 con 0x7fe0b00b70f0 2023-03-29T10:57:23.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.839+0000 7fe0b69ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0b00d08d0 con 0x7fe0b00cf970 2023-03-29T10:57:23.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.839+0000 7fe0af7fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe0b00cfd40 0x7fe0b00d01c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53588/0 (socket says 172.21.15.111:53588) 2023-03-29T10:57:23.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.839+0000 7fe0af7fe700 1 -- 172.21.15.111:0/2302220675 learned_addr learned my addr 172.21.15.111:0/2302220675 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:23.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.839+0000 7fe0affff700 1 --2- 172.21.15.111:0/2302220675 >> v2:172.21.15.111:3300/0 conn(0x7fe0b00cf970 0x7fe0b00cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:23.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.839+0000 7fe0aeffd700 1 -- 172.21.15.111:0/2302220675 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352330592 0 0) 0x7fe0b00cdc10 con 0x7fe0b00cfd40 2023-03-29T10:57:23.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.839+0000 7fe0aeffd700 1 -- 172.21.15.111:0/2302220675 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0980035b0 con 0x7fe0b00cfd40 2023-03-29T10:57:23.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.839+0000 7fe0affff700 1 -- 172.21.15.111:0/2302220675 >> v1:172.21.15.119:6789/0 conn(0x7fe0b00b70f0 legacy=0x7fe0b00b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:23.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.839+0000 7fe0affff700 1 -- 172.21.15.111:0/2302220675 >> v1:172.21.15.111:6789/0 conn(0x7fe0b00cfd40 legacy=0x7fe0b00d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:23.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.839+0000 7fe0affff700 1 -- 172.21.15.111:0/2302220675 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b00cd650 con 0x7fe0b00cf970 2023-03-29T10:57:23.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.840+0000 7fe0affff700 1 --2- 172.21.15.111:0/2302220675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0b00cf970 0x7fe0b00cf110 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009840 tx=0x7fe0a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1d09dad2736bf6 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:23.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.840+0000 7fe0aeffd700 1 -- 172.21.15.111:0/2302220675 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a0004230 con 0x7fe0b00cf970 2023-03-29T10:57:23.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.840+0000 7fe0aeffd700 1 -- 172.21.15.111:0/2302220675 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a0004390 con 0x7fe0b00cf970 2023-03-29T10:57:23.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.840+0000 7fe0aeffd700 1 -- 172.21.15.111:0/2302220675 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a00036c0 con 0x7fe0b00cf970 2023-03-29T10:57:23.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.840+0000 7fe0b69ff700 1 -- 172.21.15.111:0/2302220675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0b00cf970 msgr2=0x7fe0b00cf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:23.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.840+0000 7fe0b69ff700 1 --2- 172.21.15.111:0/2302220675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0b00cf970 0x7fe0b00cf110 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe0a0009840 tx=0x7fe0a0009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:23.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.840+0000 7fe0b69ff700 1 -- 172.21.15.111:0/2302220675 shutdown_connections 2023-03-29T10:57:23.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.840+0000 7fe0b69ff700 1 --2- 172.21.15.111:0/2302220675 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0b00cf970 0x7fe0b00cf110 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:23.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.840+0000 7fe0b69ff700 1 -- 172.21.15.111:0/2302220675 >> 172.21.15.111:0/2302220675 conn(0x7fe0b0071b10 msgr2=0x7fe0b0071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:23.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.840+0000 7fe0b69ff700 1 -- 172.21.15.111:0/2302220675 shutdown_connections 2023-03-29T10:57:23.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.840+0000 7fe0b69ff700 1 -- 172.21.15.111:0/2302220675 wait complete. 2023-03-29T10:57:23.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.842+0000 7fe0b69ff700 1 Processor -- start 2023-03-29T10:57:23.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.842+0000 7fe0b69ff700 1 -- start start 2023-03-29T10:57:23.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.843+0000 7fe0b69ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0b00b70f0 0x7fe0b0159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:23.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.843+0000 7fe0b69ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe0b00cf970 0x7fe0b00b7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:23.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.843+0000 7fe0b69ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0b00cfd40 0x7fe0b00cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:23.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.843+0000 7fe0b69ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b015a3a0 con 0x7fe0b00cfd40 2023-03-29T10:57:23.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.843+0000 7fe0b69ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b00b8650 con 0x7fe0b00b70f0 2023-03-29T10:57:23.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.843+0000 7fe0b69ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b0147400 con 0x7fe0b00cf970 2023-03-29T10:57:23.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.843+0000 7fe0b4f9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0b00cfd40 0x7fe0b00cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:23.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.843+0000 7fe0b4f9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0b00cfd40 0x7fe0b00cc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50822/0 (socket says 172.21.15.111:50822) 2023-03-29T10:57:23.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.843+0000 7fe0b4f9c700 1 -- 172.21.15.111:0/2832460866 learned_addr learned my addr 172.21.15.111:0/2832460866 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:23.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.844+0000 7fe0b4f9c700 1 -- 172.21.15.111:0/2832460866 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe0b00cf970 msgr2=0x7fe0b00b7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:23.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.844+0000 7fe0b4f9c700 1 --2- 172.21.15.111:0/2832460866 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe0b00cf970 0x7fe0b00b7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:23.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.844+0000 7fe0b4f9c700 1 -- 172.21.15.111:0/2832460866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0b00b70f0 msgr2=0x7fe0b0159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:23.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.844+0000 7fe0b4f9c700 1 --2- 172.21.15.111:0/2832460866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0b00b70f0 0x7fe0b0159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:23.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.844+0000 7fe0b4f9c700 1 -- 172.21.15.111:0/2832460866 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a00094a0 con 0x7fe0b00cfd40 2023-03-29T10:57:23.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.844+0000 7fe0b4f9c700 1 --2- 172.21.15.111:0/2832460866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0b00cfd40 0x7fe0b00cc470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800b970 tx=0x7fe0a800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:23.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.844+0000 7fe0acff9700 1 -- 172.21.15.111:0/2832460866 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a800c9b0 con 0x7fe0b00cfd40 2023-03-29T10:57:23.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.844+0000 7fe0acff9700 1 -- 172.21.15.111:0/2832460866 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a800cb10 con 0x7fe0b00cfd40 2023-03-29T10:57:23.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.844+0000 7fe0b69ff700 1 -- 172.21.15.111:0/2832460866 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b00b68f0 con 0x7fe0b00cfd40 2023-03-29T10:57:23.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.844+0000 7fe0acff9700 1 -- 172.21.15.111:0/2832460866 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8014440 con 0x7fe0b00cfd40 2023-03-29T10:57:23.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.844+0000 7fe0b69ff700 1 -- 172.21.15.111:0/2832460866 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b00cd930 con 0x7fe0b00cfd40 2023-03-29T10:57:23.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.845+0000 7fe0b69ff700 1 -- 172.21.15.111:0/2832460866 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b00cf650 con 0x7fe0b00cfd40 2023-03-29T10:57:23.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.846+0000 7fe0acff9700 1 -- 172.21.15.111:0/2832460866 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fe0a801d3b0 con 0x7fe0b00cfd40 2023-03-29T10:57:23.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.847+0000 7fe0acff9700 1 --2- 172.21.15.111:0/2832460866 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe09c069de0 0x7fe09c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:23.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.847+0000 7fe0acff9700 1 -- 172.21.15.111:0/2832460866 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe0a80878c0 con 0x7fe0b00cfd40 2023-03-29T10:57:23.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.847+0000 7fe0affff700 1 --2- 172.21.15.111:0/2832460866 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe09c069de0 0x7fe09c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:23.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.848+0000 7fe0affff700 1 --2- 172.21.15.111:0/2832460866 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe09c069de0 0x7fe09c06c290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe0b01f57c0 tx=0x7fe0a0009f90 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:23.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:23.851+0000 7fe0acff9700 1 -- 172.21.15.111:0/2832460866 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fe0a8053450 con 0x7fe0b00cfd40 2023-03-29T10:57:24.086 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.086+0000 7fe0b69ff700 1 -- 172.21.15.111:0/2832460866 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0b0074430 con 0x7fe0b00cfd40 2023-03-29T10:57:24.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.086+0000 7fe0acff9700 1 -- 172.21.15.111:0/2832460866 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7fe0a8019de0 con 0x7fe0b00cfd40 2023-03-29T10:57:24.087 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:24.087 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/556826332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":556826332},{"type":"v1","addr":"172.21.15.111:6837","nonce":556826332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/814623168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":814623168},{"type":"v1","addr":"172.21.15.111:6835","nonce":814623168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/514422035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":514422035},{"type":"v1","addr":"172.21.15.119:6834","nonce":514422035}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4296,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3160800581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":3160800581},{"type":"v1","addr":"172.21.15.119:6835","nonce":3160800581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-03-29T10:57:24.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.089+0000 7fe0b69ff700 1 -- 172.21.15.111:0/2832460866 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe09c069de0 msgr2=0x7fe09c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:24.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.089+0000 7fe0b69ff700 1 --2- 172.21.15.111:0/2832460866 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe09c069de0 0x7fe09c06c290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe0b01f57c0 tx=0x7fe0a0009f90 comp rx=0 tx=0).stop 2023-03-29T10:57:24.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.089+0000 7fe0b69ff700 1 -- 172.21.15.111:0/2832460866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0b00cfd40 msgr2=0x7fe0b00cc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:24.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.089+0000 7fe0b69ff700 1 --2- 172.21.15.111:0/2832460866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0b00cfd40 0x7fe0b00cc470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800b970 tx=0x7fe0a800bd30 comp rx=0 tx=0).stop 2023-03-29T10:57:24.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.089+0000 7fe0b69ff700 1 -- 172.21.15.111:0/2832460866 shutdown_connections 2023-03-29T10:57:24.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.089+0000 7fe0b69ff700 1 --2- 172.21.15.111:0/2832460866 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe09c069de0 0x7fe09c06c290 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.089+0000 7fe0b69ff700 1 --2- 172.21.15.111:0/2832460866 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe0b00cfd40 0x7fe0b00cc470 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.089+0000 7fe0b69ff700 1 --2- 172.21.15.111:0/2832460866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0b00b70f0 0x7fe0b0159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.089+0000 7fe0b69ff700 1 --2- 172.21.15.111:0/2832460866 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe0b00cf970 0x7fe0b00b7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.089+0000 7fe0b69ff700 1 -- 172.21.15.111:0/2832460866 >> 172.21.15.111:0/2832460866 conn(0x7fe0b0071b10 msgr2=0x7fe0b00d1990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:24.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.089+0000 7fe0b69ff700 1 -- 172.21.15.111:0/2832460866 shutdown_connections 2023-03-29T10:57:24.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.089+0000 7fe0b69ff700 1 -- 172.21.15.111:0/2832460866 wait complete. 2023-03-29T10:57:24.091 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 3 2023-03-29T10:57:24.107 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:57:24.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.290+0000 7fea81dff700 1 Processor -- start 2023-03-29T10:57:24.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.290+0000 7fea81dff700 1 -- start start 2023-03-29T10:57:24.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.291+0000 7fea81dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea7c0cfd40 0x7fea7c0d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:24.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.292+0000 7fea81dff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea7c0cdc10 con 0x7fea7c0cf970 2023-03-29T10:57:24.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.292+0000 7fea81dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea7c0b6c60 con 0x7fea7c0b70f0 2023-03-29T10:57:24.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.292+0000 7fea81dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c0cd650 con 0x7fea7c0cfd40 2023-03-29T10:57:24.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.292+0000 7fea7b7fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fea7c0cf970 0x7fea7c0cf110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53600/0 (socket says 172.21.15.111:53600) 2023-03-29T10:57:24.293 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.292+0000 7fea7b7fe700 1 -- 172.21.15.111:0/1881519934 learned_addr learned my addr 172.21.15.111:0/1881519934 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:24.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.292+0000 7fea73fff700 1 --2- 172.21.15.111:0/1881519934 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea7c0cfd40 0x7fea7c0d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:24.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.292+0000 7fea7affd700 1 -- 172.21.15.111:0/1881519934 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2254869970 0 0) 0x7fea7c0cdc10 con 0x7fea7c0cf970 2023-03-29T10:57:24.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.292+0000 7fea7affd700 1 -- 172.21.15.111:0/1881519934 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea680035b0 con 0x7fea7c0cf970 2023-03-29T10:57:24.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.292+0000 7fea7affd700 1 -- 172.21.15.111:0/1881519934 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 722325996 0 0) 0x7fea7c0b6c60 con 0x7fea7c0b70f0 2023-03-29T10:57:24.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.292+0000 7fea7affd700 1 -- 172.21.15.111:0/1881519934 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea7c0cdc10 con 0x7fea7c0b70f0 2023-03-29T10:57:24.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.292+0000 7fea7affd700 1 -- 172.21.15.111:0/1881519934 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1433012880 0 0) 0x7fea680035b0 con 0x7fea7c0cf970 2023-03-29T10:57:24.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.292+0000 7fea73fff700 1 -- 172.21.15.111:0/1881519934 >> v1:172.21.15.119:6789/0 conn(0x7fea7c0b70f0 legacy=0x7fea7c0b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:24.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.293+0000 7fea73fff700 1 -- 172.21.15.111:0/1881519934 >> v1:172.21.15.111:6789/0 conn(0x7fea7c0cf970 legacy=0x7fea7c0cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:24.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.293+0000 7fea73fff700 1 -- 172.21.15.111:0/1881519934 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7c0cd930 con 0x7fea7c0cfd40 2023-03-29T10:57:24.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.293+0000 7fea73fff700 1 --2- 172.21.15.111:0/1881519934 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea7c0cfd40 0x7fea7c0d0190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fea64009870 tx=0x7fea64009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7b2dd686103e773 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:24.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.293+0000 7fea7affd700 1 -- 172.21.15.111:0/1881519934 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c001670 con 0x7fea7c0cfd40 2023-03-29T10:57:24.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.293+0000 7fea7affd700 1 -- 172.21.15.111:0/1881519934 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea64004030 con 0x7fea7c0cfd40 2023-03-29T10:57:24.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.293+0000 7fea7affd700 1 -- 172.21.15.111:0/1881519934 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea640043a0 con 0x7fea7c0cfd40 2023-03-29T10:57:24.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.293+0000 7fea81dff700 1 -- 172.21.15.111:0/1881519934 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea7c0cfd40 msgr2=0x7fea7c0d0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:24.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.293+0000 7fea81dff700 1 --2- 172.21.15.111:0/1881519934 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea7c0cfd40 0x7fea7c0d0190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fea64009870 tx=0x7fea64009b80 comp rx=0 tx=0).stop 2023-03-29T10:57:24.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.294+0000 7fea81dff700 1 -- 172.21.15.111:0/1881519934 shutdown_connections 2023-03-29T10:57:24.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.294+0000 7fea81dff700 1 --2- 172.21.15.111:0/1881519934 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea7c0cfd40 0x7fea7c0d0190 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.294+0000 7fea81dff700 1 -- 172.21.15.111:0/1881519934 >> 172.21.15.111:0/1881519934 conn(0x7fea7c071b10 msgr2=0x7fea7c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:24.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.294+0000 7fea81dff700 1 -- 172.21.15.111:0/1881519934 shutdown_connections 2023-03-29T10:57:24.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.294+0000 7fea81dff700 1 -- 172.21.15.111:0/1881519934 wait complete. 2023-03-29T10:57:24.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.295+0000 7fea81dff700 1 Processor -- start 2023-03-29T10:57:24.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.295+0000 7fea81dff700 1 -- start start 2023-03-29T10:57:24.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.296+0000 7fea81dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7c0b70f0 0x7fea7c159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:24.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.296+0000 7fea81dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea7c0cf970 0x7fea7c15a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:24.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.296+0000 7fea81dff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea7c0cfd40 0x7fea7c1e6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:24.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.296+0000 7fea81dff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c1e8200 con 0x7fea7c0cfd40 2023-03-29T10:57:24.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.296+0000 7fea81dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c15ef30 con 0x7fea7c0b70f0 2023-03-29T10:57:24.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.296+0000 7fea81dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c1e7680 con 0x7fea7c0cf970 2023-03-29T10:57:24.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.296+0000 7fea7bfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea7c0cfd40 0x7fea7c1e6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:24.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.296+0000 7fea7bfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea7c0cfd40 0x7fea7c1e6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50832/0 (socket says 172.21.15.111:50832) 2023-03-29T10:57:24.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.296+0000 7fea7bfff700 1 -- 172.21.15.111:0/2245053686 learned_addr learned my addr 172.21.15.111:0/2245053686 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:24.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.297+0000 7fea7b7fe700 1 --2- 172.21.15.111:0/2245053686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7c0b70f0 0x7fea7c159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:24.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.297+0000 7fea73fff700 1 --2- 172.21.15.111:0/2245053686 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea7c0cf970 0x7fea7c15a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:24.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.297+0000 7fea7bfff700 1 -- 172.21.15.111:0/2245053686 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea7c0cf970 msgr2=0x7fea7c15a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:24.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.297+0000 7fea7bfff700 1 --2- 172.21.15.111:0/2245053686 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea7c0cf970 0x7fea7c15a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.297+0000 7fea7bfff700 1 -- 172.21.15.111:0/2245053686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7c0b70f0 msgr2=0x7fea7c159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:24.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.297+0000 7fea7bfff700 1 --2- 172.21.15.111:0/2245053686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7c0b70f0 0x7fea7c159d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.297+0000 7fea7bfff700 1 -- 172.21.15.111:0/2245053686 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea640094a0 con 0x7fea7c0cfd40 2023-03-29T10:57:24.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.297+0000 7fea7bfff700 1 --2- 172.21.15.111:0/2245053686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea7c0cfd40 0x7fea7c1e6aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fea7400cbc0 tx=0x7fea7400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:24.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.297+0000 7fea78ff9700 1 -- 172.21.15.111:0/2245053686 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea74003c00 con 0x7fea7c0cfd40 2023-03-29T10:57:24.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.297+0000 7fea78ff9700 1 -- 172.21.15.111:0/2245053686 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea74003d60 con 0x7fea7c0cfd40 2023-03-29T10:57:24.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.297+0000 7fea81dff700 1 -- 172.21.15.111:0/2245053686 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea7c0d06d0 con 0x7fea7c0cfd40 2023-03-29T10:57:24.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.297+0000 7fea78ff9700 1 -- 172.21.15.111:0/2245053686 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7400abb0 con 0x7fea7c0cfd40 2023-03-29T10:57:24.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.297+0000 7fea81dff700 1 -- 172.21.15.111:0/2245053686 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7c0b68f0 con 0x7fea7c0cfd40 2023-03-29T10:57:24.299 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.298+0000 7fea81dff700 1 -- 172.21.15.111:0/2245053686 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7c0d0860 con 0x7fea7c0cfd40 2023-03-29T10:57:24.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.299+0000 7fea78ff9700 1 -- 172.21.15.111:0/2245053686 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fea7400ad10 con 0x7fea7c0cfd40 2023-03-29T10:57:24.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.300+0000 7fea78ff9700 1 --2- 172.21.15.111:0/2245053686 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fea5c069de0 0x7fea5c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:24.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.300+0000 7fea78ff9700 1 -- 172.21.15.111:0/2245053686 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fea74087d70 con 0x7fea7c0cfd40 2023-03-29T10:57:24.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.300+0000 7fea7b7fe700 1 --2- 172.21.15.111:0/2245053686 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fea5c069de0 0x7fea5c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:24.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.301+0000 7fea7b7fe700 1 --2- 172.21.15.111:0/2245053686 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fea5c069de0 0x7fea5c06c290 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0058d0 tx=0x7fea6c001430 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:24.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.304+0000 7fea78ff9700 1 -- 172.21.15.111:0/2245053686 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fea74053960 con 0x7fea7c0cfd40 2023-03-29T10:57:24.501 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.501+0000 7fea81dff700 1 -- 172.21.15.111:0/2245053686 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea7c074430 con 0x7fea7c0cfd40 2023-03-29T10:57:24.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.501+0000 7fea78ff9700 1 -- 172.21.15.111:0/2245053686 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+14312 (secure 0 0 0) 0x7fea74056f80 con 0x7fea7c0cfd40 2023-03-29T10:57:24.502 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:24.502 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":20,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:57:23.314766+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:57:24.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.504+0000 7fea81dff700 1 -- 172.21.15.111:0/2245053686 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fea5c069de0 msgr2=0x7fea5c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:24.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.504+0000 7fea81dff700 1 --2- 172.21.15.111:0/2245053686 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fea5c069de0 0x7fea5c06c290 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0058d0 tx=0x7fea6c001430 comp rx=0 tx=0).stop 2023-03-29T10:57:24.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.504+0000 7fea81dff700 1 -- 172.21.15.111:0/2245053686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea7c0cfd40 msgr2=0x7fea7c1e6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:24.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.504+0000 7fea81dff700 1 --2- 172.21.15.111:0/2245053686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea7c0cfd40 0x7fea7c1e6aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fea7400cbc0 tx=0x7fea7400cf80 comp rx=0 tx=0).stop 2023-03-29T10:57:24.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.504+0000 7fea81dff700 1 -- 172.21.15.111:0/2245053686 shutdown_connections 2023-03-29T10:57:24.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.504+0000 7fea81dff700 1 --2- 172.21.15.111:0/2245053686 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fea5c069de0 0x7fea5c06c290 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.504+0000 7fea81dff700 1 --2- 172.21.15.111:0/2245053686 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea7c0cfd40 0x7fea7c1e6aa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.504+0000 7fea81dff700 1 --2- 172.21.15.111:0/2245053686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea7c0b70f0 0x7fea7c159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.504+0000 7fea81dff700 1 --2- 172.21.15.111:0/2245053686 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea7c0cf970 0x7fea7c15a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.504+0000 7fea81dff700 1 -- 172.21.15.111:0/2245053686 >> 172.21.15.111:0/2245053686 conn(0x7fea7c071b10 msgr2=0x7fea7c15c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:24.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.504+0000 7fea81dff700 1 -- 172.21.15.111:0/2245053686 shutdown_connections 2023-03-29T10:57:24.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.504+0000 7fea81dff700 1 -- 172.21.15.111:0/2245053686 wait complete. 2023-03-29T10:57:24.522 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-29T10:57:16.183979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-29T10:57:21.239809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-03-29T10:57:24.522 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-29T10:57:24.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.708+0000 7f49a89ff700 1 Processor -- start 2023-03-29T10:57:24.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.708+0000 7f49a89ff700 1 -- start start 2023-03-29T10:57:24.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.709+0000 7f49a89ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f49a40b87a0 0x7f49a40cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:24.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.709+0000 7f49a89ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a40cdc60 con 0x7f49a41601e0 2023-03-29T10:57:24.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.709+0000 7f49a89ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a40cd6a0 con 0x7f49a4160d40 2023-03-29T10:57:24.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.709+0000 7f49a89ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49a40d02d0 con 0x7f49a40b87a0 2023-03-29T10:57:24.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.710+0000 7f49a1d9b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f49a41601e0 0x7f49a4160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53612/0 (socket says 172.21.15.111:53612) 2023-03-29T10:57:24.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.710+0000 7f49a1d9b700 1 -- 172.21.15.111:0/1815871154 learned_addr learned my addr 172.21.15.111:0/1815871154 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:24.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.710+0000 7f49a159a700 1 -- 172.21.15.111:0/1815871154 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810033089 0 0) 0x7f49a40cdc60 con 0x7f49a41601e0 2023-03-29T10:57:24.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.710+0000 7f49a159a700 1 -- 172.21.15.111:0/1815871154 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f498c0035b0 con 0x7f49a41601e0 2023-03-29T10:57:24.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.710+0000 7f49a259c700 1 --2- 172.21.15.111:0/1815871154 >> v2:172.21.15.119:3300/0 conn(0x7f49a40b87a0 0x7f49a40cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:24.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.710+0000 7f49a159a700 1 -- 172.21.15.111:0/1815871154 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1309645612 0 0) 0x7f498c0035b0 con 0x7f49a41601e0 2023-03-29T10:57:24.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.710+0000 7f49a159a700 1 -- 172.21.15.111:0/1815871154 >> v2:172.21.15.119:3300/0 conn(0x7f49a40b87a0 msgr2=0x7f49a40cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:24.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.710+0000 7f49a159a700 1 --2- 172.21.15.111:0/1815871154 >> v2:172.21.15.119:3300/0 conn(0x7f49a40b87a0 0x7f49a40cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.710+0000 7f49a159a700 1 -- 172.21.15.111:0/1815871154 >> v1:172.21.15.119:6789/0 conn(0x7f49a4160d40 legacy=0x7f49a40d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:24.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.710+0000 7f49a159a700 1 -- 172.21.15.111:0/1815871154 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a40cd980 con 0x7f49a41601e0 2023-03-29T10:57:24.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.710+0000 7f49a159a700 1 -- 172.21.15.111:0/1815871154 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f4998001670 con 0x7f49a41601e0 2023-03-29T10:57:24.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.710+0000 7f49a259c700 1 --2- 172.21.15.111:0/1815871154 >> v2:172.21.15.119:3300/0 conn(0x7f49a40b87a0 0x7f49a40cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:57:24.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.710+0000 7f49a159a700 1 -- 172.21.15.111:0/1815871154 >> v1:172.21.15.111:6789/0 conn(0x7f49a41601e0 legacy=0x7f49a4160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:24.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.711+0000 7f49a159a700 1 --2- 172.21.15.111:0/1815871154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f498c003a50 0x7f498c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:24.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.711+0000 7f49a159a700 1 --2- 172.21.15.111:0/1815871154 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f498c004850 0x7f498c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:24.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.711+0000 7f49a159a700 1 --2- 172.21.15.111:0/1815871154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f498c005250 0x7f498c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:24.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.711+0000 7f49a159a700 1 -- 172.21.15.111:0/1815871154 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a40d02d0 con 0x7f498c005250 2023-03-29T10:57:24.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.711+0000 7f49a159a700 1 -- 172.21.15.111:0/1815871154 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c00c190 con 0x7f498c003a50 2023-03-29T10:57:24.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.711+0000 7f49a159a700 1 -- 172.21.15.111:0/1815871154 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c00c350 con 0x7f498c004850 2023-03-29T10:57:24.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.711+0000 7f49a2d9d700 1 --2- 172.21.15.111:0/1815871154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f498c005250 0x7f498c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:24.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.711+0000 7f49a259c700 1 --2- 172.21.15.111:0/1815871154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f498c003a50 0x7f498c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:24.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.711+0000 7f49a1d9b700 1 --2- 172.21.15.111:0/1815871154 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f498c004850 0x7f498c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:24.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.711+0000 7f49a2d9d700 1 -- 172.21.15.111:0/1815871154 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f498c004850 msgr2=0x7f498c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:24.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.711+0000 7f49a2d9d700 1 --2- 172.21.15.111:0/1815871154 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f498c004850 0x7f498c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.711+0000 7f49a2d9d700 1 -- 172.21.15.111:0/1815871154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f498c003a50 msgr2=0x7f498c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:24.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.711+0000 7f49a2d9d700 1 --2- 172.21.15.111:0/1815871154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f498c003a50 0x7f498c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.711+0000 7f49a2d9d700 1 -- 172.21.15.111:0/1815871154 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c0035b0 con 0x7f498c005250 2023-03-29T10:57:24.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.711+0000 7f49a1d9b700 1 --2- 172.21.15.111:0/1815871154 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f498c004850 0x7f498c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:24.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.711+0000 7f49a259c700 1 --2- 172.21.15.111:0/1815871154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f498c003a50 0x7f498c003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:24.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.711+0000 7f49a2d9d700 1 -- 172.21.15.111:0/1815871154 reap_dead start 2023-03-29T10:57:24.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.712+0000 7f49a2d9d700 1 --2- 172.21.15.111:0/1815871154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f498c005250 0x7f498c00ba10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f498c005830 tx=0x7f498c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae10f104fc3d76f server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:24.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.712+0000 7f49a159a700 1 -- 172.21.15.111:0/1815871154 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c001db0 con 0x7f498c005250 2023-03-29T10:57:24.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.712+0000 7f49a159a700 1 -- 172.21.15.111:0/1815871154 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499c00cc70 con 0x7f498c005250 2023-03-29T10:57:24.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.712+0000 7f49a159a700 1 -- 172.21.15.111:0/1815871154 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c003a00 con 0x7f498c005250 2023-03-29T10:57:24.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.712+0000 7f49a89ff700 1 -- 172.21.15.111:0/1815871154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f498c005250 msgr2=0x7f498c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:24.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.712+0000 7f49a89ff700 1 --2- 172.21.15.111:0/1815871154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f498c005250 0x7f498c00ba10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f498c005830 tx=0x7f498c005880 comp rx=0 tx=0).stop 2023-03-29T10:57:24.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.712+0000 7f49a89ff700 1 -- 172.21.15.111:0/1815871154 shutdown_connections 2023-03-29T10:57:24.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.712+0000 7f49a89ff700 1 --2- 172.21.15.111:0/1815871154 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f498c005250 0x7f498c00ba10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.712+0000 7f49a89ff700 1 -- 172.21.15.111:0/1815871154 >> 172.21.15.111:0/1815871154 conn(0x7f49a4071b10 msgr2=0x7f49a4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:24.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.712+0000 7f49a89ff700 1 -- 172.21.15.111:0/1815871154 shutdown_connections 2023-03-29T10:57:24.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.712+0000 7f49a89ff700 1 -- 172.21.15.111:0/1815871154 wait complete. 2023-03-29T10:57:24.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.714+0000 7f49a89ff700 1 Processor -- start 2023-03-29T10:57:24.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.714+0000 7f49a89ff700 1 -- start start 2023-03-29T10:57:24.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.715+0000 7f49a89ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f498c005250 0x7f49a414a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.715+0000 7f49a89ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a40b6e60 0x7f49a40b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.715+0000 7f49a89ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f49a415d3b0 0x7f49a415d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.715+0000 7f49a89ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a41f3370 con 0x7f498c005250 2023-03-29T10:57:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.715+0000 7f49a89ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a414a870 con 0x7f49a40b6e60 2023-03-29T10:57:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.715+0000 7f49a89ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a40b7810 con 0x7f49a415d3b0 2023-03-29T10:57:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.715+0000 7f49a259c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f498c005250 0x7f49a414a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.715+0000 7f49a259c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f498c005250 0x7f49a414a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50860/0 (socket says 172.21.15.111:50860) 2023-03-29T10:57:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.715+0000 7f49a259c700 1 -- 172.21.15.111:0/152854140 learned_addr learned my addr 172.21.15.111:0/152854140 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.716+0000 7f49a2d9d700 1 --2- 172.21.15.111:0/152854140 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f49a415d3b0 0x7f49a415d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.716+0000 7f49a1d9b700 1 --2- 172.21.15.111:0/152854140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a40b6e60 0x7f49a40b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.716+0000 7f49a259c700 1 -- 172.21.15.111:0/152854140 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f49a415d3b0 msgr2=0x7f49a415d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.716+0000 7f49a259c700 1 --2- 172.21.15.111:0/152854140 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f49a415d3b0 0x7f49a415d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.716+0000 7f49a259c700 1 -- 172.21.15.111:0/152854140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a40b6e60 msgr2=0x7f49a40b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.716+0000 7f49a259c700 1 --2- 172.21.15.111:0/152854140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a40b6e60 0x7f49a40b72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.716+0000 7f49a259c700 1 -- 172.21.15.111:0/152854140 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a40cfd80 con 0x7f498c005250 2023-03-29T10:57:24.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.716+0000 7f49a259c700 1 --2- 172.21.15.111:0/152854140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f498c005250 0x7f49a414a330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4994010810 tx=0x7f4994010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:24.718 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.716+0000 7f4992ffd700 1 -- 172.21.15.111:0/152854140 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499400e9a0 con 0x7f498c005250 2023-03-29T10:57:24.718 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.716+0000 7f4992ffd700 1 -- 172.21.15.111:0/152854140 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499400eb00 con 0x7f498c005250 2023-03-29T10:57:24.718 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.716+0000 7f4992ffd700 1 -- 172.21.15.111:0/152854140 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499400ee70 con 0x7f498c005250 2023-03-29T10:57:24.718 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.716+0000 7f49a89ff700 1 -- 172.21.15.111:0/152854140 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a40cfbf0 con 0x7f498c005250 2023-03-29T10:57:24.718 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.716+0000 7f49a89ff700 1 -- 172.21.15.111:0/152854140 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a40cff10 con 0x7f498c005250 2023-03-29T10:57:24.718 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.717+0000 7f49a89ff700 1 -- 172.21.15.111:0/152854140 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a4160630 con 0x7f498c005250 2023-03-29T10:57:24.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.718+0000 7f4992ffd700 1 -- 172.21.15.111:0/152854140 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f4994015470 con 0x7f498c005250 2023-03-29T10:57:24.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.719+0000 7f4992ffd700 1 --2- 172.21.15.111:0/152854140 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4984069de0 0x7f498406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:24.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.719+0000 7f4992ffd700 1 -- 172.21.15.111:0/152854140 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f49940889b0 con 0x7f498c005250 2023-03-29T10:57:24.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.719+0000 7f49a1d9b700 1 --2- 172.21.15.111:0/152854140 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4984069de0 0x7f498406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:24.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.720+0000 7f49a1d9b700 1 --2- 172.21.15.111:0/152854140 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4984069de0 0x7f498406c290 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f49a415b730 tx=0x7f4998009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:24.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.723+0000 7f4992ffd700 1 -- 172.21.15.111:0/152854140 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f49940545a0 con 0x7f498c005250 2023-03-29T10:57:24.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.915+0000 7f49a89ff700 1 -- 172.21.15.111:0/152854140 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49a40b18e0 con 0x7f498c005250 2023-03-29T10:57:24.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.916+0000 7f4992ffd700 1 -- 172.21.15.111:0/152854140 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4994057bc0 con 0x7f498c005250 2023-03-29T10:57:24.916 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-03-29T10:57:24.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.918+0000 7f49a89ff700 1 -- 172.21.15.111:0/152854140 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4984069de0 msgr2=0x7f498406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:24.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.918+0000 7f49a89ff700 1 --2- 172.21.15.111:0/152854140 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4984069de0 0x7f498406c290 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f49a415b730 tx=0x7f4998009040 comp rx=0 tx=0).stop 2023-03-29T10:57:24.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.918+0000 7f49a89ff700 1 -- 172.21.15.111:0/152854140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f498c005250 msgr2=0x7f49a414a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:24.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.918+0000 7f49a89ff700 1 --2- 172.21.15.111:0/152854140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f498c005250 0x7f49a414a330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4994010810 tx=0x7f4994010bd0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.918+0000 7f49a89ff700 1 -- 172.21.15.111:0/152854140 shutdown_connections 2023-03-29T10:57:24.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.918+0000 7f49a89ff700 1 --2- 172.21.15.111:0/152854140 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4984069de0 0x7f498406c290 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.918+0000 7f49a89ff700 1 --2- 172.21.15.111:0/152854140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f498c005250 0x7f49a414a330 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.918+0000 7f49a89ff700 1 --2- 172.21.15.111:0/152854140 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f49a415d3b0 0x7f49a415d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.918+0000 7f49a89ff700 1 --2- 172.21.15.111:0/152854140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a40b6e60 0x7f49a40b72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:24.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.918+0000 7f49a89ff700 1 -- 172.21.15.111:0/152854140 >> 172.21.15.111:0/152854140 conn(0x7f49a4071b10 msgr2=0x7f49a40cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:24.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.918+0000 7f49a89ff700 1 -- 172.21.15.111:0/152854140 shutdown_connections 2023-03-29T10:57:24.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:24.918+0000 7f49a89ff700 1 -- 172.21.15.111:0/152854140 wait complete. 2023-03-29T10:57:24.935 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-29T10:57:25.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.120+0000 7fec4c7ff700 1 Processor -- start 2023-03-29T10:57:25.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.120+0000 7fec4c7ff700 1 -- start start 2023-03-29T10:57:25.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.121+0000 7fec4c7ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fec440b87a0 0x7fec440cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:25.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.121+0000 7fec4c7ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fec44160d40 0x7fec440d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:25.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.121+0000 7fec4c7ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec440cdc60 con 0x7fec441601e0 2023-03-29T10:57:25.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.121+0000 7fec4c7ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec440d02d0 con 0x7fec440b87a0 2023-03-29T10:57:25.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.121+0000 7fec4c7ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec440d0cb0 con 0x7fec44160d40 2023-03-29T10:57:25.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.121+0000 7fec4a59b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fec440b87a0 0x7fec440cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:25.123 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.121+0000 7fec49d9a700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fec441601e0 0x7fec44160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53628/0 (socket says 172.21.15.111:53628) 2023-03-29T10:57:25.123 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.121+0000 7fec49d9a700 1 -- 172.21.15.111:0/1024344291 learned_addr learned my addr 172.21.15.111:0/1024344291 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:25.123 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.122+0000 7fec4ad9c700 1 --2- 172.21.15.111:0/1024344291 >> v2:172.21.15.119:3300/0 conn(0x7fec44160d40 0x7fec440d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:25.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.122+0000 7fec49599700 1 -- 172.21.15.111:0/1024344291 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2648433202 0 0) 0x7fec440cdc60 con 0x7fec441601e0 2023-03-29T10:57:25.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.122+0000 7fec49599700 1 -- 172.21.15.111:0/1024344291 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec300035b0 con 0x7fec441601e0 2023-03-29T10:57:25.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.122+0000 7fec4a59b700 1 -- 172.21.15.111:0/1024344291 >> v2:172.21.15.119:3300/0 conn(0x7fec44160d40 msgr2=0x7fec440d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:25.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.122+0000 7fec4a59b700 1 --2- 172.21.15.111:0/1024344291 >> v2:172.21.15.119:3300/0 conn(0x7fec44160d40 0x7fec440d1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.122+0000 7fec4a59b700 1 -- 172.21.15.111:0/1024344291 >> v1:172.21.15.111:6789/0 conn(0x7fec441601e0 legacy=0x7fec44160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:25.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.122+0000 7fec4a59b700 1 -- 172.21.15.111:0/1024344291 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec440cd6a0 con 0x7fec440b87a0 2023-03-29T10:57:25.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.122+0000 7fec4a59b700 1 --2- 172.21.15.111:0/1024344291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec440b87a0 0x7fec440cfbf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec440d0820 tx=0x7fec40009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efcedf602e1e79de server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:25.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.122+0000 7fec49599700 1 -- 172.21.15.111:0/1024344291 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec40011830 con 0x7fec440b87a0 2023-03-29T10:57:25.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.122+0000 7fec49599700 1 -- 172.21.15.111:0/1024344291 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec40011990 con 0x7fec440b87a0 2023-03-29T10:57:25.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.122+0000 7fec49599700 1 -- 172.21.15.111:0/1024344291 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec40011d20 con 0x7fec440b87a0 2023-03-29T10:57:25.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.123+0000 7fec4c7ff700 1 -- 172.21.15.111:0/1024344291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec440b87a0 msgr2=0x7fec440cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:25.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.123+0000 7fec4c7ff700 1 --2- 172.21.15.111:0/1024344291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec440b87a0 0x7fec440cfbf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fec440d0820 tx=0x7fec40009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:25.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.123+0000 7fec4c7ff700 1 -- 172.21.15.111:0/1024344291 shutdown_connections 2023-03-29T10:57:25.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.123+0000 7fec4c7ff700 1 --2- 172.21.15.111:0/1024344291 >> v2:172.21.15.119:3300/0 conn(0x7fec44160d40 0x7fec440d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.123+0000 7fec4c7ff700 1 --2- 172.21.15.111:0/1024344291 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec440b87a0 0x7fec440cfbf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.123+0000 7fec4c7ff700 1 -- 172.21.15.111:0/1024344291 >> 172.21.15.111:0/1024344291 conn(0x7fec44071b10 msgr2=0x7fec44071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:25.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.123+0000 7fec4c7ff700 1 -- 172.21.15.111:0/1024344291 shutdown_connections 2023-03-29T10:57:25.126 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.123+0000 7fec4c7ff700 1 -- 172.21.15.111:0/1024344291 wait complete. 2023-03-29T10:57:25.128 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.125+0000 7fec4c7ff700 1 Processor -- start 2023-03-29T10:57:25.128 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.125+0000 7fec4c7ff700 1 -- start start 2023-03-29T10:57:25.128 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.126+0000 7fec4c7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec440b87a0 0x7fec440cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:25.128 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.126+0000 7fec4c7ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fec441601e0 0x7fec440b6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:25.128 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.126+0000 7fec4c7ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec44160d40 0x7fec440b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:25.128 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.126+0000 7fec4c7ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec441f2f60 con 0x7fec44160d40 2023-03-29T10:57:25.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.126+0000 7fec4c7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec441f5920 con 0x7fec440b87a0 2023-03-29T10:57:25.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.126+0000 7fec4c7ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec440ca9f0 con 0x7fec441601e0 2023-03-29T10:57:25.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.126+0000 7fec4ad9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec44160d40 0x7fec440b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:25.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.126+0000 7fec4a59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec440b87a0 0x7fec440cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:25.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.126+0000 7fec49d9a700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fec441601e0 0x7fec440b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:25.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.126+0000 7fec4a59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec440b87a0 0x7fec440cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:52110/0 (socket says 172.21.15.111:52110) 2023-03-29T10:57:25.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.126+0000 7fec4a59b700 1 -- 172.21.15.111:0/2167804605 learned_addr learned my addr 172.21.15.111:0/2167804605 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:25.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.126+0000 7fec4ad9c700 1 -- 172.21.15.111:0/2167804605 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fec441601e0 msgr2=0x7fec440b6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:25.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.126+0000 7fec4ad9c700 1 --2- 172.21.15.111:0/2167804605 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fec441601e0 0x7fec440b6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.130 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.126+0000 7fec4ad9c700 1 -- 172.21.15.111:0/2167804605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec440b87a0 msgr2=0x7fec440cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:25.130 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.126+0000 7fec4ad9c700 1 --2- 172.21.15.111:0/2167804605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec440b87a0 0x7fec440cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.130 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.126+0000 7fec4ad9c700 1 -- 172.21.15.111:0/2167804605 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec400094a0 con 0x7fec44160d40 2023-03-29T10:57:25.130 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.127+0000 7fec4ad9c700 1 --2- 172.21.15.111:0/2167804605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec44160d40 0x7fec440b6ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00eba0 tx=0x7fec3c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:25.130 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.127+0000 7fec3affd700 1 -- 172.21.15.111:0/2167804605 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c009a80 con 0x7fec44160d40 2023-03-29T10:57:25.130 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.127+0000 7fec3affd700 1 -- 172.21.15.111:0/2167804605 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c00f040 con 0x7fec44160d40 2023-03-29T10:57:25.130 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.127+0000 7fec4c7ff700 1 -- 172.21.15.111:0/2167804605 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec440d1640 con 0x7fec44160d40 2023-03-29T10:57:25.130 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.127+0000 7fec3affd700 1 -- 172.21.15.111:0/2167804605 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c00cc40 con 0x7fec44160d40 2023-03-29T10:57:25.130 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.127+0000 7fec4c7ff700 1 -- 172.21.15.111:0/2167804605 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec440cd980 con 0x7fec44160d40 2023-03-29T10:57:25.131 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.128+0000 7fec4c7ff700 1 -- 172.21.15.111:0/2167804605 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec440d17d0 con 0x7fec44160d40 2023-03-29T10:57:25.131 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.129+0000 7fec3affd700 1 -- 172.21.15.111:0/2167804605 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fec3c014480 con 0x7fec44160d40 2023-03-29T10:57:25.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.130+0000 7fec3affd700 1 --2- 172.21.15.111:0/2167804605 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fec28069de0 0x7fec2806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:25.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.130+0000 7fec3affd700 1 -- 172.21.15.111:0/2167804605 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fec3c088610 con 0x7fec44160d40 2023-03-29T10:57:25.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.130+0000 7fec4a59b700 1 --2- 172.21.15.111:0/2167804605 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fec28069de0 0x7fec2806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:25.132 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.131+0000 7fec4a59b700 1 --2- 172.21.15.111:0/2167804605 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fec28069de0 0x7fec2806c290 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fec44158740 tx=0x7fec4001c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:25.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.134+0000 7fec3affd700 1 -- 172.21.15.111:0/2167804605 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fec3c057c30 con 0x7fec44160d40 2023-03-29T10:57:25.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.329+0000 7fec4c7ff700 1 -- 172.21.15.111:0/2167804605 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fec44074430 con 0x7fec44160d40 2023-03-29T10:57:25.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.329+0000 7fec3affd700 1 -- 172.21.15.111:0/2167804605 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7fec3c018020 con 0x7fec44160d40 2023-03-29T10:57:25.330 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 8 2023-03-29T10:57:25.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.331+0000 7fec4c7ff700 1 -- 172.21.15.111:0/2167804605 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fec28069de0 msgr2=0x7fec2806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:25.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.331+0000 7fec4c7ff700 1 --2- 172.21.15.111:0/2167804605 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fec28069de0 0x7fec2806c290 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fec44158740 tx=0x7fec4001c040 comp rx=0 tx=0).stop 2023-03-29T10:57:25.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.331+0000 7fec4c7ff700 1 -- 172.21.15.111:0/2167804605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec44160d40 msgr2=0x7fec440b6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:25.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.331+0000 7fec4c7ff700 1 --2- 172.21.15.111:0/2167804605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec44160d40 0x7fec440b6ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00eba0 tx=0x7fec3c00eeb0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.331+0000 7fec4c7ff700 1 -- 172.21.15.111:0/2167804605 shutdown_connections 2023-03-29T10:57:25.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.331+0000 7fec4c7ff700 1 --2- 172.21.15.111:0/2167804605 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fec28069de0 0x7fec2806c290 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.331+0000 7fec4c7ff700 1 --2- 172.21.15.111:0/2167804605 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fec44160d40 0x7fec440b6ad0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.331+0000 7fec4c7ff700 1 --2- 172.21.15.111:0/2167804605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec440b87a0 0x7fec440cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.331+0000 7fec4c7ff700 1 --2- 172.21.15.111:0/2167804605 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fec441601e0 0x7fec440b6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.331+0000 7fec4c7ff700 1 -- 172.21.15.111:0/2167804605 >> 172.21.15.111:0/2167804605 conn(0x7fec44071b10 msgr2=0x7fec44160630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:25.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.331+0000 7fec4c7ff700 1 -- 172.21.15.111:0/2167804605 shutdown_connections 2023-03-29T10:57:25.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.332+0000 7fec4c7ff700 1 -- 172.21.15.111:0/2167804605 wait complete. 2023-03-29T10:57:25.349 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-03-29T10:57:25.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.530+0000 7f59ebbff700 1 Processor -- start 2023-03-29T10:57:25.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.530+0000 7f59ebbff700 1 -- start start 2023-03-29T10:57:25.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.531+0000 7f59ebbff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f59e40cfd40 0x7f59e40d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:25.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.531+0000 7f59ebbff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59e40b70f0 0x7f59e40b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:25.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.531+0000 7f59ebbff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59e40cdc10 con 0x7f59e40cf970 2023-03-29T10:57:25.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.531+0000 7f59ebbff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59e40b6c60 con 0x7f59e40cfd40 2023-03-29T10:57:25.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.531+0000 7f59ebbff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e40cd650 con 0x7f59e40b70f0 2023-03-29T10:57:25.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.531+0000 7f59e999b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f59e40cf970 0x7f59e40cf110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53632/0 (socket says 172.21.15.111:53632) 2023-03-29T10:57:25.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.531+0000 7f59e999b700 1 -- 172.21.15.111:0/2761672049 learned_addr learned my addr 172.21.15.111:0/2761672049 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:25.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.531+0000 7f59e919a700 1 --2- 172.21.15.111:0/2761672049 >> v2:172.21.15.111:3300/0 conn(0x7f59e40cfd40 0x7f59e40d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:25.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.531+0000 7f59ea19c700 1 --2- 172.21.15.111:0/2761672049 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59e40b70f0 0x7f59e40b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:25.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.532+0000 7f59e8999700 1 -- 172.21.15.111:0/2761672049 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093414182 0 0) 0x7f59e40cdc10 con 0x7f59e40cf970 2023-03-29T10:57:25.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.532+0000 7f59e8999700 1 -- 172.21.15.111:0/2761672049 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59d00035b0 con 0x7f59e40cf970 2023-03-29T10:57:25.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.532+0000 7f59e919a700 1 -- 172.21.15.111:0/2761672049 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59e40b70f0 msgr2=0x7f59e40b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:25.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.532+0000 7f59e919a700 1 --2- 172.21.15.111:0/2761672049 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59e40b70f0 0x7f59e40b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.532+0000 7f59e919a700 1 -- 172.21.15.111:0/2761672049 >> v1:172.21.15.111:6789/0 conn(0x7f59e40cf970 legacy=0x7f59e40cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:25.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.532+0000 7f59e919a700 1 -- 172.21.15.111:0/2761672049 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e40cd930 con 0x7f59e40cfd40 2023-03-29T10:57:25.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.532+0000 7f59e919a700 1 --2- 172.21.15.111:0/2761672049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e40cfd40 0x7f59e40d01c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009840 tx=0x7f59d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce57ca549222b6a server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:25.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.532+0000 7f59e8999700 1 -- 172.21.15.111:0/2761672049 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4011830 con 0x7f59e40cfd40 2023-03-29T10:57:25.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.532+0000 7f59e8999700 1 -- 172.21.15.111:0/2761672049 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d4011990 con 0x7f59e40cfd40 2023-03-29T10:57:25.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.532+0000 7f59e8999700 1 -- 172.21.15.111:0/2761672049 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4011d20 con 0x7f59e40cfd40 2023-03-29T10:57:25.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.533+0000 7f59ebbff700 1 -- 172.21.15.111:0/2761672049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e40cfd40 msgr2=0x7f59e40d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:25.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.533+0000 7f59ebbff700 1 --2- 172.21.15.111:0/2761672049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e40cfd40 0x7f59e40d01c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009840 tx=0x7f59d4009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:25.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.533+0000 7f59ebbff700 1 -- 172.21.15.111:0/2761672049 shutdown_connections 2023-03-29T10:57:25.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.533+0000 7f59ebbff700 1 --2- 172.21.15.111:0/2761672049 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59e40b70f0 0x7f59e40b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.533+0000 7f59ebbff700 1 --2- 172.21.15.111:0/2761672049 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e40cfd40 0x7f59e40d01c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.533+0000 7f59ebbff700 1 -- 172.21.15.111:0/2761672049 >> 172.21.15.111:0/2761672049 conn(0x7f59e4071b10 msgr2=0x7f59e4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:25.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.533+0000 7f59ebbff700 1 -- 172.21.15.111:0/2761672049 shutdown_connections 2023-03-29T10:57:25.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.533+0000 7f59ebbff700 1 -- 172.21.15.111:0/2761672049 wait complete. 2023-03-29T10:57:25.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.535+0000 7f59ebbff700 1 Processor -- start 2023-03-29T10:57:25.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.535+0000 7f59ebbff700 1 -- start start 2023-03-29T10:57:25.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.536+0000 7f59ebbff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59e40b70f0 0x7f59e41f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:25.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.536+0000 7f59ebbff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59e40cf970 0x7f59e40b7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:25.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.536+0000 7f59ebbff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e40cfd40 0x7f59e40b8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:25.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.536+0000 7f59ebbff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e406c7f0 con 0x7f59e40cfd40 2023-03-29T10:57:25.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.536+0000 7f59ebbff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e40c73e0 con 0x7f59e40b70f0 2023-03-29T10:57:25.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.536+0000 7f59ebbff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e40c76c0 con 0x7f59e40cf970 2023-03-29T10:57:25.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.536+0000 7f59ea19c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e40cfd40 0x7f59e40b8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:25.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.536+0000 7f59ea19c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e40cfd40 0x7f59e40b8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50900/0 (socket says 172.21.15.111:50900) 2023-03-29T10:57:25.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.536+0000 7f59ea19c700 1 -- 172.21.15.111:0/1724833268 learned_addr learned my addr 172.21.15.111:0/1724833268 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:25.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.536+0000 7f59e999b700 1 --2- 172.21.15.111:0/1724833268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59e40b70f0 0x7f59e41f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:25.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.536+0000 7f59e919a700 1 --2- 172.21.15.111:0/1724833268 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59e40cf970 0x7f59e40b7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:25.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.536+0000 7f59ea19c700 1 -- 172.21.15.111:0/1724833268 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59e40cf970 msgr2=0x7f59e40b7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:25.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.536+0000 7f59ea19c700 1 --2- 172.21.15.111:0/1724833268 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59e40cf970 0x7f59e40b7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.536+0000 7f59ea19c700 1 -- 172.21.15.111:0/1724833268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59e40b70f0 msgr2=0x7f59e41f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:25.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.536+0000 7f59ea19c700 1 --2- 172.21.15.111:0/1724833268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59e40b70f0 0x7f59e41f5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.536+0000 7f59ea19c700 1 -- 172.21.15.111:0/1724833268 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d40094a0 con 0x7f59e40cfd40 2023-03-29T10:57:25.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.536+0000 7f59e999b700 1 --2- 172.21.15.111:0/1724833268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59e40b70f0 0x7f59e41f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:57:25.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.537+0000 7f59ea19c700 1 --2- 172.21.15.111:0/1724833268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e40cfd40 0x7f59e40b8270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00efc0 tx=0x7f59dc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:25.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.537+0000 7f59da7fc700 1 -- 172.21.15.111:0/1724833268 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc009ea0 con 0x7f59e40cfd40 2023-03-29T10:57:25.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.537+0000 7f59da7fc700 1 -- 172.21.15.111:0/1724833268 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59dc00f040 con 0x7f59e40cfd40 2023-03-29T10:57:25.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.537+0000 7f59ebbff700 1 -- 172.21.15.111:0/1724833268 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e40d0700 con 0x7f59e40cfd40 2023-03-29T10:57:25.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.537+0000 7f59da7fc700 1 -- 172.21.15.111:0/1724833268 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc014480 con 0x7f59e40cfd40 2023-03-29T10:57:25.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.537+0000 7f59ebbff700 1 -- 172.21.15.111:0/1724833268 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e40b68f0 con 0x7f59e40cfd40 2023-03-29T10:57:25.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.538+0000 7f59ebbff700 1 -- 172.21.15.111:0/1724833268 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e40d36d0 con 0x7f59e40cfd40 2023-03-29T10:57:25.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.539+0000 7f59da7fc700 1 -- 172.21.15.111:0/1724833268 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f59dc0145e0 con 0x7f59e40cfd40 2023-03-29T10:57:25.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.540+0000 7f59da7fc700 1 --2- 172.21.15.111:0/1724833268 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f59c8069de0 0x7f59c806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:25.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.540+0000 7f59da7fc700 1 -- 172.21.15.111:0/1724833268 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f59dc088510 con 0x7f59e40cfd40 2023-03-29T10:57:25.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.540+0000 7f59e999b700 1 --2- 172.21.15.111:0/1724833268 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f59c8069de0 0x7f59c806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:25.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.540+0000 7f59e999b700 1 --2- 172.21.15.111:0/1724833268 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f59c8069de0 0x7f59c806c290 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f59e00011e0 tx=0x7f59e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:25.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.544+0000 7f59da7fc700 1 -- 172.21.15.111:0/1724833268 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f59dc054100 con 0x7f59e40cfd40 2023-03-29T10:57:25.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.737+0000 7f59ebbff700 1 -- 172.21.15.111:0/1724833268 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f59e4074430 con 0x7f59e40cfd40 2023-03-29T10:57:25.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.737+0000 7f59da7fc700 1 -- 172.21.15.111:0/1724833268 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v20) v1 ==== 84+0+126 (secure 0 0 0) 0x7f59dc057720 con 0x7f59e40cfd40 2023-03-29T10:57:25.738 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:25.738 INFO:teuthology.orchestra.run.smithi111.stdout:[ 2023-03-29T10:57:25.738 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:57:25.738 INFO:teuthology.orchestra.run.smithi111.stdout: "poolnum": 1, 2023-03-29T10:57:25.738 INFO:teuthology.orchestra.run.smithi111.stdout: "poolname": ".mgr" 2023-03-29T10:57:25.738 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:57:25.738 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:57:25.738 INFO:teuthology.orchestra.run.smithi111.stdout: "poolnum": 2, 2023-03-29T10:57:25.739 INFO:teuthology.orchestra.run.smithi111.stdout: "poolname": "rbd" 2023-03-29T10:57:25.739 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:57:25.739 INFO:teuthology.orchestra.run.smithi111.stdout:] 2023-03-29T10:57:25.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.739+0000 7f59ebbff700 1 -- 172.21.15.111:0/1724833268 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f59c8069de0 msgr2=0x7f59c806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:25.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.739+0000 7f59ebbff700 1 --2- 172.21.15.111:0/1724833268 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f59c8069de0 0x7f59c806c290 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f59e00011e0 tx=0x7f59e0009040 comp rx=0 tx=0).stop 2023-03-29T10:57:25.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.739+0000 7f59ebbff700 1 -- 172.21.15.111:0/1724833268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e40cfd40 msgr2=0x7f59e40b8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:25.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.739+0000 7f59ebbff700 1 --2- 172.21.15.111:0/1724833268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e40cfd40 0x7f59e40b8270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00efc0 tx=0x7f59dc00c3a0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.740+0000 7f59ebbff700 1 -- 172.21.15.111:0/1724833268 shutdown_connections 2023-03-29T10:57:25.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.740+0000 7f59ebbff700 1 --2- 172.21.15.111:0/1724833268 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f59c8069de0 0x7f59c806c290 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.740+0000 7f59ebbff700 1 --2- 172.21.15.111:0/1724833268 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f59e40cfd40 0x7f59e40b8270 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.740+0000 7f59ebbff700 1 --2- 172.21.15.111:0/1724833268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59e40b70f0 0x7f59e41f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.740+0000 7f59ebbff700 1 --2- 172.21.15.111:0/1724833268 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f59e40cf970 0x7f59e40b7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.740+0000 7f59ebbff700 1 -- 172.21.15.111:0/1724833268 >> 172.21.15.111:0/1724833268 conn(0x7f59e4071b10 msgr2=0x7f59e4072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:25.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.740+0000 7f59ebbff700 1 -- 172.21.15.111:0/1724833268 shutdown_connections 2023-03-29T10:57:25.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.740+0000 7f59ebbff700 1 -- 172.21.15.111:0/1724833268 wait complete. 2023-03-29T10:57:25.757 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-03-29T10:57:25.758 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-03-29T10:57:25.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.942+0000 7fe78bfff700 1 Processor -- start 2023-03-29T10:57:25.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.942+0000 7fe78bfff700 1 -- start start 2023-03-29T10:57:25.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.943+0000 7fe78bfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe7840cf010 0x7fe784160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:25.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.943+0000 7fe78bfff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7840d0270 con 0x7fe78415c620 2023-03-29T10:57:25.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.943+0000 7fe78bfff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7840d2510 con 0x7fe7840cfe00 2023-03-29T10:57:25.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.943+0000 7fe78bfff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7840d27f0 con 0x7fe7840cf010 2023-03-29T10:57:25.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.943+0000 7fe789d9b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe78415c620 0x7fe7840cf6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53644/0 (socket says 172.21.15.111:53644) 2023-03-29T10:57:25.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.943+0000 7fe789d9b700 1 -- 172.21.15.111:0/2519358013 learned_addr learned my addr 172.21.15.111:0/2519358013 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:25.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.943+0000 7fe78a59c700 1 --2- 172.21.15.111:0/2519358013 >> v2:172.21.15.111:3300/0 conn(0x7fe7840cf010 0x7fe784160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:25.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.944+0000 7fe788d99700 1 -- 172.21.15.111:0/2519358013 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1182206148 0 0) 0x7fe780001110 con 0x7fe78415c620 2023-03-29T10:57:25.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.944+0000 7fe788d99700 1 -- 172.21.15.111:0/2519358013 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7700035b0 con 0x7fe78415c620 2023-03-29T10:57:25.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.944+0000 7fe788d99700 1 -- 172.21.15.111:0/2519358013 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008779914 0 0) 0x7fe7840d2510 con 0x7fe7840cfe00 2023-03-29T10:57:25.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.944+0000 7fe78a59c700 1 -- 172.21.15.111:0/2519358013 >> v1:172.21.15.119:6789/0 conn(0x7fe7840cfe00 legacy=0x7fe7840d2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:25.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.944+0000 7fe78a59c700 1 -- 172.21.15.111:0/2519358013 >> v1:172.21.15.111:6789/0 conn(0x7fe78415c620 legacy=0x7fe7840cf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:25.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.944+0000 7fe78a59c700 1 -- 172.21.15.111:0/2519358013 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7841610d0 con 0x7fe7840cf010 2023-03-29T10:57:25.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.944+0000 7fe78a59c700 1 --2- 172.21.15.111:0/2519358013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7840cf010 0x7fe784160b90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe77c009d90 tx=0x7fe77c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d4e43a40564da5 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:25.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.944+0000 7fe788d99700 1 -- 172.21.15.111:0/2519358013 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c00e070 con 0x7fe7840cf010 2023-03-29T10:57:25.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.944+0000 7fe788d99700 1 -- 172.21.15.111:0/2519358013 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77c00bb20 con 0x7fe7840cf010 2023-03-29T10:57:25.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.944+0000 7fe788d99700 1 -- 172.21.15.111:0/2519358013 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77c00bed0 con 0x7fe7840cf010 2023-03-29T10:57:25.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.945+0000 7fe78bfff700 1 -- 172.21.15.111:0/2519358013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7840cf010 msgr2=0x7fe784160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:25.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.945+0000 7fe78bfff700 1 --2- 172.21.15.111:0/2519358013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7840cf010 0x7fe784160b90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe77c009d90 tx=0x7fe77c00b330 comp rx=0 tx=0).stop 2023-03-29T10:57:25.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.945+0000 7fe78bfff700 1 -- 172.21.15.111:0/2519358013 shutdown_connections 2023-03-29T10:57:25.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.945+0000 7fe78bfff700 1 --2- 172.21.15.111:0/2519358013 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7840cf010 0x7fe784160b90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.945+0000 7fe78bfff700 1 -- 172.21.15.111:0/2519358013 >> 172.21.15.111:0/2519358013 conn(0x7fe784071b10 msgr2=0x7fe784071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:25.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.945+0000 7fe78bfff700 1 -- 172.21.15.111:0/2519358013 shutdown_connections 2023-03-29T10:57:25.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.945+0000 7fe78bfff700 1 -- 172.21.15.111:0/2519358013 wait complete. 2023-03-29T10:57:25.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.947+0000 7fe78bfff700 1 Processor -- start 2023-03-29T10:57:25.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.947+0000 7fe78bfff700 1 -- start start 2023-03-29T10:57:25.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.948+0000 7fe78bfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7840cf010 0x7fe7840a98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:25.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.948+0000 7fe78bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7840cfe00 0x7fe78415b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:25.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.948+0000 7fe78bfff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe78415c620 0x7fe7840b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:25.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.948+0000 7fe78bfff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe78415b930 con 0x7fe7840cf010 2023-03-29T10:57:25.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.948+0000 7fe78bfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7841f36b0 con 0x7fe7840cfe00 2023-03-29T10:57:25.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.948+0000 7fe78bfff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7841f3990 con 0x7fe78415c620 2023-03-29T10:57:25.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.948+0000 7fe789d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7840cf010 0x7fe7840a98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:25.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.948+0000 7fe789d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7840cf010 0x7fe7840a98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50922/0 (socket says 172.21.15.111:50922) 2023-03-29T10:57:25.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.948+0000 7fe789d9b700 1 -- 172.21.15.111:0/198988275 learned_addr learned my addr 172.21.15.111:0/198988275 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:25.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.948+0000 7fe78a59c700 1 --2- 172.21.15.111:0/198988275 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe78415c620 0x7fe7840b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:25.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.948+0000 7fe78959a700 1 --2- 172.21.15.111:0/198988275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7840cfe00 0x7fe78415b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:25.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.948+0000 7fe789d9b700 1 -- 172.21.15.111:0/198988275 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe78415c620 msgr2=0x7fe7840b7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:25.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.948+0000 7fe789d9b700 1 --2- 172.21.15.111:0/198988275 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe78415c620 0x7fe7840b7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.948+0000 7fe789d9b700 1 -- 172.21.15.111:0/198988275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7840cfe00 msgr2=0x7fe78415b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:25.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.948+0000 7fe789d9b700 1 --2- 172.21.15.111:0/198988275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7840cfe00 0x7fe78415b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:25.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.948+0000 7fe789d9b700 1 -- 172.21.15.111:0/198988275 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77c009a70 con 0x7fe7840cf010 2023-03-29T10:57:25.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.948+0000 7fe789d9b700 1 --2- 172.21.15.111:0/198988275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7840cf010 0x7fe7840a98c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe78000c4d0 tx=0x7fe78000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:25.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.949+0000 7fe77a7fc700 1 -- 172.21.15.111:0/198988275 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780005d70 con 0x7fe7840cf010 2023-03-29T10:57:25.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.949+0000 7fe77a7fc700 1 -- 172.21.15.111:0/198988275 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7800070d0 con 0x7fe7840cf010 2023-03-29T10:57:25.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.949+0000 7fe78bfff700 1 -- 172.21.15.111:0/198988275 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7840cf430 con 0x7fe7840cf010 2023-03-29T10:57:25.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.949+0000 7fe77a7fc700 1 -- 172.21.15.111:0/198988275 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe780007480 con 0x7fe7840cf010 2023-03-29T10:57:25.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.949+0000 7fe78bfff700 1 -- 172.21.15.111:0/198988275 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe78415c9f0 con 0x7fe7840cf010 2023-03-29T10:57:25.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.950+0000 7fe78bfff700 1 -- 172.21.15.111:0/198988275 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7840cd650 con 0x7fe7840cf010 2023-03-29T10:57:25.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.951+0000 7fe77a7fc700 1 -- 172.21.15.111:0/198988275 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fe7800075e0 con 0x7fe7840cf010 2023-03-29T10:57:25.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.952+0000 7fe77a7fc700 1 --2- 172.21.15.111:0/198988275 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe768069de0 0x7fe76806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:25.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.952+0000 7fe77a7fc700 1 -- 172.21.15.111:0/198988275 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe780086fd0 con 0x7fe7840cf010 2023-03-29T10:57:25.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.952+0000 7fe78959a700 1 --2- 172.21.15.111:0/198988275 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe768069de0 0x7fe76806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:25.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.952+0000 7fe78959a700 1 --2- 172.21.15.111:0/198988275 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe768069de0 0x7fe76806c290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe7840b85e0 tx=0x7fe774009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:25.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:25.956+0000 7fe77a7fc700 1 -- 172.21.15.111:0/198988275 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fe780052c60 con 0x7fe7840cf010 2023-03-29T10:57:26.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.152+0000 7fe78bfff700 1 -- 172.21.15.111:0/198988275 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe7840d0270 con 0x7fe7840cf010 2023-03-29T10:57:26.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.334+0000 7fe77a7fc700 1 -- 172.21.15.111:0/198988275 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v21) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe7840d0270 con 0x7fe7840cf010 2023-03-29T10:57:26.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.336+0000 7fe78bfff700 1 -- 172.21.15.111:0/198988275 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe768069de0 msgr2=0x7fe76806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:26.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.336+0000 7fe78bfff700 1 --2- 172.21.15.111:0/198988275 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe768069de0 0x7fe76806c290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe7840b85e0 tx=0x7fe774009040 comp rx=0 tx=0).stop 2023-03-29T10:57:26.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.336+0000 7fe78bfff700 1 -- 172.21.15.111:0/198988275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7840cf010 msgr2=0x7fe7840a98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:26.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.336+0000 7fe78bfff700 1 --2- 172.21.15.111:0/198988275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7840cf010 0x7fe7840a98c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe78000c4d0 tx=0x7fe78000c890 comp rx=0 tx=0).stop 2023-03-29T10:57:26.337 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.336+0000 7fe78bfff700 1 -- 172.21.15.111:0/198988275 shutdown_connections 2023-03-29T10:57:26.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.336+0000 7fe78bfff700 1 --2- 172.21.15.111:0/198988275 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe768069de0 0x7fe76806c290 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:26.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.336+0000 7fe78bfff700 1 --2- 172.21.15.111:0/198988275 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7840cf010 0x7fe7840a98c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:26.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.336+0000 7fe78bfff700 1 --2- 172.21.15.111:0/198988275 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe78415c620 0x7fe7840b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:26.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.336+0000 7fe78bfff700 1 --2- 172.21.15.111:0/198988275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7840cfe00 0x7fe78415b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:26.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.336+0000 7fe78bfff700 1 -- 172.21.15.111:0/198988275 >> 172.21.15.111:0/198988275 conn(0x7fe784071b10 msgr2=0x7fe7840cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:26.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.337+0000 7fe78bfff700 1 -- 172.21.15.111:0/198988275 shutdown_connections 2023-03-29T10:57:26.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.337+0000 7fe78bfff700 1 -- 172.21.15.111:0/198988275 wait complete. 2023-03-29T10:57:26.341 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_metadata' created 2023-03-29T10:57:26.358 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-03-29T10:57:26.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.548+0000 7fbbe65ff700 1 Processor -- start 2023-03-29T10:57:26.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.548+0000 7fbbe65ff700 1 -- start start 2023-03-29T10:57:26.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.549+0000 7fbbe65ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbbe00cd760 0x7fbbe00cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:26.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.549+0000 7fbbe65ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe00cc9e0 con 0x7fbbe00adf70 2023-03-29T10:57:26.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.549+0000 7fbbe65ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe015de20 con 0x7fbbe015cf10 2023-03-29T10:57:26.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.549+0000 7fbbe65ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00cd580 con 0x7fbbe00cd760 2023-03-29T10:57:26.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.550+0000 7fbbd7fff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fbbe00adf70 0x7fbbe00cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53654/0 (socket says 172.21.15.111:53654) 2023-03-29T10:57:26.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.550+0000 7fbbd7fff700 1 -- 172.21.15.111:0/4199660591 learned_addr learned my addr 172.21.15.111:0/4199660591 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.550+0000 7fbbdffff700 1 --2- 172.21.15.111:0/4199660591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbbe00cd760 0x7fbbe00cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.550+0000 7fbbdf7fe700 1 -- 172.21.15.111:0/4199660591 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339837911 0 0) 0x7fbbe00cc9e0 con 0x7fbbe00adf70 2023-03-29T10:57:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.550+0000 7fbbdf7fe700 1 -- 172.21.15.111:0/4199660591 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbcc0035b0 con 0x7fbbe00adf70 2023-03-29T10:57:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.550+0000 7fbbdf7fe700 1 -- 172.21.15.111:0/4199660591 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 65439351 0 0) 0x7fbbcc0035b0 con 0x7fbbe00adf70 2023-03-29T10:57:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.550+0000 7fbbdf7fe700 1 -- 172.21.15.111:0/4199660591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbbe00cd760 msgr2=0x7fbbe00cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.550+0000 7fbbdf7fe700 1 --2- 172.21.15.111:0/4199660591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbbe00cd760 0x7fbbe00cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.550+0000 7fbbdf7fe700 1 -- 172.21.15.111:0/4199660591 >> v1:172.21.15.119:6789/0 conn(0x7fbbe015cf10 legacy=0x7fbbe006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.550+0000 7fbbdf7fe700 1 -- 172.21.15.111:0/4199660591 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe00ce070 con 0x7fbbe00adf70 2023-03-29T10:57:26.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.550+0000 7fbbdf7fe700 1 -- 172.21.15.111:0/4199660591 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7fbbc80017f0 con 0x7fbbe00adf70 2023-03-29T10:57:26.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.550+0000 7fbbdf7fe700 1 -- 172.21.15.111:0/4199660591 >> v1:172.21.15.111:6789/0 conn(0x7fbbe00adf70 legacy=0x7fbbe00cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:26.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.550+0000 7fbbdf7fe700 1 --2- 172.21.15.111:0/4199660591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbcc003a30 0x7fbbcc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:26.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.550+0000 7fbbdf7fe700 1 --2- 172.21.15.111:0/4199660591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbcc004850 0x7fbbcc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:26.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.550+0000 7fbbdf7fe700 1 --2- 172.21.15.111:0/4199660591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbbe00cd760 0x7fbbcc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:26.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.550+0000 7fbbdf7fe700 1 -- 172.21.15.111:0/4199660591 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00cd580 con 0x7fbbcc003a30 2023-03-29T10:57:26.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.551+0000 7fbbdf7fe700 1 -- 172.21.15.111:0/4199660591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc00bac0 con 0x7fbbcc004850 2023-03-29T10:57:26.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.551+0000 7fbbdf7fe700 1 -- 172.21.15.111:0/4199660591 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc00bc80 con 0x7fbbe00cd760 2023-03-29T10:57:26.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.551+0000 7fbbdffff700 1 --2- 172.21.15.111:0/4199660591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbcc003a30 0x7fbbcc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:26.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.551+0000 7fbbd7fff700 1 --2- 172.21.15.111:0/4199660591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbcc004850 0x7fbbcc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:26.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.551+0000 7fbbe4b9c700 1 --2- 172.21.15.111:0/4199660591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbbe00cd760 0x7fbbcc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:26.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.551+0000 7fbbdffff700 1 -- 172.21.15.111:0/4199660591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbbe00cd760 msgr2=0x7fbbcc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:26.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.551+0000 7fbbdffff700 1 --2- 172.21.15.111:0/4199660591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbbe00cd760 0x7fbbcc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:26.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.551+0000 7fbbdffff700 1 -- 172.21.15.111:0/4199660591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbcc004850 msgr2=0x7fbbcc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:26.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.551+0000 7fbbdffff700 1 --2- 172.21.15.111:0/4199660591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbcc004850 0x7fbbcc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:26.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.551+0000 7fbbdffff700 1 -- 172.21.15.111:0/4199660591 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc0035b0 con 0x7fbbcc003a30 2023-03-29T10:57:26.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.551+0000 7fbbdffff700 1 --2- 172.21.15.111:0/4199660591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbcc003a30 0x7fbbcc003e00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009890 tx=0x7fbbcc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57a2eaf3b38b8b2 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:26.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.552+0000 7fbbdf7fe700 1 -- 172.21.15.111:0/4199660591 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd000c070 con 0x7fbbcc003a30 2023-03-29T10:57:26.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.552+0000 7fbbdf7fe700 1 -- 172.21.15.111:0/4199660591 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd0013610 con 0x7fbbcc003a30 2023-03-29T10:57:26.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.552+0000 7fbbdf7fe700 1 -- 172.21.15.111:0/4199660591 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00139a0 con 0x7fbbcc003a30 2023-03-29T10:57:26.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.552+0000 7fbbe65ff700 1 -- 172.21.15.111:0/4199660591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbcc003a30 msgr2=0x7fbbcc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:26.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.552+0000 7fbbe65ff700 1 --2- 172.21.15.111:0/4199660591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbcc003a30 0x7fbbcc003e00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009890 tx=0x7fbbcc005880 comp rx=0 tx=0).stop 2023-03-29T10:57:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.552+0000 7fbbe4b9c700 1 -- 172.21.15.111:0/4199660591 reap_dead start 2023-03-29T10:57:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.552+0000 7fbbe65ff700 1 -- 172.21.15.111:0/4199660591 shutdown_connections 2023-03-29T10:57:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.552+0000 7fbbe65ff700 1 -- 172.21.15.111:0/4199660591 >> 172.21.15.111:0/4199660591 conn(0x7fbbe00cf720 msgr2=0x7fbbe00cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.552+0000 7fbbe65ff700 1 -- 172.21.15.111:0/4199660591 shutdown_connections 2023-03-29T10:57:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.552+0000 7fbbe65ff700 1 -- 172.21.15.111:0/4199660591 wait complete. 2023-03-29T10:57:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.554+0000 7fbbe65ff700 1 Processor -- start 2023-03-29T10:57:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.554+0000 7fbbe65ff700 1 -- start start 2023-03-29T10:57:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.555+0000 7fbbe65ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbbcc003a30 0x7fbbe0150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.555+0000 7fbbe65ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbe00b40d0 0x7fbbe00b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.555+0000 7fbbe65ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbe00b4540 0x7fbbe00b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.555+0000 7fbbe65ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe015c2b0 con 0x7fbbe00b40d0 2023-03-29T10:57:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.555+0000 7fbbe65ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe015b630 con 0x7fbbe00b4540 2023-03-29T10:57:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.555+0000 7fbbe65ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe015b910 con 0x7fbbcc003a30 2023-03-29T10:57:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.555+0000 7fbbd7fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbe00b40d0 0x7fbbe00b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:26.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.555+0000 7fbbd7fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbe00b40d0 0x7fbbe00b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50944/0 (socket says 172.21.15.111:50944) 2023-03-29T10:57:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.555+0000 7fbbd7fff700 1 -- 172.21.15.111:0/2039932024 learned_addr learned my addr 172.21.15.111:0/2039932024 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.555+0000 7fbbdffff700 1 --2- 172.21.15.111:0/2039932024 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbbcc003a30 0x7fbbe0150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.555+0000 7fbbe4b9c700 1 --2- 172.21.15.111:0/2039932024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbe00b4540 0x7fbbe00b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.555+0000 7fbbd7fff700 1 -- 172.21.15.111:0/2039932024 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbbcc003a30 msgr2=0x7fbbe0150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.555+0000 7fbbd7fff700 1 --2- 172.21.15.111:0/2039932024 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbbcc003a30 0x7fbbe0150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.556+0000 7fbbd7fff700 1 -- 172.21.15.111:0/2039932024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbe00b4540 msgr2=0x7fbbe00b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.556+0000 7fbbd7fff700 1 --2- 172.21.15.111:0/2039932024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbe00b4540 0x7fbbe00b2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.556+0000 7fbbd7fff700 1 -- 172.21.15.111:0/2039932024 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe015db40 con 0x7fbbe00b40d0 2023-03-29T10:57:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.556+0000 7fbbe4b9c700 1 --2- 172.21.15.111:0/2039932024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbe00b4540 0x7fbbe00b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:57:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.556+0000 7fbbd7fff700 1 --2- 172.21.15.111:0/2039932024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbe00b40d0 0x7fbbe00b2040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80011d0 tx=0x7fbbc800fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.556+0000 7fbbdd7fa700 1 -- 172.21.15.111:0/2039932024 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc80072e0 con 0x7fbbe00b40d0 2023-03-29T10:57:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.556+0000 7fbbdd7fa700 1 -- 172.21.15.111:0/2039932024 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc8007440 con 0x7fbbe00b40d0 2023-03-29T10:57:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.556+0000 7fbbe65ff700 1 -- 172.21.15.111:0/2039932024 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe015d860 con 0x7fbbe00b40d0 2023-03-29T10:57:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.556+0000 7fbbdd7fa700 1 -- 172.21.15.111:0/2039932024 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8005830 con 0x7fbbe00b40d0 2023-03-29T10:57:26.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.556+0000 7fbbe65ff700 1 -- 172.21.15.111:0/2039932024 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe00cc670 con 0x7fbbe00b40d0 2023-03-29T10:57:26.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.557+0000 7fbbe65ff700 1 -- 172.21.15.111:0/2039932024 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe0071b50 con 0x7fbbe00b40d0 2023-03-29T10:57:26.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.558+0000 7fbbdd7fa700 1 -- 172.21.15.111:0/2039932024 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fbbc8005990 con 0x7fbbe00b40d0 2023-03-29T10:57:26.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.559+0000 7fbbdd7fa700 1 --2- 172.21.15.111:0/2039932024 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbbc0069de0 0x7fbbc006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:26.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.559+0000 7fbbdd7fa700 1 -- 172.21.15.111:0/2039932024 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fbbc8087da0 con 0x7fbbe00b40d0 2023-03-29T10:57:26.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.559+0000 7fbbdffff700 1 --2- 172.21.15.111:0/2039932024 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbbc0069de0 0x7fbbc006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:26.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.560+0000 7fbbdffff700 1 --2- 172.21.15.111:0/2039932024 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbbc0069de0 0x7fbbc006c290 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000a010 tx=0x7fbbd0020040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:26.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.564+0000 7fbbdd7fa700 1 -- 172.21.15.111:0/2039932024 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fbbc80537a0 con 0x7fbbe00b40d0 2023-03-29T10:57:26.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:26.758+0000 7fbbe65ff700 1 -- 172.21.15.111:0/2039932024 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fbbe0070fe0 con 0x7fbbe00b40d0 2023-03-29T10:57:27.346 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.345+0000 7fbbdd7fa700 1 -- 172.21.15.111:0/2039932024 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v22) v1 ==== 171+0+0 (secure 0 0 0) 0x7fbbc801a310 con 0x7fbbe00b40d0 2023-03-29T10:57:27.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.353+0000 7fbbe65ff700 1 -- 172.21.15.111:0/2039932024 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbbc0069de0 msgr2=0x7fbbc006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:27.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.353+0000 7fbbe65ff700 1 --2- 172.21.15.111:0/2039932024 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbbc0069de0 0x7fbbc006c290 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000a010 tx=0x7fbbd0020040 comp rx=0 tx=0).stop 2023-03-29T10:57:27.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.353+0000 7fbbe65ff700 1 -- 172.21.15.111:0/2039932024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbe00b40d0 msgr2=0x7fbbe00b2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:27.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.353+0000 7fbbe65ff700 1 --2- 172.21.15.111:0/2039932024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbe00b40d0 0x7fbbe00b2040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbbc80011d0 tx=0x7fbbc800fac0 comp rx=0 tx=0).stop 2023-03-29T10:57:27.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.354+0000 7fbbe65ff700 1 -- 172.21.15.111:0/2039932024 shutdown_connections 2023-03-29T10:57:27.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.354+0000 7fbbe65ff700 1 --2- 172.21.15.111:0/2039932024 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbbc0069de0 0x7fbbc006c290 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbbd000a010 tx=0x7fbbd0020040 comp rx=0 tx=0).stop 2023-03-29T10:57:27.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.354+0000 7fbbe65ff700 1 --2- 172.21.15.111:0/2039932024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbe00b4540 0x7fbbe00b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:27.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.354+0000 7fbbe65ff700 1 --2- 172.21.15.111:0/2039932024 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbbcc003a30 0x7fbbe0150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:27.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.354+0000 7fbbe65ff700 1 --2- 172.21.15.111:0/2039932024 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbe00b40d0 0x7fbbe00b2040 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:27.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.355+0000 7fbbe65ff700 1 -- 172.21.15.111:0/2039932024 >> 172.21.15.111:0/2039932024 conn(0x7fbbe00cf720 msgr2=0x7fbbe00cb780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:27.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.358+0000 7fbbe65ff700 1 -- 172.21.15.111:0/2039932024 shutdown_connections 2023-03-29T10:57:27.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.362+0000 7fbbe65ff700 1 -- 172.21.15.111:0/2039932024 wait complete. 2023-03-29T10:57:27.368 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data' created 2023-03-29T10:57:27.385 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-03-29T10:57:27.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.572+0000 7f1edd3ff700 1 Processor -- start 2023-03-29T10:57:27.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.572+0000 7f1edd3ff700 1 -- start start 2023-03-29T10:57:27.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.573+0000 7f1edd3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ed815d460 0x7f1ed80d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:27.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.573+0000 7f1edd3ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed80cd4c0 con 0x7f1ed815d090 2023-03-29T10:57:27.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.573+0000 7f1edd3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed80cd7a0 con 0x7f1ed815d8d0 2023-03-29T10:57:27.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.573+0000 7f1edd3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed80cda80 con 0x7f1ed815d460 2023-03-29T10:57:27.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.573+0000 7f1ed6ffd700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1ed815d090 0x7f1ed80cfc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53660/0 (socket says 172.21.15.111:53660) 2023-03-29T10:57:27.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.573+0000 7f1ed6ffd700 1 -- 172.21.15.111:0/1850818730 learned_addr learned my addr 172.21.15.111:0/1850818730 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:27.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.574+0000 7f1ece7fc700 1 --2- 172.21.15.111:0/1850818730 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ed815d460 0x7f1ed80d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:27.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.574+0000 7f1ed67fc700 1 -- 172.21.15.111:0/1850818730 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078020036 0 0) 0x7f1ed80cd4c0 con 0x7f1ed815d090 2023-03-29T10:57:27.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.574+0000 7f1ed67fc700 1 -- 172.21.15.111:0/1850818730 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1eb80035b0 con 0x7f1ed815d090 2023-03-29T10:57:27.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.574+0000 7f1ed67fc700 1 -- 172.21.15.111:0/1850818730 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2387474216 0 0) 0x7f1eb80035b0 con 0x7f1ed815d090 2023-03-29T10:57:27.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.574+0000 7f1ed67fc700 1 -- 172.21.15.111:0/1850818730 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ed815d460 msgr2=0x7f1ed80d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:27.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.574+0000 7f1ed67fc700 1 --2- 172.21.15.111:0/1850818730 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ed815d460 0x7f1ed80d2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:27.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.574+0000 7f1ed67fc700 1 -- 172.21.15.111:0/1850818730 >> v1:172.21.15.119:6789/0 conn(0x7f1ed815d8d0 legacy=0x7f1ed80d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:27.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.574+0000 7f1ed67fc700 1 -- 172.21.15.111:0/1850818730 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed80cf440 con 0x7f1ed815d090 2023-03-29T10:57:27.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.574+0000 7f1ed67fc700 1 -- 172.21.15.111:0/1850818730 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f1ec8001670 con 0x7f1ed815d090 2023-03-29T10:57:27.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.574+0000 7f1ed67fc700 1 -- 172.21.15.111:0/1850818730 >> v1:172.21.15.111:6789/0 conn(0x7f1ed815d090 legacy=0x7f1ed80cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:27.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.574+0000 7f1ed67fc700 1 --2- 172.21.15.111:0/1850818730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1eb8003a30 0x7f1eb8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:27.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.574+0000 7f1ed67fc700 1 --2- 172.21.15.111:0/1850818730 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ed815d460 0x7f1eb8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:27.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.574+0000 7f1ed67fc700 1 --2- 172.21.15.111:0/1850818730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1eb8004d90 0x7f1eb8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:27.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.574+0000 7f1ed67fc700 1 -- 172.21.15.111:0/1850818730 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed80cda80 con 0x7f1eb8003a30 2023-03-29T10:57:27.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.574+0000 7f1ed67fc700 1 -- 172.21.15.111:0/1850818730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb800bac0 con 0x7f1eb8004d90 2023-03-29T10:57:27.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.574+0000 7f1ed67fc700 1 -- 172.21.15.111:0/1850818730 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb800bc50 con 0x7f1ed815d460 2023-03-29T10:57:27.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.574+0000 7f1ed6ffd700 1 --2- 172.21.15.111:0/1850818730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1eb8003a30 0x7f1eb8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:27.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.574+0000 7f1ece7fc700 1 --2- 172.21.15.111:0/1850818730 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ed815d460 0x7f1eb8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:27.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.574+0000 7f1ed77fe700 1 --2- 172.21.15.111:0/1850818730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1eb8004d90 0x7f1eb8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:27.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.575+0000 7f1ed6ffd700 1 -- 172.21.15.111:0/1850818730 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ed815d460 msgr2=0x7f1eb8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:27.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.575+0000 7f1ed6ffd700 1 --2- 172.21.15.111:0/1850818730 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ed815d460 0x7f1eb8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:27.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.575+0000 7f1ed6ffd700 1 -- 172.21.15.111:0/1850818730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1eb8004d90 msgr2=0x7f1eb8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:27.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.575+0000 7f1ed6ffd700 1 --2- 172.21.15.111:0/1850818730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1eb8004d90 0x7f1eb8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:27.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.575+0000 7f1ed6ffd700 1 -- 172.21.15.111:0/1850818730 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb80035b0 con 0x7f1eb8003a30 2023-03-29T10:57:27.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.575+0000 7f1ed6ffd700 1 --2- 172.21.15.111:0/1850818730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1eb8003a30 0x7f1eb8003e40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800c240 tx=0x7f1eb8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a1fbeb0cbede88 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:27.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.575+0000 7f1ed67fc700 1 -- 172.21.15.111:0/1850818730 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8007c10 con 0x7f1eb8003a30 2023-03-29T10:57:27.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.575+0000 7f1ed67fc700 1 -- 172.21.15.111:0/1850818730 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec800ce00 con 0x7f1eb8003a30 2023-03-29T10:57:27.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.575+0000 7f1ed67fc700 1 -- 172.21.15.111:0/1850818730 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8005ba0 con 0x7f1eb8003a30 2023-03-29T10:57:27.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.575+0000 7f1edd3ff700 1 -- 172.21.15.111:0/1850818730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1eb8003a30 msgr2=0x7f1eb8003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:27.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.575+0000 7f1edd3ff700 1 --2- 172.21.15.111:0/1850818730 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1eb8003a30 0x7f1eb8003e40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800c240 tx=0x7f1eb8005880 comp rx=0 tx=0).stop 2023-03-29T10:57:27.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.575+0000 7f1ed77fe700 1 -- 172.21.15.111:0/1850818730 reap_dead start 2023-03-29T10:57:27.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.576+0000 7f1edd3ff700 1 -- 172.21.15.111:0/1850818730 shutdown_connections 2023-03-29T10:57:27.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.576+0000 7f1edd3ff700 1 -- 172.21.15.111:0/1850818730 >> 172.21.15.111:0/1850818730 conn(0x7f1ed8071b10 msgr2=0x7f1ed8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:27.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.576+0000 7f1edd3ff700 1 -- 172.21.15.111:0/1850818730 shutdown_connections 2023-03-29T10:57:27.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.576+0000 7f1edd3ff700 1 -- 172.21.15.111:0/1850818730 wait complete. 2023-03-29T10:57:27.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.577+0000 7f1edd3ff700 1 Processor -- start 2023-03-29T10:57:27.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.577+0000 7f1edd3ff700 1 -- start start 2023-03-29T10:57:27.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.578+0000 7f1edd3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1eb8003a30 0x7f1ed81e8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:27.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.578+0000 7f1edd3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ed81e8860 0x7f1ed81593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:27.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.578+0000 7f1edd3ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ed81598e0 0x7f1ed81571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:27.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.578+0000 7f1edd3ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed815a220 con 0x7f1ed81598e0 2023-03-29T10:57:27.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.578+0000 7f1edd3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed8159f40 con 0x7f1eb8003a30 2023-03-29T10:57:27.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.578+0000 7f1edd3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed81545d0 con 0x7f1ed81e8860 2023-03-29T10:57:27.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.579+0000 7f1ed77fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ed81598e0 0x7f1ed81571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:27.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.579+0000 7f1ed77fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ed81598e0 0x7f1ed81571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50960/0 (socket says 172.21.15.111:50960) 2023-03-29T10:57:27.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.579+0000 7f1ed77fe700 1 -- 172.21.15.111:0/60614014 learned_addr learned my addr 172.21.15.111:0/60614014 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:27.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.579+0000 7f1ed6ffd700 1 --2- 172.21.15.111:0/60614014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1eb8003a30 0x7f1ed81e8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:27.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.579+0000 7f1ece7fc700 1 --2- 172.21.15.111:0/60614014 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ed81e8860 0x7f1ed81593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:27.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.579+0000 7f1ed77fe700 1 -- 172.21.15.111:0/60614014 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ed81e8860 msgr2=0x7f1ed81593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:27.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.579+0000 7f1ed77fe700 1 --2- 172.21.15.111:0/60614014 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ed81e8860 0x7f1ed81593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:27.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.579+0000 7f1ed77fe700 1 -- 172.21.15.111:0/60614014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1eb8003a30 msgr2=0x7f1ed81e8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:27.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.579+0000 7f1ed77fe700 1 --2- 172.21.15.111:0/60614014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1eb8003a30 0x7f1ed81e8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:27.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.579+0000 7f1ed77fe700 1 -- 172.21.15.111:0/60614014 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed80cf0c0 con 0x7f1ed81598e0 2023-03-29T10:57:27.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.579+0000 7f1ed77fe700 1 --2- 172.21.15.111:0/60614014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ed81598e0 0x7f1ed81571b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80b1a90 tx=0x7f1ed000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:27.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.579+0000 7f1ecffff700 1 -- 172.21.15.111:0/60614014 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0003c10 con 0x7f1ed81598e0 2023-03-29T10:57:27.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.579+0000 7f1ecffff700 1 -- 172.21.15.111:0/60614014 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed0003d90 con 0x7f1ed81598e0 2023-03-29T10:57:27.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.579+0000 7f1edd3ff700 1 -- 172.21.15.111:0/60614014 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed80cf720 con 0x7f1ed81598e0 2023-03-29T10:57:27.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.579+0000 7f1ecffff700 1 -- 172.21.15.111:0/60614014 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed000dbd0 con 0x7f1ed81598e0 2023-03-29T10:57:27.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.580+0000 7f1edd3ff700 1 -- 172.21.15.111:0/60614014 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed80b2500 con 0x7f1ed81598e0 2023-03-29T10:57:27.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.581+0000 7f1edd3ff700 1 -- 172.21.15.111:0/60614014 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed8154430 con 0x7f1ed81598e0 2023-03-29T10:57:27.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.582+0000 7f1ecffff700 1 -- 172.21.15.111:0/60614014 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f1ed000dd30 con 0x7f1ed81598e0 2023-03-29T10:57:27.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.582+0000 7f1ecffff700 1 --2- 172.21.15.111:0/60614014 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1ebc069e10 0x7f1ebc06c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:27.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.583+0000 7f1ecffff700 1 -- 172.21.15.111:0/60614014 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f1ed0088230 con 0x7f1ed81598e0 2023-03-29T10:57:27.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.583+0000 7f1ed6ffd700 1 --2- 172.21.15.111:0/60614014 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1ebc069e10 0x7f1ebc06c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:27.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.583+0000 7f1ed6ffd700 1 --2- 172.21.15.111:0/60614014 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1ebc069e10 0x7f1ebc06c2c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8001930 tx=0x7f1ec8007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:27.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.587+0000 7f1ecffff700 1 -- 172.21.15.111:0/60614014 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f1ed8154430 con 0x7f1ed81598e0 2023-03-29T10:57:27.822 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.821+0000 7f1edd3ff700 1 -- 172.21.15.111:0/60614014 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1ed8154430 con 0x7f1ed81598e0 2023-03-29T10:57:27.829 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.828+0000 7f1ecffff700 1 -- 172.21.15.111:0/60614014 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1ed8154430 con 0x7f1ed81598e0 2023-03-29T10:57:27.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.836+0000 7f1edd3ff700 1 -- 172.21.15.111:0/60614014 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1ebc069e10 msgr2=0x7f1ebc06c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:27.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.836+0000 7f1edd3ff700 1 --2- 172.21.15.111:0/60614014 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1ebc069e10 0x7f1ebc06c2c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8001930 tx=0x7f1ec8007000 comp rx=0 tx=0).stop 2023-03-29T10:57:27.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.836+0000 7f1edd3ff700 1 -- 172.21.15.111:0/60614014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ed81598e0 msgr2=0x7f1ed81571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:27.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.836+0000 7f1edd3ff700 1 --2- 172.21.15.111:0/60614014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ed81598e0 0x7f1ed81571b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1ed80b1a90 tx=0x7f1ed000d430 comp rx=0 tx=0).stop 2023-03-29T10:57:27.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.838+0000 7f1edd3ff700 1 -- 172.21.15.111:0/60614014 shutdown_connections 2023-03-29T10:57:27.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.838+0000 7f1edd3ff700 1 --2- 172.21.15.111:0/60614014 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1ebc069e10 0x7f1ebc06c2c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:27.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.838+0000 7f1edd3ff700 1 --2- 172.21.15.111:0/60614014 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1ed81598e0 0x7f1ed81571b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:27.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.838+0000 7f1edd3ff700 1 --2- 172.21.15.111:0/60614014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1eb8003a30 0x7f1ed81e8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:27.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.838+0000 7f1edd3ff700 1 --2- 172.21.15.111:0/60614014 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1ed81e8860 0x7f1ed81593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:27.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.838+0000 7f1edd3ff700 1 -- 172.21.15.111:0/60614014 >> 172.21.15.111:0/60614014 conn(0x7f1ed8071b10 msgr2=0x7f1ed80b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:27.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.844+0000 7f1edd3ff700 1 -- 172.21.15.111:0/60614014 shutdown_connections 2023-03-29T10:57:27.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:27.844+0000 7f1edd3ff700 1 -- 172.21.15.111:0/60614014 wait complete. 2023-03-29T10:57:27.849 INFO:teuthology.orchestra.run.smithi111.stderr:new fs with metadata pool 3 and data pool 4 2023-03-29T10:57:27.870 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-03-29T10:57:27.870 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-03-29T10:57:28.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.058+0000 7f040d9ff700 1 Processor -- start 2023-03-29T10:57:28.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.058+0000 7f040d9ff700 1 -- start start 2023-03-29T10:57:28.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.059+0000 7f040d9ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f04080cfbf0 0x7f04081608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:28.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.059+0000 7f040d9ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04080d0020 con 0x7f04080b87a0 2023-03-29T10:57:28.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.059+0000 7f040d9ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04080cdc60 con 0x7f04080d1100 2023-03-29T10:57:28.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.059+0000 7f040d9ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04080d02d0 con 0x7f04080cfbf0 2023-03-29T10:57:28.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.060+0000 7f03fe7fc700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f04080cfbf0 0x7f04081608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:28.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.060+0000 7f0406ffd700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f04080b87a0 0x7f04081601e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53676/0 (socket says 172.21.15.111:53676) 2023-03-29T10:57:28.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.060+0000 7f0406ffd700 1 -- 172.21.15.111:0/827309309 learned_addr learned my addr 172.21.15.111:0/827309309 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:28.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.060+0000 7f03fe7fc700 1 -- 172.21.15.111:0/827309309 >> v1:172.21.15.119:6789/0 conn(0x7f04080d1100 legacy=0x7f04080d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:28.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.060+0000 7f03fe7fc700 1 -- 172.21.15.111:0/827309309 >> v1:172.21.15.111:6789/0 conn(0x7f04080b87a0 legacy=0x7f04081601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:28.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.060+0000 7f03fe7fc700 1 -- 172.21.15.111:0/827309309 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04080cd6a0 con 0x7f04080cfbf0 2023-03-29T10:57:28.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.060+0000 7f04067fc700 1 -- 172.21.15.111:0/827309309 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3959242506 0 0) 0x7f04080d0020 con 0x7f04080b87a0 2023-03-29T10:57:28.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.060+0000 7f03fe7fc700 1 --2- 172.21.15.111:0/827309309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04080cfbf0 0x7f04081608f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009840 tx=0x7f03f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6f87fb20d47600 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:28.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.060+0000 7f04067fc700 1 -- 172.21.15.111:0/827309309 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f0011830 con 0x7f04080cfbf0 2023-03-29T10:57:28.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.060+0000 7f04067fc700 1 -- 172.21.15.111:0/827309309 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f0011990 con 0x7f04080cfbf0 2023-03-29T10:57:28.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.061+0000 7f04067fc700 1 -- 172.21.15.111:0/827309309 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f0011d20 con 0x7f04080cfbf0 2023-03-29T10:57:28.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.061+0000 7f040d9ff700 1 -- 172.21.15.111:0/827309309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04080cfbf0 msgr2=0x7f04081608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:28.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.061+0000 7f040d9ff700 1 --2- 172.21.15.111:0/827309309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04080cfbf0 0x7f04081608f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f03f0009840 tx=0x7f03f0009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:28.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.061+0000 7f040d9ff700 1 -- 172.21.15.111:0/827309309 shutdown_connections 2023-03-29T10:57:28.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.061+0000 7f040d9ff700 1 --2- 172.21.15.111:0/827309309 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04080cfbf0 0x7f04081608f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:28.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.061+0000 7f040d9ff700 1 -- 172.21.15.111:0/827309309 >> 172.21.15.111:0/827309309 conn(0x7f0408071b10 msgr2=0x7f0408071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:28.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.061+0000 7f040d9ff700 1 -- 172.21.15.111:0/827309309 shutdown_connections 2023-03-29T10:57:28.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.061+0000 7f040d9ff700 1 -- 172.21.15.111:0/827309309 wait complete. 2023-03-29T10:57:28.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.063+0000 7f040d9ff700 1 Processor -- start 2023-03-29T10:57:28.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.064+0000 7f040d9ff700 1 -- start start 2023-03-29T10:57:28.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.065+0000 7f040d9ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04080b87a0 0x7f04081f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:28.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.065+0000 7f040d9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04080cfbf0 0x7f04080c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:28.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.065+0000 7f040d9ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f04080d1100 0x7f04080b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:28.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.065+0000 7f040d9ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04081f1e90 con 0x7f04080b87a0 2023-03-29T10:57:28.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.065+0000 7f040d9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04080caee0 con 0x7f04080cfbf0 2023-03-29T10:57:28.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.065+0000 7f040d9ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04080ca360 con 0x7f04080d1100 2023-03-29T10:57:28.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.065+0000 7f0406ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04080b87a0 0x7f04081f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:28.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.065+0000 7f0406ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04080b87a0 0x7f04081f1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50972/0 (socket says 172.21.15.111:50972) 2023-03-29T10:57:28.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.065+0000 7f0406ffd700 1 -- 172.21.15.111:0/866339556 learned_addr learned my addr 172.21.15.111:0/866339556 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:28.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.065+0000 7f03fe7fc700 1 --2- 172.21.15.111:0/866339556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04080cfbf0 0x7f04080c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:28.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.065+0000 7f04077fe700 1 --2- 172.21.15.111:0/866339556 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f04080d1100 0x7f04080b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:28.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.065+0000 7f0406ffd700 1 -- 172.21.15.111:0/866339556 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f04080d1100 msgr2=0x7f04080b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:28.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.065+0000 7f0406ffd700 1 --2- 172.21.15.111:0/866339556 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f04080d1100 0x7f04080b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:28.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.065+0000 7f0406ffd700 1 -- 172.21.15.111:0/866339556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04080cfbf0 msgr2=0x7f04080c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:28.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.065+0000 7f0406ffd700 1 --2- 172.21.15.111:0/866339556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04080cfbf0 0x7f04080c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:28.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.065+0000 7f0406ffd700 1 -- 172.21.15.111:0/866339556 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f00094a0 con 0x7f04080b87a0 2023-03-29T10:57:28.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.065+0000 7f03fe7fc700 1 --2- 172.21.15.111:0/866339556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04080cfbf0 0x7f04080c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:57:28.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.066+0000 7f0406ffd700 1 --2- 172.21.15.111:0/866339556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04080b87a0 0x7f04081f1950 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f03f800cc80 tx=0x7f03f800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:28.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.066+0000 7f0404ff9700 1 -- 172.21.15.111:0/866339556 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f800ade0 con 0x7f04080b87a0 2023-03-29T10:57:28.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.066+0000 7f0404ff9700 1 -- 172.21.15.111:0/866339556 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f800ce80 con 0x7f04080b87a0 2023-03-29T10:57:28.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.066+0000 7f040d9ff700 1 -- 172.21.15.111:0/866339556 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0408160e30 con 0x7f04080b87a0 2023-03-29T10:57:28.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.066+0000 7f0404ff9700 1 -- 172.21.15.111:0/866339556 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f8007310 con 0x7f04080b87a0 2023-03-29T10:57:28.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.066+0000 7f040d9ff700 1 -- 172.21.15.111:0/866339556 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04080cd980 con 0x7f04080b87a0 2023-03-29T10:57:28.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.067+0000 7f040d9ff700 1 -- 172.21.15.111:0/866339556 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0408160fc0 con 0x7f04080b87a0 2023-03-29T10:57:28.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.068+0000 7f0404ff9700 1 -- 172.21.15.111:0/866339556 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f03f8007470 con 0x7f04080b87a0 2023-03-29T10:57:28.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.069+0000 7f0404ff9700 1 --2- 172.21.15.111:0/866339556 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f03e8069de0 0x7f03e806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:28.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.069+0000 7f0404ff9700 1 -- 172.21.15.111:0/866339556 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f03f8087f40 con 0x7f04080b87a0 2023-03-29T10:57:28.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.069+0000 7f03fe7fc700 1 --2- 172.21.15.111:0/866339556 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f03e8069de0 0x7f03e806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:28.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.070+0000 7f03fe7fc700 1 --2- 172.21.15.111:0/866339556 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f03e8069de0 0x7f03e806c290 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f04081f53d0 tx=0x7f03f001c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:28.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.073+0000 7f0404ff9700 1 -- 172.21.15.111:0/866339556 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f03f8053790 con 0x7f04080b87a0 2023-03-29T10:57:28.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.282+0000 7f040d9ff700 1 -- 172.21.15.111:0/866339556 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f04081f1fd0 con 0x7f04080b87a0 2023-03-29T10:57:28.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.832+0000 7f0404ff9700 1 -- 172.21.15.111:0/866339556 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v24) v1 ==== 157+0+0 (secure 0 0 0) 0x7f03f80180e0 con 0x7f04080b87a0 2023-03-29T10:57:28.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.835+0000 7f040d9ff700 1 -- 172.21.15.111:0/866339556 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f03e8069de0 msgr2=0x7f03e806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:28.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.835+0000 7f040d9ff700 1 --2- 172.21.15.111:0/866339556 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f03e8069de0 0x7f03e806c290 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f04081f53d0 tx=0x7f03f001c040 comp rx=0 tx=0).stop 2023-03-29T10:57:28.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.835+0000 7f040d9ff700 1 -- 172.21.15.111:0/866339556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04080b87a0 msgr2=0x7f04081f1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:28.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.835+0000 7f040d9ff700 1 --2- 172.21.15.111:0/866339556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04080b87a0 0x7f04081f1950 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f03f800cc80 tx=0x7f03f800a340 comp rx=0 tx=0).stop 2023-03-29T10:57:28.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.835+0000 7f040d9ff700 1 -- 172.21.15.111:0/866339556 shutdown_connections 2023-03-29T10:57:28.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.835+0000 7f040d9ff700 1 --2- 172.21.15.111:0/866339556 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f03e8069de0 0x7f03e806c290 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:28.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.835+0000 7f040d9ff700 1 --2- 172.21.15.111:0/866339556 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f04080b87a0 0x7f04081f1950 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:28.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.835+0000 7f040d9ff700 1 --2- 172.21.15.111:0/866339556 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f04080d1100 0x7f04080b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:28.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.835+0000 7f040d9ff700 1 --2- 172.21.15.111:0/866339556 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04080cfbf0 0x7f04080c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:28.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.835+0000 7f040d9ff700 1 -- 172.21.15.111:0/866339556 >> 172.21.15.111:0/866339556 conn(0x7f0408071b10 msgr2=0x7f04080b7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:28.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.835+0000 7f040d9ff700 1 -- 172.21.15.111:0/866339556 shutdown_connections 2023-03-29T10:57:28.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:28.835+0000 7f040d9ff700 1 -- 172.21.15.111:0/866339556 wait complete. 2023-03-29T10:57:28.853 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-03-29T10:57:29.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.042+0000 7f38631ff700 1 Processor -- start 2023-03-29T10:57:29.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.042+0000 7f38631ff700 1 -- start start 2023-03-29T10:57:29.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.043+0000 7f38631ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f385c0cfbf0 0x7f385c1608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:29.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.043+0000 7f38631ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f385c0d1100 0x7f385c0d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:29.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.043+0000 7f38631ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f385c0d0020 con 0x7f385c0b87a0 2023-03-29T10:57:29.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.043+0000 7f38631ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385c0d02d0 con 0x7f385c0cfbf0 2023-03-29T10:57:29.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.043+0000 7f38631ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385c0d0cb0 con 0x7f385c0d1100 2023-03-29T10:57:29.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.044+0000 7f3860f9b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f385c0b87a0 0x7f385c1601e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:32912/0 (socket says 172.21.15.111:32912) 2023-03-29T10:57:29.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.044+0000 7f385bfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f385c0cfbf0 0x7f385c1608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:29.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.044+0000 7f3860f9b700 1 -- 172.21.15.111:0/1571901334 learned_addr learned my addr 172.21.15.111:0/1571901334 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:29.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.044+0000 7f386179c700 1 --2- 172.21.15.111:0/1571901334 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f385c0d1100 0x7f385c0d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:29.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.044+0000 7f385bfff700 1 -- 172.21.15.111:0/1571901334 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f385c0d1100 msgr2=0x7f385c0d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:29.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.044+0000 7f385bfff700 1 --2- 172.21.15.111:0/1571901334 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f385c0d1100 0x7f385c0d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:29.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.044+0000 7f385bfff700 1 -- 172.21.15.111:0/1571901334 >> v1:172.21.15.111:6789/0 conn(0x7f385c0b87a0 legacy=0x7f385c1601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:29.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.044+0000 7f385bfff700 1 -- 172.21.15.111:0/1571901334 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385c0cdc60 con 0x7f385c0cfbf0 2023-03-29T10:57:29.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.044+0000 7f385b7fe700 1 -- 172.21.15.111:0/1571901334 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894878477 0 0) 0x7f385c0d0020 con 0x7f385c0b87a0 2023-03-29T10:57:29.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.044+0000 7f385bfff700 1 --2- 172.21.15.111:0/1571901334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f385c0cfbf0 0x7f385c1608f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3848009840 tx=0x7f3848009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979453bc37bc1104 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:29.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.045+0000 7f385b7fe700 1 -- 172.21.15.111:0/1571901334 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848011830 con 0x7f385c0cfbf0 2023-03-29T10:57:29.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.045+0000 7f385b7fe700 1 -- 172.21.15.111:0/1571901334 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3848011990 con 0x7f385c0cfbf0 2023-03-29T10:57:29.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.045+0000 7f385b7fe700 1 -- 172.21.15.111:0/1571901334 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848011d20 con 0x7f385c0cfbf0 2023-03-29T10:57:29.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.045+0000 7f38631ff700 1 -- 172.21.15.111:0/1571901334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f385c0cfbf0 msgr2=0x7f385c1608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:29.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.045+0000 7f38631ff700 1 --2- 172.21.15.111:0/1571901334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f385c0cfbf0 0x7f385c1608f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3848009840 tx=0x7f3848009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:29.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.045+0000 7f38631ff700 1 -- 172.21.15.111:0/1571901334 shutdown_connections 2023-03-29T10:57:29.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.045+0000 7f38631ff700 1 --2- 172.21.15.111:0/1571901334 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f385c0d1100 0x7f385c0d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:29.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.045+0000 7f38631ff700 1 --2- 172.21.15.111:0/1571901334 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f385c0cfbf0 0x7f385c1608f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:29.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.045+0000 7f38631ff700 1 -- 172.21.15.111:0/1571901334 >> 172.21.15.111:0/1571901334 conn(0x7f385c071b10 msgr2=0x7f385c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:29.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.045+0000 7f38631ff700 1 -- 172.21.15.111:0/1571901334 shutdown_connections 2023-03-29T10:57:29.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.045+0000 7f38631ff700 1 -- 172.21.15.111:0/1571901334 wait complete. 2023-03-29T10:57:29.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.047+0000 7f38631ff700 1 Processor -- start 2023-03-29T10:57:29.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.047+0000 7f38631ff700 1 -- start start 2023-03-29T10:57:29.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.048+0000 7f38631ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f385c0b87a0 0x7f385c14deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:29.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.048+0000 7f38631ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f385c0cfbf0 0x7f385c0c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:29.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.048+0000 7f38631ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f385c0d1100 0x7f385c0c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:29.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.048+0000 7f38631ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c0c9f80 con 0x7f385c0d1100 2023-03-29T10:57:29.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.048+0000 7f38631ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c14e3f0 con 0x7f385c0cfbf0 2023-03-29T10:57:29.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.048+0000 7f38631ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385c15ecd0 con 0x7f385c0b87a0 2023-03-29T10:57:29.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.048+0000 7f386179c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f385c0d1100 0x7f385c0c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:29.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.048+0000 7f386179c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f385c0d1100 0x7f385c0c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38808/0 (socket says 172.21.15.111:38808) 2023-03-29T10:57:29.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.048+0000 7f386179c700 1 -- 172.21.15.111:0/2438305857 learned_addr learned my addr 172.21.15.111:0/2438305857 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:29.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.048+0000 7f385bfff700 1 --2- 172.21.15.111:0/2438305857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f385c0cfbf0 0x7f385c0c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:29.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.048+0000 7f3860f9b700 1 --2- 172.21.15.111:0/2438305857 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f385c0b87a0 0x7f385c14deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:29.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.049+0000 7f386179c700 1 -- 172.21.15.111:0/2438305857 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f385c0b87a0 msgr2=0x7f385c14deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:29.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.049+0000 7f386179c700 1 --2- 172.21.15.111:0/2438305857 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f385c0b87a0 0x7f385c14deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:29.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.049+0000 7f386179c700 1 -- 172.21.15.111:0/2438305857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f385c0cfbf0 msgr2=0x7f385c0c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:29.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.049+0000 7f386179c700 1 --2- 172.21.15.111:0/2438305857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f385c0cfbf0 0x7f385c0c7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:29.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.049+0000 7f386179c700 1 -- 172.21.15.111:0/2438305857 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38480094a0 con 0x7f385c0d1100 2023-03-29T10:57:29.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.049+0000 7f386179c700 1 --2- 172.21.15.111:0/2438305857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f385c0d1100 0x7f385c0c8360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3854007e60 tx=0x7f385400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:29.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.049+0000 7f38597fa700 1 -- 172.21.15.111:0/2438305857 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385400ed30 con 0x7f385c0d1100 2023-03-29T10:57:29.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.049+0000 7f38597fa700 1 -- 172.21.15.111:0/2438305857 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3854014420 con 0x7f385c0d1100 2023-03-29T10:57:29.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.049+0000 7f38631ff700 1 -- 172.21.15.111:0/2438305857 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385c0cd980 con 0x7f385c0d1100 2023-03-29T10:57:29.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.049+0000 7f38597fa700 1 -- 172.21.15.111:0/2438305857 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38540147d0 con 0x7f385c0d1100 2023-03-29T10:57:29.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.049+0000 7f38631ff700 1 -- 172.21.15.111:0/2438305857 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385c0cd6a0 con 0x7f385c0d1100 2023-03-29T10:57:29.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.050+0000 7f38631ff700 1 -- 172.21.15.111:0/2438305857 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f385c0ca0f0 con 0x7f385c0d1100 2023-03-29T10:57:29.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.051+0000 7f38597fa700 1 -- 172.21.15.111:0/2438305857 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f3854014930 con 0x7f385c0d1100 2023-03-29T10:57:29.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.052+0000 7f38597fa700 1 --2- 172.21.15.111:0/2438305857 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3840069de0 0x7f384006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:29.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.052+0000 7f38597fa700 1 -- 172.21.15.111:0/2438305857 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f38540886e0 con 0x7f385c0d1100 2023-03-29T10:57:29.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.052+0000 7f3860f9b700 1 --2- 172.21.15.111:0/2438305857 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3840069de0 0x7f384006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:29.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.053+0000 7f3860f9b700 1 --2- 172.21.15.111:0/2438305857 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3840069de0 0x7f384006c290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3850007fd0 tx=0x7f3850007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:29.056 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.056+0000 7f38597fa700 1 -- 172.21.15.111:0/2438305857 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f3854053e30 con 0x7f385c0d1100 2023-03-29T10:57:29.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:29.251+0000 7f38631ff700 1 -- 172.21.15.111:0/2438305857 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f385c074430 con 0x7f385c0d1100 2023-03-29T10:57:30.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:30.842+0000 7f38597fa700 1 -- 172.21.15.111:0/2438305857 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v26) v1 ==== 228+0+0 (secure 0 0 0) 0x7f3854057450 con 0x7f385c0d1100 2023-03-29T10:57:30.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:30.848+0000 7f38631ff700 1 -- 172.21.15.111:0/2438305857 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3840069de0 msgr2=0x7f384006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:30.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:30.848+0000 7f38631ff700 1 --2- 172.21.15.111:0/2438305857 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3840069de0 0x7f384006c290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3850007fd0 tx=0x7f3850007de0 comp rx=0 tx=0).stop 2023-03-29T10:57:30.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:30.848+0000 7f38631ff700 1 -- 172.21.15.111:0/2438305857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f385c0d1100 msgr2=0x7f385c0c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:30.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:30.848+0000 7f38631ff700 1 --2- 172.21.15.111:0/2438305857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f385c0d1100 0x7f385c0c8360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3854007e60 tx=0x7f385400e3c0 comp rx=0 tx=0).stop 2023-03-29T10:57:30.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:30.849+0000 7f38631ff700 1 -- 172.21.15.111:0/2438305857 shutdown_connections 2023-03-29T10:57:30.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:30.849+0000 7f38631ff700 1 --2- 172.21.15.111:0/2438305857 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3840069de0 0x7f384006c290 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3850007fd0 tx=0x7f3850007de0 comp rx=0 tx=0).stop 2023-03-29T10:57:30.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:30.849+0000 7f38631ff700 1 --2- 172.21.15.111:0/2438305857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f385c0d1100 0x7f385c0c8360 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:30.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:30.849+0000 7f38631ff700 1 --2- 172.21.15.111:0/2438305857 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f385c0b87a0 0x7f385c14deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:30.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:30.849+0000 7f38631ff700 1 --2- 172.21.15.111:0/2438305857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f385c0cfbf0 0x7f385c0c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:30.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:30.849+0000 7f38631ff700 1 -- 172.21.15.111:0/2438305857 >> 172.21.15.111:0/2438305857 conn(0x7f385c071b10 msgr2=0x7f385c0b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:30.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:30.856+0000 7f38631ff700 1 -- 172.21.15.111:0/2438305857 shutdown_connections 2023-03-29T10:57:30.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:30.856+0000 7f38631ff700 1 -- 172.21.15.111:0/2438305857 wait complete. 2023-03-29T10:57:30.858 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data_ec' created 2023-03-29T10:57:30.877 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-03-29T10:57:31.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.066+0000 7f1ce27ff700 1 Processor -- start 2023-03-29T10:57:31.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.066+0000 7f1ce27ff700 1 -- start start 2023-03-29T10:57:31.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.067+0000 7f1ce27ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1cdc0b87a0 0x7f1cdc0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:31.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.067+0000 7f1ce27ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cdc160d40 0x7f1cdc0d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:31.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.067+0000 7f1ce27ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cdc0cdc60 con 0x7f1cdc1601e0 2023-03-29T10:57:31.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.067+0000 7f1ce27ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cdc0d02d0 con 0x7f1cdc0b87a0 2023-03-29T10:57:31.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.067+0000 7f1ce27ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0d0cb0 con 0x7f1cdc160d40 2023-03-29T10:57:31.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.067+0000 7f1cdb7fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1cdc1601e0 0x7f1cdc160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:32914/0 (socket says 172.21.15.111:32914) 2023-03-29T10:57:31.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.067+0000 7f1cdb7fe700 1 -- 172.21.15.111:0/1580828281 learned_addr learned my addr 172.21.15.111:0/1580828281 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:31.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1ce0d9c700 1 --2- 172.21.15.111:0/1580828281 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cdc160d40 0x7f1cdc0d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:31.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1cdbfff700 1 --2- 172.21.15.111:0/1580828281 >> v2:172.21.15.119:3300/0 conn(0x7f1cdc0b87a0 0x7f1cdc0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:31.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1cdaffd700 1 -- 172.21.15.111:0/1580828281 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414195047 0 0) 0x7f1cdc0cdc60 con 0x7f1cdc1601e0 2023-03-29T10:57:31.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1cdaffd700 1 -- 172.21.15.111:0/1580828281 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cc40035b0 con 0x7f1cdc1601e0 2023-03-29T10:57:31.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1cdaffd700 1 -- 172.21.15.111:0/1580828281 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2927105599 0 0) 0x7f1cc40035b0 con 0x7f1cdc1601e0 2023-03-29T10:57:31.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1cdaffd700 1 -- 172.21.15.111:0/1580828281 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cdc160d40 msgr2=0x7f1cdc0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:31.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1cdaffd700 1 --2- 172.21.15.111:0/1580828281 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cdc160d40 0x7f1cdc0d1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:31.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1cdaffd700 1 -- 172.21.15.111:0/1580828281 >> v2:172.21.15.119:3300/0 conn(0x7f1cdc0b87a0 msgr2=0x7f1cdc0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:31.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1cdaffd700 1 --2- 172.21.15.111:0/1580828281 >> v2:172.21.15.119:3300/0 conn(0x7f1cdc0b87a0 0x7f1cdc0cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:31.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1cdaffd700 1 -- 172.21.15.111:0/1580828281 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc0cd6a0 con 0x7f1cdc1601e0 2023-03-29T10:57:31.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1cdaffd700 1 -- 172.21.15.111:0/1580828281 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f1cd0001670 con 0x7f1cdc1601e0 2023-03-29T10:57:31.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1cdaffd700 1 -- 172.21.15.111:0/1580828281 >> v1:172.21.15.111:6789/0 conn(0x7f1cdc1601e0 legacy=0x7f1cdc160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:31.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1cdaffd700 1 --2- 172.21.15.111:0/1580828281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc4003a00 0x7f1cc4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:31.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1cdaffd700 1 --2- 172.21.15.111:0/1580828281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc4004430 0x7f1cc40048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:31.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1cdaffd700 1 --2- 172.21.15.111:0/1580828281 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cdc160d40 0x7f1cc4004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:31.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1cdaffd700 1 -- 172.21.15.111:0/1580828281 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0d02d0 con 0x7f1cc4003a00 2023-03-29T10:57:31.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1cdaffd700 1 -- 172.21.15.111:0/1580828281 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc4005700 con 0x7f1cc4004430 2023-03-29T10:57:31.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1cdaffd700 1 -- 172.21.15.111:0/1580828281 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc400ba90 con 0x7f1cdc160d40 2023-03-29T10:57:31.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1cdbfff700 1 --2- 172.21.15.111:0/1580828281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc4003a00 0x7f1cc4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:31.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1cdb7fe700 1 --2- 172.21.15.111:0/1580828281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc4004430 0x7f1cc40048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:31.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.068+0000 7f1ce0d9c700 1 --2- 172.21.15.111:0/1580828281 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cdc160d40 0x7f1cc4004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:31.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.069+0000 7f1cdbfff700 1 -- 172.21.15.111:0/1580828281 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cdc160d40 msgr2=0x7f1cc4004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:31.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.069+0000 7f1cdbfff700 1 --2- 172.21.15.111:0/1580828281 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cdc160d40 0x7f1cc4004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:31.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.069+0000 7f1cdbfff700 1 -- 172.21.15.111:0/1580828281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc4004430 msgr2=0x7f1cc40048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:31.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.069+0000 7f1cdbfff700 1 --2- 172.21.15.111:0/1580828281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cc4004430 0x7f1cc40048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:31.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.069+0000 7f1cdbfff700 1 -- 172.21.15.111:0/1580828281 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc40035b0 con 0x7f1cc4003a00 2023-03-29T10:57:31.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.069+0000 7f1cdbfff700 1 --2- 172.21.15.111:0/1580828281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc4003a00 0x7f1cc4003e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009b60 tx=0x7f1cc4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac17407fa4f75a9 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:31.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.069+0000 7f1cdaffd700 1 -- 172.21.15.111:0/1580828281 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc00e6c0 con 0x7f1cc4003a00 2023-03-29T10:57:31.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.069+0000 7f1cdaffd700 1 -- 172.21.15.111:0/1580828281 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc00e820 con 0x7f1cc4003a00 2023-03-29T10:57:31.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.069+0000 7f1cdaffd700 1 -- 172.21.15.111:0/1580828281 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc00eb90 con 0x7f1cc4003a00 2023-03-29T10:57:31.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.070+0000 7f1ce27ff700 1 -- 172.21.15.111:0/1580828281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc4003a00 msgr2=0x7f1cc4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:31.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.070+0000 7f1ce27ff700 1 --2- 172.21.15.111:0/1580828281 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cc4003a00 0x7f1cc4003e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc009b60 tx=0x7f1cc4005880 comp rx=0 tx=0).stop 2023-03-29T10:57:31.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.070+0000 7f1ce0d9c700 1 -- 172.21.15.111:0/1580828281 reap_dead start 2023-03-29T10:57:31.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.070+0000 7f1ce27ff700 1 -- 172.21.15.111:0/1580828281 shutdown_connections 2023-03-29T10:57:31.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.070+0000 7f1ce27ff700 1 -- 172.21.15.111:0/1580828281 >> 172.21.15.111:0/1580828281 conn(0x7f1cdc071b10 msgr2=0x7f1cdc071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:31.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.070+0000 7f1ce27ff700 1 -- 172.21.15.111:0/1580828281 shutdown_connections 2023-03-29T10:57:31.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.070+0000 7f1ce27ff700 1 -- 172.21.15.111:0/1580828281 wait complete. 2023-03-29T10:57:31.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.072+0000 7f1ce27ff700 1 Processor -- start 2023-03-29T10:57:31.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.072+0000 7f1ce27ff700 1 -- start start 2023-03-29T10:57:31.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.073+0000 7f1ce27ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cc4003a00 0x7f1cdc0c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:31.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.073+0000 7f1ce27ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cdc0cc550 0x7f1cdc0d2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:31.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.073+0000 7f1ce27ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cdc150f00 0x7f1cdc1513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:31.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.073+0000 7f1ce27ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0ca4c0 con 0x7f1cdc0cc550 2023-03-29T10:57:31.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.073+0000 7f1ce27ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc15ced0 con 0x7f1cdc150f00 2023-03-29T10:57:31.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.073+0000 7f1ce27ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0ccdf0 con 0x7f1cc4003a00 2023-03-29T10:57:31.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.073+0000 7f1cdb7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cdc0cc550 0x7f1cdc0d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:31.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.073+0000 7f1cdb7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cdc0cc550 0x7f1cdc0d2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38822/0 (socket says 172.21.15.111:38822) 2023-03-29T10:57:31.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.073+0000 7f1cdb7fe700 1 -- 172.21.15.111:0/4069041140 learned_addr learned my addr 172.21.15.111:0/4069041140 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:31.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.073+0000 7f1cdbfff700 1 --2- 172.21.15.111:0/4069041140 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cc4003a00 0x7f1cdc0c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:31.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.073+0000 7f1ce0d9c700 1 --2- 172.21.15.111:0/4069041140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cdc150f00 0x7f1cdc1513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:31.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.073+0000 7f1cdb7fe700 1 -- 172.21.15.111:0/4069041140 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cc4003a00 msgr2=0x7f1cdc0c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:31.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.073+0000 7f1cdb7fe700 1 --2- 172.21.15.111:0/4069041140 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cc4003a00 0x7f1cdc0c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:31.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.073+0000 7f1cdb7fe700 1 -- 172.21.15.111:0/4069041140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cdc150f00 msgr2=0x7f1cdc1513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:31.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.073+0000 7f1cdb7fe700 1 --2- 172.21.15.111:0/4069041140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cdc150f00 0x7f1cdc1513a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:31.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.073+0000 7f1cdb7fe700 1 -- 172.21.15.111:0/4069041140 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc0d1100 con 0x7f1cdc0cc550 2023-03-29T10:57:31.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.073+0000 7f1ce0d9c700 1 --2- 172.21.15.111:0/4069041140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cdc150f00 0x7f1cdc1513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:57:31.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.073+0000 7f1cdb7fe700 1 --2- 172.21.15.111:0/4069041140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cdc0cc550 0x7f1cdc0d2850 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0007340 tx=0x7f1cd0007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:31.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.074+0000 7f1cd8ff9700 1 -- 172.21.15.111:0/4069041140 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd000f6c0 con 0x7f1cdc0cc550 2023-03-29T10:57:31.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.074+0000 7f1cd8ff9700 1 -- 172.21.15.111:0/4069041140 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd000f820 con 0x7f1cdc0cc550 2023-03-29T10:57:31.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.074+0000 7f1cd8ff9700 1 -- 172.21.15.111:0/4069041140 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd000fbd0 con 0x7f1cdc0cc550 2023-03-29T10:57:31.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.074+0000 7f1ce27ff700 1 -- 172.21.15.111:0/4069041140 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cdc0cd980 con 0x7f1cdc0cc550 2023-03-29T10:57:31.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.074+0000 7f1ce27ff700 1 -- 172.21.15.111:0/4069041140 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cdc0d1290 con 0x7f1cdc0cc550 2023-03-29T10:57:31.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.075+0000 7f1ce27ff700 1 -- 172.21.15.111:0/4069041140 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cdc15cca0 con 0x7f1cdc0cc550 2023-03-29T10:57:31.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.076+0000 7f1cd8ff9700 1 -- 172.21.15.111:0/4069041140 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f1cd000fd30 con 0x7f1cdc0cc550 2023-03-29T10:57:31.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.077+0000 7f1cd8ff9700 1 --2- 172.21.15.111:0/4069041140 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1cc8069de0 0x7f1cc806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:31.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.077+0000 7f1cd8ff9700 1 -- 172.21.15.111:0/4069041140 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f1cd00882f0 con 0x7f1cdc0cc550 2023-03-29T10:57:31.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.077+0000 7f1cdbfff700 1 --2- 172.21.15.111:0/4069041140 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1cc8069de0 0x7f1cc806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:31.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.077+0000 7f1cdbfff700 1 --2- 172.21.15.111:0/4069041140 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1cc8069de0 0x7f1cc806c290 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0090b0 tx=0x7f1ccc01f040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:31.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.081+0000 7f1cd8ff9700 1 -- 172.21.15.111:0/4069041140 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f1cdc15cca0 con 0x7f1cdc0cc550 2023-03-29T10:57:31.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.276+0000 7f1ce27ff700 1 -- 172.21.15.111:0/4069041140 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f1cdc15cc70 con 0x7f1cdc0cc550 2023-03-29T10:57:31.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.846+0000 7f1cd8ff9700 1 -- 172.21.15.111:0/4069041140 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1cd0053830 con 0x7f1cdc0cc550 2023-03-29T10:57:31.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.855+0000 7f1ce27ff700 1 -- 172.21.15.111:0/4069041140 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1cc8069de0 msgr2=0x7f1cc806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:31.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.855+0000 7f1ce27ff700 1 --2- 172.21.15.111:0/4069041140 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1cc8069de0 0x7f1cc806c290 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0090b0 tx=0x7f1ccc01f040 comp rx=0 tx=0).stop 2023-03-29T10:57:31.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.855+0000 7f1ce27ff700 1 -- 172.21.15.111:0/4069041140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cdc0cc550 msgr2=0x7f1cdc0d2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:31.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.855+0000 7f1ce27ff700 1 --2- 172.21.15.111:0/4069041140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cdc0cc550 0x7f1cdc0d2850 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0007340 tx=0x7f1cd0007650 comp rx=0 tx=0).stop 2023-03-29T10:57:31.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.857+0000 7f1ce27ff700 1 -- 172.21.15.111:0/4069041140 shutdown_connections 2023-03-29T10:57:31.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.857+0000 7f1ce27ff700 1 --2- 172.21.15.111:0/4069041140 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1cc8069de0 0x7f1cc806c290 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:31.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.857+0000 7f1ce27ff700 1 --2- 172.21.15.111:0/4069041140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cdc150f00 0x7f1cdc1513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:31.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.857+0000 7f1ce27ff700 1 --2- 172.21.15.111:0/4069041140 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cc4003a00 0x7f1cdc0c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:31.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.857+0000 7f1ce27ff700 1 --2- 172.21.15.111:0/4069041140 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cdc0cc550 0x7f1cdc0d2850 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:31.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.857+0000 7f1ce27ff700 1 -- 172.21.15.111:0/4069041140 >> 172.21.15.111:0/4069041140 conn(0x7f1cdc071b10 msgr2=0x7f1cdc0b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:31.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.861+0000 7f1ce27ff700 1 -- 172.21.15.111:0/4069041140 shutdown_connections 2023-03-29T10:57:31.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:31.861+0000 7f1ce27ff700 1 -- 172.21.15.111:0/4069041140 wait complete. 2023-03-29T10:57:31.866 INFO:teuthology.orchestra.run.smithi111.stderr:set pool 5 allow_ec_overwrites to true 2023-03-29T10:57:31.889 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-03-29T10:57:32.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.077+0000 7efde87ff700 1 Processor -- start 2023-03-29T10:57:32.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.077+0000 7efde87ff700 1 -- start start 2023-03-29T10:57:32.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.078+0000 7efde87ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efde015c590 0x7efde0160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.078+0000 7efde87ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde00cd520 con 0x7efde00cf010 2023-03-29T10:57:32.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.078+0000 7efde87ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde00d2640 con 0x7efde00cff50 2023-03-29T10:57:32.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.078+0000 7efde87ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efde01610d0 con 0x7efde015c590 2023-03-29T10:57:32.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.078+0000 7efde5d9a700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7efde00cf010 0x7efde00cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:32928/0 (socket says 172.21.15.111:32928) 2023-03-29T10:57:32.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.078+0000 7efde5d9a700 1 -- 172.21.15.111:0/4172425893 learned_addr learned my addr 172.21.15.111:0/4172425893 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:32.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.078+0000 7efde659b700 1 --2- 172.21.15.111:0/4172425893 >> v2:172.21.15.119:3300/0 conn(0x7efde015c590 0x7efde0160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde5599700 1 -- 172.21.15.111:0/4172425893 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385348757 0 0) 0x7efde00cd520 con 0x7efde00cf010 2023-03-29T10:57:32.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde5599700 1 -- 172.21.15.111:0/4172425893 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdcc0035b0 con 0x7efde00cf010 2023-03-29T10:57:32.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde5599700 1 -- 172.21.15.111:0/4172425893 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455478532 0 0) 0x7efde00d2640 con 0x7efde00cff50 2023-03-29T10:57:32.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde5599700 1 -- 172.21.15.111:0/4172425893 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efde00cd520 con 0x7efde00cff50 2023-03-29T10:57:32.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde5599700 1 -- 172.21.15.111:0/4172425893 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2278049951 0 0) 0x7efdcc0035b0 con 0x7efde00cf010 2023-03-29T10:57:32.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde5599700 1 -- 172.21.15.111:0/4172425893 >> v2:172.21.15.119:3300/0 conn(0x7efde015c590 msgr2=0x7efde0160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:32.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde5599700 1 --2- 172.21.15.111:0/4172425893 >> v2:172.21.15.119:3300/0 conn(0x7efde015c590 0x7efde0160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde5599700 1 -- 172.21.15.111:0/4172425893 >> v1:172.21.15.119:6789/0 conn(0x7efde00cff50 legacy=0x7efde00d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:32.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde5599700 1 -- 172.21.15.111:0/4172425893 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde00b23c0 con 0x7efde00cf010 2023-03-29T10:57:32.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde659b700 1 --2- 172.21.15.111:0/4172425893 >> v2:172.21.15.119:3300/0 conn(0x7efde015c590 0x7efde0160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:32.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde5599700 1 -- 172.21.15.111:0/4172425893 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7efdd0001670 con 0x7efde00cf010 2023-03-29T10:57:32.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde5599700 1 -- 172.21.15.111:0/4172425893 >> v1:172.21.15.111:6789/0 conn(0x7efde00cf010 legacy=0x7efde00cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:32.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde5599700 1 --2- 172.21.15.111:0/4172425893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdcc003b90 0x7efdcc003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde5599700 1 --2- 172.21.15.111:0/4172425893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdcc004850 0x7efdcc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde5599700 1 --2- 172.21.15.111:0/4172425893 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efdcc005200 0x7efdcc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde5599700 1 -- 172.21.15.111:0/4172425893 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde01610d0 con 0x7efdcc004850 2023-03-29T10:57:32.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde5599700 1 -- 172.21.15.111:0/4172425893 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc00c060 con 0x7efdcc003b90 2023-03-29T10:57:32.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde5599700 1 -- 172.21.15.111:0/4172425893 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdcc00c220 con 0x7efdcc005200 2023-03-29T10:57:32.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde5d9a700 1 --2- 172.21.15.111:0/4172425893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdcc004850 0x7efdcc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.079+0000 7efde659b700 1 --2- 172.21.15.111:0/4172425893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdcc003b90 0x7efdcc003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.080+0000 7efde6d9c700 1 --2- 172.21.15.111:0/4172425893 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efdcc005200 0x7efdcc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.080+0000 7efde5d9a700 1 -- 172.21.15.111:0/4172425893 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efdcc005200 msgr2=0x7efdcc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:32.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.080+0000 7efde5d9a700 1 --2- 172.21.15.111:0/4172425893 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efdcc005200 0x7efdcc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.080+0000 7efde5d9a700 1 -- 172.21.15.111:0/4172425893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdcc003b90 msgr2=0x7efdcc003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:32.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.080+0000 7efde5d9a700 1 --2- 172.21.15.111:0/4172425893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdcc003b90 0x7efdcc003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.080+0000 7efde5d9a700 1 -- 172.21.15.111:0/4172425893 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdcc0035b0 con 0x7efdcc004850 2023-03-29T10:57:32.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.080+0000 7efde6d9c700 1 --2- 172.21.15.111:0/4172425893 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efdcc005200 0x7efdcc00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:57:32.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.080+0000 7efde6d9c700 1 -- 172.21.15.111:0/4172425893 reap_dead start 2023-03-29T10:57:32.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.080+0000 7efde5d9a700 1 --2- 172.21.15.111:0/4172425893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdcc004850 0x7efdcc004cc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efdd000c210 tx=0x7efdcc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4788ea673d87c6bf server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:32.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.080+0000 7efde5599700 1 -- 172.21.15.111:0/4172425893 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd000cdb0 con 0x7efdcc004850 2023-03-29T10:57:32.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.080+0000 7efde5599700 1 -- 172.21.15.111:0/4172425893 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd0001af0 con 0x7efdcc004850 2023-03-29T10:57:32.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.080+0000 7efde5599700 1 -- 172.21.15.111:0/4172425893 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0005ba0 con 0x7efdcc004850 2023-03-29T10:57:32.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.081+0000 7efde87ff700 1 -- 172.21.15.111:0/4172425893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdcc004850 msgr2=0x7efdcc004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:32.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.081+0000 7efde87ff700 1 --2- 172.21.15.111:0/4172425893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdcc004850 0x7efdcc004cc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efdd000c210 tx=0x7efdcc005880 comp rx=0 tx=0).stop 2023-03-29T10:57:32.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.081+0000 7efde87ff700 1 -- 172.21.15.111:0/4172425893 shutdown_connections 2023-03-29T10:57:32.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.081+0000 7efde87ff700 1 --2- 172.21.15.111:0/4172425893 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efdcc004850 0x7efdcc004cc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.081+0000 7efde87ff700 1 -- 172.21.15.111:0/4172425893 >> 172.21.15.111:0/4172425893 conn(0x7efde0071b10 msgr2=0x7efde0071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:32.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.081+0000 7efde87ff700 1 -- 172.21.15.111:0/4172425893 shutdown_connections 2023-03-29T10:57:32.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.081+0000 7efde87ff700 1 -- 172.21.15.111:0/4172425893 wait complete. 2023-03-29T10:57:32.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.082+0000 7efde87ff700 1 Processor -- start 2023-03-29T10:57:32.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.083+0000 7efde87ff700 1 -- start start 2023-03-29T10:57:32.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.083+0000 7efde87ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdcc004850 0x7efde0155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.083+0000 7efde87ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efde015cbe0 0x7efde015d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.083+0000 7efde87ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efde015d590 0x7efde01e63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.084+0000 7efde87ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde0152c10 con 0x7efde015d590 2023-03-29T10:57:32.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.084+0000 7efde87ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde015da00 con 0x7efdcc004850 2023-03-29T10:57:32.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.084+0000 7efde87ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde015f070 con 0x7efde015cbe0 2023-03-29T10:57:32.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.084+0000 7efde6d9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efde015d590 0x7efde01e63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.084+0000 7efde6d9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efde015d590 0x7efde01e63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38834/0 (socket says 172.21.15.111:38834) 2023-03-29T10:57:32.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.084+0000 7efde6d9c700 1 -- 172.21.15.111:0/2854230427 learned_addr learned my addr 172.21.15.111:0/2854230427 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:32.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.084+0000 7efde5d9a700 1 --2- 172.21.15.111:0/2854230427 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efde015cbe0 0x7efde015d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.084+0000 7efde659b700 1 --2- 172.21.15.111:0/2854230427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdcc004850 0x7efde0155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.084+0000 7efde6d9c700 1 -- 172.21.15.111:0/2854230427 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efde015cbe0 msgr2=0x7efde015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:32.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.084+0000 7efde6d9c700 1 --2- 172.21.15.111:0/2854230427 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efde015cbe0 0x7efde015d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.084+0000 7efde6d9c700 1 -- 172.21.15.111:0/2854230427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdcc004850 msgr2=0x7efde0155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:32.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.084+0000 7efde6d9c700 1 --2- 172.21.15.111:0/2854230427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdcc004850 0x7efde0155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.084+0000 7efde6d9c700 1 -- 172.21.15.111:0/2854230427 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde00cd770 con 0x7efde015d590 2023-03-29T10:57:32.086 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.084+0000 7efde6d9c700 1 --2- 172.21.15.111:0/2854230427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efde015d590 0x7efde01e63d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efdd8001a50 tx=0x7efdd80021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:32.086 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.085+0000 7efdd6ffd700 1 -- 172.21.15.111:0/2854230427 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd8003c10 con 0x7efde015d590 2023-03-29T10:57:32.086 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.085+0000 7efdd6ffd700 1 -- 172.21.15.111:0/2854230427 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd8003d70 con 0x7efde015d590 2023-03-29T10:57:32.086 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.085+0000 7efde87ff700 1 -- 172.21.15.111:0/2854230427 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde00cf560 con 0x7efde015d590 2023-03-29T10:57:32.086 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.085+0000 7efdd6ffd700 1 -- 172.21.15.111:0/2854230427 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd80052d0 con 0x7efde015d590 2023-03-29T10:57:32.086 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.085+0000 7efde87ff700 1 -- 172.21.15.111:0/2854230427 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde015c960 con 0x7efde015d590 2023-03-29T10:57:32.086 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.086+0000 7efde87ff700 1 -- 172.21.15.111:0/2854230427 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde01e6220 con 0x7efde015d590 2023-03-29T10:57:32.086 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.087+0000 7efdd6ffd700 1 -- 172.21.15.111:0/2854230427 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7efdd8005430 con 0x7efde015d590 2023-03-29T10:57:32.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.088+0000 7efdd6ffd700 1 --2- 172.21.15.111:0/2854230427 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7efdc4069de0 0x7efdc406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.088+0000 7efdd6ffd700 1 -- 172.21.15.111:0/2854230427 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6781+0+0 (secure 0 0 0) 0x7efdd80883a0 con 0x7efde015d590 2023-03-29T10:57:32.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.088+0000 7efde659b700 1 --2- 172.21.15.111:0/2854230427 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7efdc4069de0 0x7efdc406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.088 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.088+0000 7efde659b700 1 --2- 172.21.15.111:0/2854230427 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7efdc4069de0 0x7efdc406c290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efddc009d80 tx=0x7efddc009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:32.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.092+0000 7efdd6ffd700 1 -- 172.21.15.111:0/2854230427 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7efdd80538e0 con 0x7efde015d590 2023-03-29T10:57:32.330 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.330+0000 7efde87ff700 1 -- 172.21.15.111:0/2854230427 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7efde01e6220 con 0x7efde015d590 2023-03-29T10:57:32.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.336+0000 7efdd6ffd700 1 -- 172.21.15.111:0/2854230427 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7efdd8056f00 con 0x7efde015d590 2023-03-29T10:57:32.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.338+0000 7efde87ff700 1 -- 172.21.15.111:0/2854230427 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7efdc4069de0 msgr2=0x7efdc406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:32.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.338+0000 7efde87ff700 1 --2- 172.21.15.111:0/2854230427 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7efdc4069de0 0x7efdc406c290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efddc009d80 tx=0x7efddc009210 comp rx=0 tx=0).stop 2023-03-29T10:57:32.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.338+0000 7efde87ff700 1 -- 172.21.15.111:0/2854230427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efde015d590 msgr2=0x7efde01e63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:32.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.338+0000 7efde87ff700 1 --2- 172.21.15.111:0/2854230427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efde015d590 0x7efde01e63d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efdd8001a50 tx=0x7efdd80021d0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.338+0000 7efde87ff700 1 -- 172.21.15.111:0/2854230427 shutdown_connections 2023-03-29T10:57:32.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.338+0000 7efde87ff700 1 --2- 172.21.15.111:0/2854230427 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7efdc4069de0 0x7efdc406c290 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.338+0000 7efde87ff700 1 --2- 172.21.15.111:0/2854230427 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efde015d590 0x7efde01e63d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.338+0000 7efde87ff700 1 --2- 172.21.15.111:0/2854230427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efdcc004850 0x7efde0155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.338+0000 7efde87ff700 1 --2- 172.21.15.111:0/2854230427 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efde015cbe0 0x7efde015d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.338+0000 7efde87ff700 1 -- 172.21.15.111:0/2854230427 >> 172.21.15.111:0/2854230427 conn(0x7efde0071b10 msgr2=0x7efde0160030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:32.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.338+0000 7efde87ff700 1 -- 172.21.15.111:0/2854230427 shutdown_connections 2023-03-29T10:57:32.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.338+0000 7efde87ff700 1 -- 172.21.15.111:0/2854230427 wait complete. 2023-03-29T10:57:32.339 INFO:teuthology.orchestra.run.smithi111.stderr:added data pool 5 to fsmap 2023-03-29T10:57:32.355 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-29T10:57:32.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.538+0000 7f93393ff700 1 Processor -- start 2023-03-29T10:57:32.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.538+0000 7f93393ff700 1 -- start start 2023-03-29T10:57:32.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.539+0000 7f93393ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93340cf950 0x7f93340cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.539+0000 7f93393ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f93340d3a30 0x7f93340b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.539+0000 7f93393ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93340cdc80 con 0x7f933415c950 2023-03-29T10:57:32.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.539+0000 7f93393ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93340d0260 con 0x7f93340d3a30 2023-03-29T10:57:32.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.539+0000 7f93393ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93340cd6c0 con 0x7f93340cf950 2023-03-29T10:57:32.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.539+0000 7f93327fc700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f933415c950 0x7f93340cf0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:32942/0 (socket says 172.21.15.111:32942) 2023-03-29T10:57:32.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.539+0000 7f93327fc700 1 -- 172.21.15.111:0/1174029865 learned_addr learned my addr 172.21.15.111:0/1174029865 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:32.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.539+0000 7f93337fe700 1 --2- 172.21.15.111:0/1174029865 >> v2:172.21.15.111:3300/0 conn(0x7f93340d3a30 0x7f93340b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.539+0000 7f9332ffd700 1 --2- 172.21.15.111:0/1174029865 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93340cf950 0x7f93340cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.540+0000 7f9331ffb700 1 -- 172.21.15.111:0/1174029865 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788429867 0 0) 0x7f93340cdc80 con 0x7f933415c950 2023-03-29T10:57:32.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.540+0000 7f9331ffb700 1 -- 172.21.15.111:0/1174029865 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f931c0035b0 con 0x7f933415c950 2023-03-29T10:57:32.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.540+0000 7f93337fe700 1 -- 172.21.15.111:0/1174029865 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93340cf950 msgr2=0x7f93340cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:32.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.540+0000 7f93337fe700 1 --2- 172.21.15.111:0/1174029865 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93340cf950 0x7f93340cfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.540+0000 7f93337fe700 1 -- 172.21.15.111:0/1174029865 >> v1:172.21.15.111:6789/0 conn(0x7f933415c950 legacy=0x7f93340cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:32.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.540+0000 7f93337fe700 1 -- 172.21.15.111:0/1174029865 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93340cd9a0 con 0x7f93340d3a30 2023-03-29T10:57:32.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.540+0000 7f93337fe700 1 --2- 172.21.15.111:0/1174029865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93340d3a30 0x7f93340b5d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f932c009d90 tx=0x7f932c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40182cefe5fcd412 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:32.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.540+0000 7f9331ffb700 1 -- 172.21.15.111:0/1174029865 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c00e070 con 0x7f93340d3a30 2023-03-29T10:57:32.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.540+0000 7f9331ffb700 1 -- 172.21.15.111:0/1174029865 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932c00bb20 con 0x7f93340d3a30 2023-03-29T10:57:32.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.540+0000 7f9331ffb700 1 -- 172.21.15.111:0/1174029865 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c012430 con 0x7f93340d3a30 2023-03-29T10:57:32.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.541+0000 7f93393ff700 1 -- 172.21.15.111:0/1174029865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93340d3a30 msgr2=0x7f93340b5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:32.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.541+0000 7f93393ff700 1 --2- 172.21.15.111:0/1174029865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93340d3a30 0x7f93340b5d90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f932c009d90 tx=0x7f932c00b330 comp rx=0 tx=0).stop 2023-03-29T10:57:32.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.541+0000 7f93393ff700 1 -- 172.21.15.111:0/1174029865 shutdown_connections 2023-03-29T10:57:32.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.541+0000 7f93393ff700 1 --2- 172.21.15.111:0/1174029865 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93340d3a30 0x7f93340b5d90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.541+0000 7f93393ff700 1 --2- 172.21.15.111:0/1174029865 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93340cf950 0x7f93340cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.541+0000 7f93393ff700 1 -- 172.21.15.111:0/1174029865 >> 172.21.15.111:0/1174029865 conn(0x7f9334071b10 msgr2=0x7f9334071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:32.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.541+0000 7f93393ff700 1 -- 172.21.15.111:0/1174029865 shutdown_connections 2023-03-29T10:57:32.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.541+0000 7f93393ff700 1 -- 172.21.15.111:0/1174029865 wait complete. 2023-03-29T10:57:32.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.543+0000 7f93393ff700 1 Processor -- start 2023-03-29T10:57:32.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.544+0000 7f93393ff700 1 -- start start 2023-03-29T10:57:32.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.544+0000 7f93393ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93340cf950 0x7f933415f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.544+0000 7f93393ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93340d3a30 0x7f93340ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.544+0000 7f93393ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f933415c950 0x7f93340ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.544+0000 7f93393ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933415fb80 con 0x7f93340cf950 2023-03-29T10:57:32.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.544+0000 7f93393ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93340c8870 con 0x7f933415c950 2023-03-29T10:57:32.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.544+0000 7f93393ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93340c8b50 con 0x7f93340d3a30 2023-03-29T10:57:32.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.545+0000 7f9332ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93340cf950 0x7f933415f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.545+0000 7f9332ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93340cf950 0x7f933415f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38850/0 (socket says 172.21.15.111:38850) 2023-03-29T10:57:32.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.545+0000 7f9332ffd700 1 -- 172.21.15.111:0/1881806486 learned_addr learned my addr 172.21.15.111:0/1881806486 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:32.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.545+0000 7f93327fc700 1 --2- 172.21.15.111:0/1881806486 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93340d3a30 0x7f93340ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.545+0000 7f93337fe700 1 --2- 172.21.15.111:0/1881806486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f933415c950 0x7f93340ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.545+0000 7f9332ffd700 1 -- 172.21.15.111:0/1881806486 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93340d3a30 msgr2=0x7f93340ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:32.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.545+0000 7f9332ffd700 1 --2- 172.21.15.111:0/1881806486 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93340d3a30 0x7f93340ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.545+0000 7f9332ffd700 1 -- 172.21.15.111:0/1881806486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f933415c950 msgr2=0x7f93340ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:32.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.545+0000 7f9332ffd700 1 --2- 172.21.15.111:0/1881806486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f933415c950 0x7f93340ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.545+0000 7f9332ffd700 1 -- 172.21.15.111:0/1881806486 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c009a70 con 0x7f93340cf950 2023-03-29T10:57:32.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.545+0000 7f93337fe700 1 --2- 172.21.15.111:0/1881806486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f933415c950 0x7f93340ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:57:32.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.545+0000 7f9332ffd700 1 --2- 172.21.15.111:0/1881806486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93340cf950 0x7f933415f640 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f932400e9b0 tx=0x7f932400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:32.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.546+0000 7f931b7fe700 1 -- 172.21.15.111:0/1881806486 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324009710 con 0x7f93340cf950 2023-03-29T10:57:32.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.546+0000 7f931b7fe700 1 -- 172.21.15.111:0/1881806486 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932400c850 con 0x7f93340cf950 2023-03-29T10:57:32.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.546+0000 7f931b7fe700 1 -- 172.21.15.111:0/1881806486 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932400cbc0 con 0x7f93340cf950 2023-03-29T10:57:32.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.546+0000 7f93393ff700 1 -- 172.21.15.111:0/1881806486 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93340d19a0 con 0x7f93340cf950 2023-03-29T10:57:32.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.546+0000 7f93393ff700 1 -- 172.21.15.111:0/1881806486 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93340d16c0 con 0x7f93340cf950 2023-03-29T10:57:32.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.547+0000 7f93393ff700 1 -- 172.21.15.111:0/1881806486 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93340b62d0 con 0x7f93340cf950 2023-03-29T10:57:32.547 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.548+0000 7f931b7fe700 1 -- 172.21.15.111:0/1881806486 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f932400cd20 con 0x7f93340cf950 2023-03-29T10:57:32.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.548+0000 7f931b7fe700 1 --2- 172.21.15.111:0/1881806486 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9320069de0 0x7f932006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.549+0000 7f931b7fe700 1 -- 172.21.15.111:0/1881806486 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9324088480 con 0x7f93340cf950 2023-03-29T10:57:32.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.549+0000 7f93327fc700 1 --2- 172.21.15.111:0/1881806486 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9320069de0 0x7f932006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.549+0000 7f93327fc700 1 --2- 172.21.15.111:0/1881806486 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9320069de0 0x7f932006c290 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f93340c77a0 tx=0x7f9328007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:32.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.552+0000 7f931b7fe700 1 -- 172.21.15.111:0/1881806486 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f9324053910 con 0x7f93340cf950 2023-03-29T10:57:32.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.788+0000 7f93393ff700 1 -- 172.21.15.111:0/1881806486 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f933415ac20 con 0x7f93340cf950 2023-03-29T10:57:32.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.789+0000 7f931b7fe700 1 -- 172.21.15.111:0/1881806486 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f9324056f30 con 0x7f93340cf950 2023-03-29T10:57:32.789 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:32.789 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/556826332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":556826332},{"type":"v1","addr":"172.21.15.111:6837","nonce":556826332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/814623168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":814623168},{"type":"v1","addr":"172.21.15.111:6835","nonce":814623168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/514422035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":514422035},{"type":"v1","addr":"172.21.15.119:6834","nonce":514422035}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-29T10:57:27.824135+0000","modified":"2023-03-29T10:57:32.332114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/3160800581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":3160800581},{"type":"v1","addr":"172.21.15.119:6835","nonce":3160800581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-03-29T10:57:32.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.791+0000 7f93393ff700 1 -- 172.21.15.111:0/1881806486 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9320069de0 msgr2=0x7f932006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:32.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.791+0000 7f93393ff700 1 --2- 172.21.15.111:0/1881806486 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9320069de0 0x7f932006c290 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f93340c77a0 tx=0x7f9328007dc0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.791+0000 7f93393ff700 1 -- 172.21.15.111:0/1881806486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93340cf950 msgr2=0x7f933415f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:32.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.791+0000 7f93393ff700 1 --2- 172.21.15.111:0/1881806486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93340cf950 0x7f933415f640 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f932400e9b0 tx=0x7f932400ecc0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.791+0000 7f93393ff700 1 -- 172.21.15.111:0/1881806486 shutdown_connections 2023-03-29T10:57:32.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.792+0000 7f93393ff700 1 --2- 172.21.15.111:0/1881806486 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9320069de0 0x7f932006c290 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.792+0000 7f93393ff700 1 --2- 172.21.15.111:0/1881806486 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93340cf950 0x7f933415f640 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.792+0000 7f93393ff700 1 --2- 172.21.15.111:0/1881806486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f933415c950 0x7f93340ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.792+0000 7f93393ff700 1 --2- 172.21.15.111:0/1881806486 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93340d3a30 0x7f93340ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.792+0000 7f93393ff700 1 -- 172.21.15.111:0/1881806486 >> 172.21.15.111:0/1881806486 conn(0x7f9334071b10 msgr2=0x7f93340d2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:32.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.792+0000 7f93393ff700 1 -- 172.21.15.111:0/1881806486 shutdown_connections 2023-03-29T10:57:32.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.792+0000 7f93393ff700 1 -- 172.21.15.111:0/1881806486 wait complete. 2023-03-29T10:57:32.793 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 7 2023-03-29T10:57:32.808 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:57:32.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.991+0000 7eff7f5ff700 1 Processor -- start 2023-03-29T10:57:32.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.991+0000 7eff7f5ff700 1 -- start start 2023-03-29T10:57:32.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.992+0000 7eff7f5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7eff780cfd40 0x7eff780d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.992+0000 7eff7f5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff780b70f0 0x7eff780b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.992+0000 7eff7f5ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff780cdc10 con 0x7eff780cf970 2023-03-29T10:57:32.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.992+0000 7eff7f5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff780b6c60 con 0x7eff780cfd40 2023-03-29T10:57:32.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.992+0000 7eff7f5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff780cd650 con 0x7eff780b70f0 2023-03-29T10:57:32.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.992+0000 7eff7d39b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7eff780cf970 0x7eff780cf110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:32952/0 (socket says 172.21.15.111:32952) 2023-03-29T10:57:32.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.992+0000 7eff7d39b700 1 -- 172.21.15.111:0/3989417424 learned_addr learned my addr 172.21.15.111:0/3989417424 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:32.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff7cb9a700 1 --2- 172.21.15.111:0/3989417424 >> v2:172.21.15.119:3300/0 conn(0x7eff780cfd40 0x7eff780d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff7db9c700 1 --2- 172.21.15.111:0/3989417424 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff780b70f0 0x7eff780b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff6ffff700 1 -- 172.21.15.111:0/3989417424 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4275306544 0 0) 0x7eff780cdc10 con 0x7eff780cf970 2023-03-29T10:57:32.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff6ffff700 1 -- 172.21.15.111:0/3989417424 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff640035b0 con 0x7eff780cf970 2023-03-29T10:57:32.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff6ffff700 1 -- 172.21.15.111:0/3989417424 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2601512430 0 0) 0x7eff640035b0 con 0x7eff780cf970 2023-03-29T10:57:32.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff6ffff700 1 -- 172.21.15.111:0/3989417424 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff780b70f0 msgr2=0x7eff780b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:32.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff6ffff700 1 --2- 172.21.15.111:0/3989417424 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff780b70f0 0x7eff780b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff6ffff700 1 -- 172.21.15.111:0/3989417424 >> v2:172.21.15.119:3300/0 conn(0x7eff780cfd40 msgr2=0x7eff780d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:32.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff6ffff700 1 --2- 172.21.15.111:0/3989417424 >> v2:172.21.15.119:3300/0 conn(0x7eff780cfd40 0x7eff780d01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff6ffff700 1 -- 172.21.15.111:0/3989417424 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff780cd930 con 0x7eff780cf970 2023-03-29T10:57:32.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff6ffff700 1 -- 172.21.15.111:0/3989417424 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7eff74001670 con 0x7eff780cf970 2023-03-29T10:57:32.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff6ffff700 1 -- 172.21.15.111:0/3989417424 >> v1:172.21.15.111:6789/0 conn(0x7eff780cf970 legacy=0x7eff780cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:32.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff6ffff700 1 --2- 172.21.15.111:0/3989417424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff64003a70 0x7eff64003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff6ffff700 1 --2- 172.21.15.111:0/3989417424 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff780b70f0 0x7eff64004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff6ffff700 1 --2- 172.21.15.111:0/3989417424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff64004d90 0x7eff64005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff6ffff700 1 -- 172.21.15.111:0/3989417424 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff780b6c60 con 0x7eff64003a70 2023-03-29T10:57:32.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff6ffff700 1 -- 172.21.15.111:0/3989417424 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff780cd650 con 0x7eff64004d90 2023-03-29T10:57:32.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff6ffff700 1 -- 172.21.15.111:0/3989417424 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6400ba60 con 0x7eff780b70f0 2023-03-29T10:57:32.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff7cb9a700 1 --2- 172.21.15.111:0/3989417424 >> v2:172.21.15.119:3300/0 conn(0x7eff780cfd40 0x7eff780d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:32.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.993+0000 7eff7d39b700 1 --2- 172.21.15.111:0/3989417424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff64003a70 0x7eff64003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.994+0000 7eff7db9c700 1 --2- 172.21.15.111:0/3989417424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff64004d90 0x7eff64005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.994+0000 7eff7cb9a700 1 --2- 172.21.15.111:0/3989417424 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff780b70f0 0x7eff64004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.994+0000 7eff7d39b700 1 -- 172.21.15.111:0/3989417424 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff780b70f0 msgr2=0x7eff64004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:32.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.994+0000 7eff7d39b700 1 --2- 172.21.15.111:0/3989417424 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff780b70f0 0x7eff64004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.994+0000 7eff7d39b700 1 -- 172.21.15.111:0/3989417424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff64004d90 msgr2=0x7eff64005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:32.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.994+0000 7eff7d39b700 1 --2- 172.21.15.111:0/3989417424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff64004d90 0x7eff64005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.994+0000 7eff7d39b700 1 -- 172.21.15.111:0/3989417424 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff640035b0 con 0x7eff64003a70 2023-03-29T10:57:32.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.994+0000 7eff7d39b700 1 --2- 172.21.15.111:0/3989417424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff64003a70 0x7eff64003e80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7eff7400c350 tx=0x7eff64005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4164d9f482b067da server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:32.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.994+0000 7eff6ffff700 1 -- 172.21.15.111:0/3989417424 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7400ce50 con 0x7eff64003a70 2023-03-29T10:57:32.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.994+0000 7eff6ffff700 1 -- 172.21.15.111:0/3989417424 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff74005960 con 0x7eff64003a70 2023-03-29T10:57:32.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.994+0000 7eff6ffff700 1 -- 172.21.15.111:0/3989417424 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff74005d10 con 0x7eff64003a70 2023-03-29T10:57:32.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.995+0000 7eff7f5ff700 1 -- 172.21.15.111:0/3989417424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff64003a70 msgr2=0x7eff64003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:32.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.995+0000 7eff7f5ff700 1 --2- 172.21.15.111:0/3989417424 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff64003a70 0x7eff64003e80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7eff7400c350 tx=0x7eff64005880 comp rx=0 tx=0).stop 2023-03-29T10:57:32.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.995+0000 7eff7db9c700 1 -- 172.21.15.111:0/3989417424 reap_dead start 2023-03-29T10:57:32.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.995+0000 7eff7f5ff700 1 -- 172.21.15.111:0/3989417424 shutdown_connections 2023-03-29T10:57:32.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.995+0000 7eff7f5ff700 1 -- 172.21.15.111:0/3989417424 >> 172.21.15.111:0/3989417424 conn(0x7eff78071b10 msgr2=0x7eff78071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:32.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.995+0000 7eff7f5ff700 1 -- 172.21.15.111:0/3989417424 shutdown_connections 2023-03-29T10:57:32.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.995+0000 7eff7f5ff700 1 -- 172.21.15.111:0/3989417424 wait complete. 2023-03-29T10:57:32.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.997+0000 7eff7f5ff700 1 Processor -- start 2023-03-29T10:57:32.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.997+0000 7eff7f5ff700 1 -- start start 2023-03-29T10:57:32.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.998+0000 7eff7f5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff64003a70 0x7eff78157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.998+0000 7eff7f5ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff780b36f0 0x7eff780b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.998+0000 7eff7f5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff7815cee0 0x7eff7815d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:32.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.998+0000 7eff7f5ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff780b40a0 con 0x7eff780b36f0 2023-03-29T10:57:32.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.998+0000 7eff7f5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff780ca680 con 0x7eff7815cee0 2023-03-29T10:57:32.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.998+0000 7eff7f5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7815dc10 con 0x7eff64003a70 2023-03-29T10:57:32.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.998+0000 7eff7cb9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff780b36f0 0x7eff780b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.998+0000 7eff7cb9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff780b36f0 0x7eff780b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38862/0 (socket says 172.21.15.111:38862) 2023-03-29T10:57:32.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.998+0000 7eff7cb9a700 1 -- 172.21.15.111:0/2119903325 learned_addr learned my addr 172.21.15.111:0/2119903325 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:32.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.998+0000 7eff7db9c700 1 --2- 172.21.15.111:0/2119903325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff7815cee0 0x7eff7815d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.998+0000 7eff7d39b700 1 --2- 172.21.15.111:0/2119903325 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff64003a70 0x7eff78157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:32.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.998+0000 7eff7cb9a700 1 -- 172.21.15.111:0/2119903325 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff64003a70 msgr2=0x7eff78157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:32.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.998+0000 7eff7cb9a700 1 --2- 172.21.15.111:0/2119903325 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff64003a70 0x7eff78157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.998+0000 7eff7cb9a700 1 -- 172.21.15.111:0/2119903325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff7815cee0 msgr2=0x7eff7815d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:32.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.998+0000 7eff7cb9a700 1 --2- 172.21.15.111:0/2119903325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff7815cee0 0x7eff7815d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:32.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.998+0000 7eff7cb9a700 1 -- 172.21.15.111:0/2119903325 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff780d0700 con 0x7eff780b36f0 2023-03-29T10:57:32.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.998+0000 7eff7cb9a700 1 --2- 172.21.15.111:0/2119903325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff780b36f0 0x7eff780b3b60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7eff68010a50 tx=0x7eff68010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:33.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.999+0000 7eff6dffb700 1 -- 172.21.15.111:0/2119903325 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff680097a0 con 0x7eff780b36f0 2023-03-29T10:57:33.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.999+0000 7eff6dffb700 1 -- 172.21.15.111:0/2119903325 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6800e8d0 con 0x7eff780b36f0 2023-03-29T10:57:33.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.999+0000 7eff6dffb700 1 -- 172.21.15.111:0/2119903325 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6800ec60 con 0x7eff780b36f0 2023-03-29T10:57:33.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.999+0000 7eff7f5ff700 1 -- 172.21.15.111:0/2119903325 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff780b68f0 con 0x7eff780b36f0 2023-03-29T10:57:33.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:32.999+0000 7eff7f5ff700 1 -- 172.21.15.111:0/2119903325 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff780d36d0 con 0x7eff780b36f0 2023-03-29T10:57:33.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.000+0000 7eff7f5ff700 1 -- 172.21.15.111:0/2119903325 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff780ca7f0 con 0x7eff780b36f0 2023-03-29T10:57:33.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.001+0000 7eff6dffb700 1 -- 172.21.15.111:0/2119903325 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7eff68016460 con 0x7eff780b36f0 2023-03-29T10:57:33.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.002+0000 7eff6dffb700 1 --2- 172.21.15.111:0/2119903325 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7eff5c069de0 0x7eff5c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:33.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.002+0000 7eff6dffb700 1 -- 172.21.15.111:0/2119903325 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7eff68088650 con 0x7eff780b36f0 2023-03-29T10:57:33.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.002+0000 7eff7d39b700 1 --2- 172.21.15.111:0/2119903325 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7eff5c069de0 0x7eff5c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:33.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.002+0000 7eff7d39b700 1 --2- 172.21.15.111:0/2119903325 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7eff5c069de0 0x7eff5c06c290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff74008fd0 tx=0x7eff7400a000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:33.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.006+0000 7eff6dffb700 1 -- 172.21.15.111:0/2119903325 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7eff68053bb0 con 0x7eff780b36f0 2023-03-29T10:57:33.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.202+0000 7eff7f5ff700 1 -- 172.21.15.111:0/2119903325 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff78074430 con 0x7eff780b36f0 2023-03-29T10:57:33.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.203+0000 7eff6dffb700 1 -- 172.21.15.111:0/2119903325 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19796 (secure 0 0 0) 0x7eff680571d0 con 0x7eff780b36f0 2023-03-29T10:57:33.203 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:33.204 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":28,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:57:32.331880+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:57:33.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.206+0000 7eff7f5ff700 1 -- 172.21.15.111:0/2119903325 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7eff5c069de0 msgr2=0x7eff5c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:33.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.206+0000 7eff7f5ff700 1 --2- 172.21.15.111:0/2119903325 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7eff5c069de0 0x7eff5c06c290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff74008fd0 tx=0x7eff7400a000 comp rx=0 tx=0).stop 2023-03-29T10:57:33.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.206+0000 7eff7f5ff700 1 -- 172.21.15.111:0/2119903325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff780b36f0 msgr2=0x7eff780b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:33.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.206+0000 7eff7f5ff700 1 --2- 172.21.15.111:0/2119903325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff780b36f0 0x7eff780b3b60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7eff68010a50 tx=0x7eff68010d60 comp rx=0 tx=0).stop 2023-03-29T10:57:33.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.206+0000 7eff7f5ff700 1 -- 172.21.15.111:0/2119903325 shutdown_connections 2023-03-29T10:57:33.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.206+0000 7eff7f5ff700 1 --2- 172.21.15.111:0/2119903325 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7eff5c069de0 0x7eff5c06c290 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.206+0000 7eff7f5ff700 1 --2- 172.21.15.111:0/2119903325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff7815cee0 0x7eff7815d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.206+0000 7eff7f5ff700 1 --2- 172.21.15.111:0/2119903325 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff64003a70 0x7eff78157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.206+0000 7eff7f5ff700 1 --2- 172.21.15.111:0/2119903325 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff780b36f0 0x7eff780b3b60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.206+0000 7eff7f5ff700 1 -- 172.21.15.111:0/2119903325 >> 172.21.15.111:0/2119903325 conn(0x7eff78071b10 msgr2=0x7eff780d22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:33.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.206+0000 7eff7f5ff700 1 -- 172.21.15.111:0/2119903325 shutdown_connections 2023-03-29T10:57:33.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.206+0000 7eff7f5ff700 1 -- 172.21.15.111:0/2119903325 wait complete. 2023-03-29T10:57:33.223 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:57:33.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.407+0000 7f8c723ff700 1 Processor -- start 2023-03-29T10:57:33.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.407+0000 7f8c723ff700 1 -- start start 2023-03-29T10:57:33.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.408+0000 7f8c723ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8c6c0cf970 0x7f8c6c0cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:33.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.408+0000 7f8c723ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8c6c0cfd40 0x7f8c6c0d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:33.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.408+0000 7f8c723ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c6c160290 0x7f8c6c1606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:33.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.408+0000 7f8c723ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c6c0cdc10 con 0x7f8c6c0cf970 2023-03-29T10:57:33.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.408+0000 7f8c723ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c6c0b6c60 con 0x7f8c6c0cfd40 2023-03-29T10:57:33.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.408+0000 7f8c723ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0cd650 con 0x7f8c6c160290 2023-03-29T10:57:33.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.408+0000 7f8c6bfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8c6c0cf970 0x7f8c6c0cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:33.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.409+0000 7f8c6bfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8c6c0cf970 0x7f8c6c0cf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38872/0 (socket says 172.21.15.111:38872) 2023-03-29T10:57:33.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.409+0000 7f8c6bfff700 1 -- 172.21.15.111:0/2751070327 learned_addr learned my addr 172.21.15.111:0/2751070327 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:33.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.409+0000 7f8c6bfff700 1 -- 172.21.15.111:0/2751070327 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c6c160290 msgr2=0x7f8c6c1606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:33.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.409+0000 7f8c6b7fe700 1 --2- 172.21.15.111:0/2751070327 >> v2:172.21.15.119:3300/0 conn(0x7f8c6c0cfd40 0x7f8c6c0d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:33.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.409+0000 7f8c6bfff700 1 --2- 172.21.15.111:0/2751070327 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c6c160290 0x7f8c6c1606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.409+0000 7f8c6bfff700 1 -- 172.21.15.111:0/2751070327 >> v2:172.21.15.119:3300/0 conn(0x7f8c6c0cfd40 msgr2=0x7f8c6c0d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:33.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.409+0000 7f8c6bfff700 1 --2- 172.21.15.111:0/2751070327 >> v2:172.21.15.119:3300/0 conn(0x7f8c6c0cfd40 0x7f8c6c0d01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.409+0000 7f8c6bfff700 1 -- 172.21.15.111:0/2751070327 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6c0cd930 con 0x7f8c6c0cf970 2023-03-29T10:57:33.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.409+0000 7f8c6b7fe700 1 --2- 172.21.15.111:0/2751070327 >> v2:172.21.15.119:3300/0 conn(0x7f8c6c0cfd40 0x7f8c6c0d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:33.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.409+0000 7f8c6bfff700 1 --2- 172.21.15.111:0/2751070327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c6c0cf970 0x7f8c6c0cf110 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0b8c40 tx=0x7f8c5c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cf99d2c9ba2a7d server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:33.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.409+0000 7f8c6affd700 1 -- 172.21.15.111:0/2751070327 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c00b870 con 0x7f8c6c0cf970 2023-03-29T10:57:33.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.409+0000 7f8c6affd700 1 -- 172.21.15.111:0/2751070327 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5c00b9d0 con 0x7f8c6c0cf970 2023-03-29T10:57:33.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.409+0000 7f8c6affd700 1 -- 172.21.15.111:0/2751070327 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c00bd60 con 0x7f8c6c0cf970 2023-03-29T10:57:33.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.410+0000 7f8c723ff700 1 -- 172.21.15.111:0/2751070327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c6c0cf970 msgr2=0x7f8c6c0cf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:33.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.410+0000 7f8c723ff700 1 --2- 172.21.15.111:0/2751070327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c6c0cf970 0x7f8c6c0cf110 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0b8c40 tx=0x7f8c5c009bd0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.410+0000 7f8c723ff700 1 -- 172.21.15.111:0/2751070327 shutdown_connections 2023-03-29T10:57:33.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.410+0000 7f8c723ff700 1 --2- 172.21.15.111:0/2751070327 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c6c160290 0x7f8c6c1606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.410+0000 7f8c723ff700 1 --2- 172.21.15.111:0/2751070327 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c6c0cf970 0x7f8c6c0cf110 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.410+0000 7f8c723ff700 1 --2- 172.21.15.111:0/2751070327 >> v2:172.21.15.119:3300/0 conn(0x7f8c6c0cfd40 0x7f8c6c0d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.410+0000 7f8c723ff700 1 -- 172.21.15.111:0/2751070327 >> 172.21.15.111:0/2751070327 conn(0x7f8c6c071b10 msgr2=0x7f8c6c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:33.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.410+0000 7f8c723ff700 1 -- 172.21.15.111:0/2751070327 shutdown_connections 2023-03-29T10:57:33.413 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.410+0000 7f8c723ff700 1 -- 172.21.15.111:0/2751070327 wait complete. 2023-03-29T10:57:33.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.412+0000 7f8c723ff700 1 Processor -- start 2023-03-29T10:57:33.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.412+0000 7f8c723ff700 1 -- start start 2023-03-29T10:57:33.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.413+0000 7f8c723ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c6c0cf970 0x7f8c6c0cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:33.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.413+0000 7f8c723ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c6c0cfd40 0x7f8c6c15e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:33.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.413+0000 7f8c723ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c6c160290 0x7f8c6c15aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:33.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.413+0000 7f8c723ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c15af90 con 0x7f8c6c0cfd40 2023-03-29T10:57:33.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.413+0000 7f8c723ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0ccb30 con 0x7f8c6c0cf970 2023-03-29T10:57:33.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.413+0000 7f8c723ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0cbfb0 con 0x7f8c6c160290 2023-03-29T10:57:33.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.413+0000 7f8c6b7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c6c0cfd40 0x7f8c6c15e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:33.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.413+0000 7f8c6b7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c6c0cfd40 0x7f8c6c15e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38884/0 (socket says 172.21.15.111:38884) 2023-03-29T10:57:33.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.413+0000 7f8c6b7fe700 1 -- 172.21.15.111:0/554001820 learned_addr learned my addr 172.21.15.111:0/554001820 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:33.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.413+0000 7f8c6bfff700 1 --2- 172.21.15.111:0/554001820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c6c0cf970 0x7f8c6c0cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:33.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.413+0000 7f8c7099c700 1 --2- 172.21.15.111:0/554001820 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c6c160290 0x7f8c6c15aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:33.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.413+0000 7f8c6b7fe700 1 -- 172.21.15.111:0/554001820 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c6c160290 msgr2=0x7f8c6c15aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:33.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.413+0000 7f8c6b7fe700 1 --2- 172.21.15.111:0/554001820 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c6c160290 0x7f8c6c15aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.413+0000 7f8c6b7fe700 1 -- 172.21.15.111:0/554001820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c6c0cf970 msgr2=0x7f8c6c0cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:33.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.413+0000 7f8c6b7fe700 1 --2- 172.21.15.111:0/554001820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c6c0cf970 0x7f8c6c0cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.413+0000 7f8c6b7fe700 1 -- 172.21.15.111:0/554001820 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5c009550 con 0x7f8c6c0cfd40 2023-03-29T10:57:33.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.414+0000 7f8c6b7fe700 1 --2- 172.21.15.111:0/554001820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c6c0cfd40 0x7f8c6c15e7f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000c7f0 tx=0x7f8c6000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:33.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.414+0000 7f8c68ff9700 1 -- 172.21.15.111:0/554001820 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c60011800 con 0x7f8c6c0cfd40 2023-03-29T10:57:33.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.414+0000 7f8c68ff9700 1 -- 172.21.15.111:0/554001820 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c60011960 con 0x7f8c6c0cfd40 2023-03-29T10:57:33.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.414+0000 7f8c68ff9700 1 -- 172.21.15.111:0/554001820 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c60011cf0 con 0x7f8c6c0cfd40 2023-03-29T10:57:33.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.414+0000 7f8c723ff700 1 -- 172.21.15.111:0/554001820 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6c0cf650 con 0x7f8c6c0cfd40 2023-03-29T10:57:33.418 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.414+0000 7f8c723ff700 1 -- 172.21.15.111:0/554001820 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6c0b68f0 con 0x7f8c6c0cfd40 2023-03-29T10:57:33.418 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.415+0000 7f8c723ff700 1 -- 172.21.15.111:0/554001820 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c6c0d0700 con 0x7f8c6c0cfd40 2023-03-29T10:57:33.418 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.416+0000 7f8c68ff9700 1 -- 172.21.15.111:0/554001820 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f8c60018360 con 0x7f8c6c0cfd40 2023-03-29T10:57:33.419 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.417+0000 7f8c68ff9700 1 --2- 172.21.15.111:0/554001820 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8c58069de0 0x7f8c5806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:33.419 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.417+0000 7f8c68ff9700 1 -- 172.21.15.111:0/554001820 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8c60088120 con 0x7f8c6c0cfd40 2023-03-29T10:57:33.419 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.417+0000 7f8c6bfff700 1 --2- 172.21.15.111:0/554001820 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8c58069de0 0x7f8c5806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:33.419 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.418+0000 7f8c6bfff700 1 --2- 172.21.15.111:0/554001820 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8c58069de0 0x7f8c5806c290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c014040 tx=0x7f8c5c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:33.420 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.421+0000 7f8c68ff9700 1 -- 172.21.15.111:0/554001820 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f8c60053670 con 0x7f8c6c0cfd40 2023-03-29T10:57:33.551 INFO:tasks.ceph.mds.d.smithi119.stderr:2023-03-29T10:57:33.551+0000 7ff9fd1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-03-29T10:57:33.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.618+0000 7f8c723ff700 1 -- 172.21.15.111:0/554001820 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c6c074430 con 0x7f8c6c0cfd40 2023-03-29T10:57:33.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.619+0000 7f8c68ff9700 1 -- 172.21.15.111:0/554001820 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19796 (secure 0 0 0) 0x7f8c60056c90 con 0x7f8c6c0cfd40 2023-03-29T10:57:33.619 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:33.619 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":28,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:57:32.331880+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:57:33.621 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.622+0000 7f8c723ff700 1 -- 172.21.15.111:0/554001820 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8c58069de0 msgr2=0x7f8c5806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:33.621 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.622+0000 7f8c723ff700 1 --2- 172.21.15.111:0/554001820 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8c58069de0 0x7f8c5806c290 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c014040 tx=0x7f8c5c008030 comp rx=0 tx=0).stop 2023-03-29T10:57:33.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.622+0000 7f8c723ff700 1 -- 172.21.15.111:0/554001820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c6c0cfd40 msgr2=0x7f8c6c15e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:33.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.622+0000 7f8c723ff700 1 --2- 172.21.15.111:0/554001820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c6c0cfd40 0x7f8c6c15e7f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000c7f0 tx=0x7f8c6000cb00 comp rx=0 tx=0).stop 2023-03-29T10:57:33.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.622+0000 7f8c723ff700 1 -- 172.21.15.111:0/554001820 shutdown_connections 2023-03-29T10:57:33.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.622+0000 7f8c723ff700 1 --2- 172.21.15.111:0/554001820 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8c58069de0 0x7f8c5806c290 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.622+0000 7f8c723ff700 1 --2- 172.21.15.111:0/554001820 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c6c160290 0x7f8c6c15aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.622+0000 7f8c723ff700 1 --2- 172.21.15.111:0/554001820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c6c0cf970 0x7f8c6c0cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.622+0000 7f8c723ff700 1 --2- 172.21.15.111:0/554001820 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c6c0cfd40 0x7f8c6c15e7f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.622+0000 7f8c723ff700 1 -- 172.21.15.111:0/554001820 >> 172.21.15.111:0/554001820 conn(0x7f8c6c071b10 msgr2=0x7f8c6c0b70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:33.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.622+0000 7f8c723ff700 1 -- 172.21.15.111:0/554001820 shutdown_connections 2023-03-29T10:57:33.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.622+0000 7f8c723ff700 1 -- 172.21.15.111:0/554001820 wait complete. 2023-03-29T10:57:33.652 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-29T10:57:33.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.834+0000 7f8aeedff700 1 Processor -- start 2023-03-29T10:57:33.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.834+0000 7f8aeedff700 1 -- start start 2023-03-29T10:57:33.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.835+0000 7f8aeedff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ae80cfd40 0x7f8ae80d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:33.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.835+0000 7f8aeedff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8ae80b70f0 0x7f8ae80b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:33.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.835+0000 7f8aeedff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae80cdc10 con 0x7f8ae80cf970 2023-03-29T10:57:33.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.835+0000 7f8aeedff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae80b6c60 con 0x7f8ae80b70f0 2023-03-29T10:57:33.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.836+0000 7f8aeedff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae80cd650 con 0x7f8ae80cfd40 2023-03-29T10:57:33.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.836+0000 7f8aecb9b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f8ae80cf970 0x7f8ae80cf110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:32960/0 (socket says 172.21.15.111:32960) 2023-03-29T10:57:33.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.836+0000 7f8aecb9b700 1 -- 172.21.15.111:0/3317796944 learned_addr learned my addr 172.21.15.111:0/3317796944 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:33.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.836+0000 7f8aed39c700 1 --2- 172.21.15.111:0/3317796944 >> v2:172.21.15.111:3300/0 conn(0x7f8ae80b70f0 0x7f8ae80b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:33.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.836+0000 7f8ae7fff700 1 --2- 172.21.15.111:0/3317796944 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ae80cfd40 0x7f8ae80d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:33.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.836+0000 7f8ae77fe700 1 -- 172.21.15.111:0/3317796944 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 139638830 0 0) 0x7f8ae80cdc10 con 0x7f8ae80cf970 2023-03-29T10:57:33.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.836+0000 7f8ae77fe700 1 -- 172.21.15.111:0/3317796944 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ad80035b0 con 0x7f8ae80cf970 2023-03-29T10:57:33.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.836+0000 7f8aed39c700 1 -- 172.21.15.111:0/3317796944 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ae80cfd40 msgr2=0x7f8ae80d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:33.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.836+0000 7f8aed39c700 1 --2- 172.21.15.111:0/3317796944 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ae80cfd40 0x7f8ae80d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.836+0000 7f8aed39c700 1 -- 172.21.15.111:0/3317796944 >> v1:172.21.15.111:6789/0 conn(0x7f8ae80cf970 legacy=0x7f8ae80cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:33.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.836+0000 7f8aed39c700 1 -- 172.21.15.111:0/3317796944 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae80cd930 con 0x7f8ae80b70f0 2023-03-29T10:57:33.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.836+0000 7f8aed39c700 1 --2- 172.21.15.111:0/3317796944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ae80b70f0 0x7f8ae80b7530 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0009d90 tx=0x7f8ae000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b378a7d59ec5897f server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:33.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.837+0000 7f8ae77fe700 1 -- 172.21.15.111:0/3317796944 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae000e070 con 0x7f8ae80b70f0 2023-03-29T10:57:33.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.837+0000 7f8ae77fe700 1 -- 172.21.15.111:0/3317796944 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae000bb20 con 0x7f8ae80b70f0 2023-03-29T10:57:33.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.837+0000 7f8ae77fe700 1 -- 172.21.15.111:0/3317796944 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0012430 con 0x7f8ae80b70f0 2023-03-29T10:57:33.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.837+0000 7f8aeedff700 1 -- 172.21.15.111:0/3317796944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ae80b70f0 msgr2=0x7f8ae80b7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:33.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.837+0000 7f8aeedff700 1 --2- 172.21.15.111:0/3317796944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ae80b70f0 0x7f8ae80b7530 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8ae0009d90 tx=0x7f8ae000b330 comp rx=0 tx=0).stop 2023-03-29T10:57:33.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.837+0000 7f8aeedff700 1 -- 172.21.15.111:0/3317796944 shutdown_connections 2023-03-29T10:57:33.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.837+0000 7f8aeedff700 1 --2- 172.21.15.111:0/3317796944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ae80b70f0 0x7f8ae80b7530 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.837+0000 7f8aeedff700 1 --2- 172.21.15.111:0/3317796944 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ae80cfd40 0x7f8ae80d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.837+0000 7f8aeedff700 1 -- 172.21.15.111:0/3317796944 >> 172.21.15.111:0/3317796944 conn(0x7f8ae8071b10 msgr2=0x7f8ae8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:33.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.837+0000 7f8aeedff700 1 -- 172.21.15.111:0/3317796944 shutdown_connections 2023-03-29T10:57:33.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.837+0000 7f8aeedff700 1 -- 172.21.15.111:0/3317796944 wait complete. 2023-03-29T10:57:33.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.839+0000 7f8aeedff700 1 Processor -- start 2023-03-29T10:57:33.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.839+0000 7f8aeedff700 1 -- start start 2023-03-29T10:57:33.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.840+0000 7f8aeedff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ae80b70f0 0x7f8ae81498c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:33.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.840+0000 7f8aeedff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ae80cf970 0x7f8ae8147fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:33.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.840+0000 7f8aeedff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae80cfd40 0x7f8ae814a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:33.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.840+0000 7f8aeedff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8149430 con 0x7f8ae80b70f0 2023-03-29T10:57:33.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.840+0000 7f8aeedff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8149e00 con 0x7f8ae80cfd40 2023-03-29T10:57:33.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.840+0000 7f8aeedff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8148820 con 0x7f8ae80cf970 2023-03-29T10:57:33.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.840+0000 7f8aecb9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ae80b70f0 0x7f8ae81498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:33.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.840+0000 7f8aecb9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ae80b70f0 0x7f8ae81498c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38902/0 (socket says 172.21.15.111:38902) 2023-03-29T10:57:33.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.840+0000 7f8aecb9b700 1 -- 172.21.15.111:0/1249836880 learned_addr learned my addr 172.21.15.111:0/1249836880 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:33.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.841+0000 7f8ae7fff700 1 --2- 172.21.15.111:0/1249836880 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ae80cf970 0x7f8ae8147fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:33.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.841+0000 7f8aed39c700 1 --2- 172.21.15.111:0/1249836880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae80cfd40 0x7f8ae814a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:33.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.841+0000 7f8aecb9b700 1 -- 172.21.15.111:0/1249836880 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ae80cf970 msgr2=0x7f8ae8147fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:33.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.841+0000 7f8aecb9b700 1 --2- 172.21.15.111:0/1249836880 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ae80cf970 0x7f8ae8147fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.841+0000 7f8aecb9b700 1 -- 172.21.15.111:0/1249836880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae80cfd40 msgr2=0x7f8ae814a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:33.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.841+0000 7f8aecb9b700 1 --2- 172.21.15.111:0/1249836880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae80cfd40 0x7f8ae814a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:33.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.841+0000 7f8aecb9b700 1 -- 172.21.15.111:0/1249836880 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae0009a70 con 0x7f8ae80b70f0 2023-03-29T10:57:33.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.841+0000 7f8ae7fff700 1 --2- 172.21.15.111:0/1249836880 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ae80cf970 0x7f8ae8147fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:33.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.841+0000 7f8aed39c700 1 --2- 172.21.15.111:0/1249836880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae80cfd40 0x7f8ae814a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:33.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.841+0000 7f8aecb9b700 1 --2- 172.21.15.111:0/1249836880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ae80b70f0 0x7f8ae81498c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8adc001eb0 tx=0x7f8adc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:33.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.841+0000 7f8ae57fa700 1 -- 172.21.15.111:0/1249836880 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc005d70 con 0x7f8ae80b70f0 2023-03-29T10:57:33.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.841+0000 7f8ae57fa700 1 -- 172.21.15.111:0/1249836880 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8adc0070d0 con 0x7f8ae80b70f0 2023-03-29T10:57:33.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.841+0000 7f8aeedff700 1 -- 172.21.15.111:0/1249836880 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae80d06d0 con 0x7f8ae80b70f0 2023-03-29T10:57:33.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.841+0000 7f8ae57fa700 1 -- 172.21.15.111:0/1249836880 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc007460 con 0x7f8ae80b70f0 2023-03-29T10:57:33.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.841+0000 7f8aeedff700 1 -- 172.21.15.111:0/1249836880 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae80b68f0 con 0x7f8ae80b70f0 2023-03-29T10:57:33.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.842+0000 7f8aeedff700 1 -- 172.21.15.111:0/1249836880 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae80d0860 con 0x7f8ae80b70f0 2023-03-29T10:57:33.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.843+0000 7f8ae57fa700 1 -- 172.21.15.111:0/1249836880 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f8adc0075c0 con 0x7f8ae80b70f0 2023-03-29T10:57:33.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.844+0000 7f8ae57fa700 1 --2- 172.21.15.111:0/1249836880 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8acc069de0 0x7f8acc06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:33.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.845+0000 7f8ae57fa700 1 -- 172.21.15.111:0/1249836880 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8adc087970 con 0x7f8ae80b70f0 2023-03-29T10:57:33.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.845+0000 7f8ae7fff700 1 --2- 172.21.15.111:0/1249836880 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8acc069de0 0x7f8acc06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:33.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.845+0000 7f8ae7fff700 1 --2- 172.21.15.111:0/1249836880 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8acc069de0 0x7f8acc06c290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8ae81f5600 tx=0x7f8ad4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:33.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:33.848+0000 7f8ae57fa700 1 -- 172.21.15.111:0/1249836880 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f8adc052e00 con 0x7f8ae80b70f0 2023-03-29T10:57:34.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.080+0000 7f8aeedff700 1 -- 172.21.15.111:0/1249836880 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ae8149710 con 0x7f8ae80b70f0 2023-03-29T10:57:34.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.081+0000 7f8ae57fa700 1 -- 172.21.15.111:0/1249836880 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f8adc017090 con 0x7f8ae80b70f0 2023-03-29T10:57:34.081 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:34.081 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/556826332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":556826332},{"type":"v1","addr":"172.21.15.111:6837","nonce":556826332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/814623168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":814623168},{"type":"v1","addr":"172.21.15.111:6835","nonce":814623168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/514422035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":514422035},{"type":"v1","addr":"172.21.15.119:6834","nonce":514422035}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-29T10:57:27.824135+0000","modified":"2023-03-29T10:57:32.332114+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/3160800581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":3160800581},{"type":"v1","addr":"172.21.15.119:6835","nonce":3160800581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-03-29T10:57:34.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.083+0000 7f8aeedff700 1 -- 172.21.15.111:0/1249836880 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8acc069de0 msgr2=0x7f8acc06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:34.084 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.083+0000 7f8aeedff700 1 --2- 172.21.15.111:0/1249836880 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8acc069de0 0x7f8acc06c290 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8ae81f5600 tx=0x7f8ad4008040 comp rx=0 tx=0).stop 2023-03-29T10:57:34.084 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.083+0000 7f8aeedff700 1 -- 172.21.15.111:0/1249836880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ae80b70f0 msgr2=0x7f8ae81498c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:34.084 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.083+0000 7f8aeedff700 1 --2- 172.21.15.111:0/1249836880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ae80b70f0 0x7f8ae81498c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8adc001eb0 tx=0x7f8adc00c860 comp rx=0 tx=0).stop 2023-03-29T10:57:34.084 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.083+0000 7f8aeedff700 1 -- 172.21.15.111:0/1249836880 shutdown_connections 2023-03-29T10:57:34.084 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.083+0000 7f8aeedff700 1 --2- 172.21.15.111:0/1249836880 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8acc069de0 0x7f8acc06c290 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.084 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.083+0000 7f8aeedff700 1 --2- 172.21.15.111:0/1249836880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ae80b70f0 0x7f8ae81498c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.084 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.083+0000 7f8aeedff700 1 --2- 172.21.15.111:0/1249836880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ae80cfd40 0x7f8ae814a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.083+0000 7f8aeedff700 1 --2- 172.21.15.111:0/1249836880 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ae80cf970 0x7f8ae8147fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.083+0000 7f8aeedff700 1 -- 172.21.15.111:0/1249836880 >> 172.21.15.111:0/1249836880 conn(0x7f8ae8071b10 msgr2=0x7f8ae80b74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:34.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.083+0000 7f8aeedff700 1 -- 172.21.15.111:0/1249836880 shutdown_connections 2023-03-29T10:57:34.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.083+0000 7f8aeedff700 1 -- 172.21.15.111:0/1249836880 wait complete. 2023-03-29T10:57:34.085 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 7 2023-03-29T10:57:34.502 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-03-29T10:57:27.824135+0000', 'modified': '2023-03-29T10:57:32.332114+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4296}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4296': {'gid': 4296, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6835/3160800581', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6833', 'nonce': 3160800581}, {'type': 'v1', 'addr': '172.21.15.119:6835', 'nonce': 3160800581}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-03-29T10:57:34.503 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-03-29T10:57:34.503 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-03-29T10:57:34.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.687+0000 7ff2e0dff700 1 Processor -- start 2023-03-29T10:57:34.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.687+0000 7ff2e0dff700 1 -- start start 2023-03-29T10:57:34.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.688+0000 7ff2e0dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff2dc0b87a0 0x7ff2dc0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.688+0000 7ff2e0dff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2dc0cdc60 con 0x7ff2dc1601e0 2023-03-29T10:57:34.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.688+0000 7ff2e0dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2dc0cd6a0 con 0x7ff2dc160d40 2023-03-29T10:57:34.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.688+0000 7ff2e0dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2dc0d02d0 con 0x7ff2dc0b87a0 2023-03-29T10:57:34.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.688+0000 7ff2d9d9b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff2dc1601e0 0x7ff2dc160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:32966/0 (socket says 172.21.15.111:32966) 2023-03-29T10:57:34.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.688+0000 7ff2d9d9b700 1 -- 172.21.15.111:0/1170389860 learned_addr learned my addr 172.21.15.111:0/1170389860 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:34.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.689+0000 7ff2da59c700 1 --2- 172.21.15.111:0/1170389860 >> v2:172.21.15.119:3300/0 conn(0x7ff2dc0b87a0 0x7ff2dc0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.689+0000 7ff2d959a700 1 -- 172.21.15.111:0/1170389860 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2591290654 0 0) 0x7ff2dc0cdc60 con 0x7ff2dc1601e0 2023-03-29T10:57:34.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.689+0000 7ff2d959a700 1 -- 172.21.15.111:0/1170389860 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c40035b0 con 0x7ff2dc1601e0 2023-03-29T10:57:34.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.689+0000 7ff2d959a700 1 -- 172.21.15.111:0/1170389860 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112392518 0 0) 0x7ff2dc0cd6a0 con 0x7ff2dc160d40 2023-03-29T10:57:34.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.689+0000 7ff2d959a700 1 -- 172.21.15.111:0/1170389860 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2dc0cdc60 con 0x7ff2dc160d40 2023-03-29T10:57:34.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.689+0000 7ff2d959a700 1 -- 172.21.15.111:0/1170389860 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3809556569 0 0) 0x7ff2c40035b0 con 0x7ff2dc1601e0 2023-03-29T10:57:34.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.689+0000 7ff2da59c700 1 -- 172.21.15.111:0/1170389860 >> v1:172.21.15.119:6789/0 conn(0x7ff2dc160d40 legacy=0x7ff2dc0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:34.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.689+0000 7ff2da59c700 1 -- 172.21.15.111:0/1170389860 >> v1:172.21.15.111:6789/0 conn(0x7ff2dc1601e0 legacy=0x7ff2dc160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:34.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.689+0000 7ff2da59c700 1 -- 172.21.15.111:0/1170389860 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2dc0cd980 con 0x7ff2dc0b87a0 2023-03-29T10:57:34.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.689+0000 7ff2da59c700 1 --2- 172.21.15.111:0/1170389860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2dc0b87a0 0x7ff2dc0cfbf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009840 tx=0x7ff2cc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fb08b80e06f80d server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:34.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.690+0000 7ff2d959a700 1 -- 172.21.15.111:0/1170389860 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0001670 con 0x7ff2dc0b87a0 2023-03-29T10:57:34.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.690+0000 7ff2d959a700 1 -- 172.21.15.111:0/1170389860 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2cc004030 con 0x7ff2dc0b87a0 2023-03-29T10:57:34.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.690+0000 7ff2d959a700 1 -- 172.21.15.111:0/1170389860 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc0043c0 con 0x7ff2dc0b87a0 2023-03-29T10:57:34.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.690+0000 7ff2e0dff700 1 -- 172.21.15.111:0/1170389860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2dc0b87a0 msgr2=0x7ff2dc0cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:34.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.690+0000 7ff2e0dff700 1 --2- 172.21.15.111:0/1170389860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2dc0b87a0 0x7ff2dc0cfbf0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009840 tx=0x7ff2cc009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:34.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.690+0000 7ff2e0dff700 1 -- 172.21.15.111:0/1170389860 shutdown_connections 2023-03-29T10:57:34.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.690+0000 7ff2e0dff700 1 --2- 172.21.15.111:0/1170389860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2dc0b87a0 0x7ff2dc0cfbf0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.690+0000 7ff2e0dff700 1 -- 172.21.15.111:0/1170389860 >> 172.21.15.111:0/1170389860 conn(0x7ff2dc071b10 msgr2=0x7ff2dc071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:34.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.690+0000 7ff2e0dff700 1 -- 172.21.15.111:0/1170389860 shutdown_connections 2023-03-29T10:57:34.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.690+0000 7ff2e0dff700 1 -- 172.21.15.111:0/1170389860 wait complete. 2023-03-29T10:57:34.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.692+0000 7ff2e0dff700 1 Processor -- start 2023-03-29T10:57:34.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.692+0000 7ff2e0dff700 1 -- start start 2023-03-29T10:57:34.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.693+0000 7ff2e0dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2dc0b87a0 0x7ff2dc14d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.693+0000 7ff2e0dff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2dc1601e0 0x7ff2dc1f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.693+0000 7ff2e0dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff2dc160d40 0x7ff2dc0cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.693+0000 7ff2e0dff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc1f66a0 con 0x7ff2dc1601e0 2023-03-29T10:57:34.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.693+0000 7ff2e0dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc1f60e0 con 0x7ff2dc0b87a0 2023-03-29T10:57:34.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.693+0000 7ff2e0dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2dc1f63c0 con 0x7ff2dc160d40 2023-03-29T10:57:34.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.693+0000 7ff2d9d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2dc1601e0 0x7ff2dc1f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.693+0000 7ff2d9d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2dc1601e0 0x7ff2dc1f5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38912/0 (socket says 172.21.15.111:38912) 2023-03-29T10:57:34.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.693+0000 7ff2d9d9b700 1 -- 172.21.15.111:0/3744308070 learned_addr learned my addr 172.21.15.111:0/3744308070 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:34.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.693+0000 7ff2da59c700 1 --2- 172.21.15.111:0/3744308070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2dc0b87a0 0x7ff2dc14d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.694+0000 7ff2dad9d700 1 --2- 172.21.15.111:0/3744308070 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff2dc160d40 0x7ff2dc0cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.694+0000 7ff2d9d9b700 1 -- 172.21.15.111:0/3744308070 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff2dc160d40 msgr2=0x7ff2dc0cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:34.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.694+0000 7ff2d9d9b700 1 --2- 172.21.15.111:0/3744308070 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff2dc160d40 0x7ff2dc0cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.694+0000 7ff2d9d9b700 1 -- 172.21.15.111:0/3744308070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2dc0b87a0 msgr2=0x7ff2dc14d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:34.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.694+0000 7ff2d9d9b700 1 --2- 172.21.15.111:0/3744308070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2dc0b87a0 0x7ff2dc14d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.694+0000 7ff2d9d9b700 1 -- 172.21.15.111:0/3744308070 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2cc0094a0 con 0x7ff2dc1601e0 2023-03-29T10:57:34.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.694+0000 7ff2d9d9b700 1 --2- 172.21.15.111:0/3744308070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2dc1601e0 0x7ff2dc1f5700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000c750 tx=0x7ff2d000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:34.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.694+0000 7ff2caffd700 1 -- 172.21.15.111:0/3744308070 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0004d40 con 0x7ff2dc1601e0 2023-03-29T10:57:34.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.694+0000 7ff2caffd700 1 -- 172.21.15.111:0/3744308070 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d00070d0 con 0x7ff2dc1601e0 2023-03-29T10:57:34.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.694+0000 7ff2e0dff700 1 -- 172.21.15.111:0/3744308070 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2dc0d19a0 con 0x7ff2dc1601e0 2023-03-29T10:57:34.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.694+0000 7ff2caffd700 1 -- 172.21.15.111:0/3744308070 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0004ea0 con 0x7ff2dc1601e0 2023-03-29T10:57:34.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.694+0000 7ff2e0dff700 1 -- 172.21.15.111:0/3744308070 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2dc0d1810 con 0x7ff2dc1601e0 2023-03-29T10:57:34.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.696+0000 7ff2caffd700 1 -- 172.21.15.111:0/3744308070 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7ff2d0007440 con 0x7ff2dc1601e0 2023-03-29T10:57:34.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.697+0000 7ff2caffd700 1 --2- 172.21.15.111:0/3744308070 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff2bc069de0 0x7ff2bc06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.697+0000 7ff2caffd700 1 -- 172.21.15.111:0/3744308070 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff2d0088150 con 0x7ff2dc1601e0 2023-03-29T10:57:34.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.698+0000 7ff2da59c700 1 --2- 172.21.15.111:0/3744308070 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff2bc069de0 0x7ff2bc06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.698+0000 7ff2da59c700 1 --2- 172.21.15.111:0/3744308070 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff2bc069de0 0x7ff2bc06c290 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0090b0 tx=0x7ff2cc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:34.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.699+0000 7ff2f4789b80 1 Processor -- start 2023-03-29T10:57:34.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.699+0000 7ff2f4789b80 1 -- start start 2023-03-29T10:57:34.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.700+0000 7ff2f4789b80 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55da94322060 0x55da94322430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.700+0000 7ff2f4789b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94361c50 0x55da943481a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.700+0000 7ff2f4789b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da943236e0 0x55da943486e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.700+0000 7ff2f4789b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da942a3960 con 0x55da943236e0 2023-03-29T10:57:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.700+0000 7ff2f4789b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da9422d120 con 0x55da94361c50 2023-03-29T10:57:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.700+0000 7ff2f4789b80 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da9427f690 con 0x55da94322060 2023-03-29T10:57:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.700+0000 7ff2dad9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da943236e0 0x55da943486e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.700+0000 7ff2dad9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da943236e0 0x55da943486e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38926/0 (socket says 172.21.15.111:38926) 2023-03-29T10:57:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.700+0000 7ff2dad9d700 1 -- 172.21.15.111:0/3103925259 learned_addr learned my addr 172.21.15.111:0/3103925259 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.701+0000 7ff2dad9d700 1 -- 172.21.15.111:0/3103925259 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55da94322060 msgr2=0x55da94322430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.701+0000 7ff2dad9d700 1 --2- 172.21.15.111:0/3103925259 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55da94322060 0x55da94322430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.701+0000 7ff2d9d9b700 1 --2- 172.21.15.111:0/3103925259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94361c50 0x55da943481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.701+0000 7ff2dad9d700 1 -- 172.21.15.111:0/3103925259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94361c50 msgr2=0x55da943481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.701+0000 7ff2dad9d700 1 --2- 172.21.15.111:0/3103925259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94361c50 0x55da943481a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.701+0000 7ff2dad9d700 1 -- 172.21.15.111:0/3103925259 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da9406c540 con 0x55da943236e0 2023-03-29T10:57:34.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.701+0000 7ff2dad9d700 1 --2- 172.21.15.111:0/3103925259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da943236e0 0x55da943486e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4002210 tx=0x7ff2d4001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29171c06b42d571 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:34.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.701+0000 7ff2bbfff700 1 -- 172.21.15.111:0/3103925259 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d400fd80 con 0x55da943236e0 2023-03-29T10:57:34.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.701+0000 7ff2bbfff700 1 -- 172.21.15.111:0/3103925259 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d4003a00 con 0x55da943236e0 2023-03-29T10:57:34.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.701+0000 7ff2bbfff700 1 -- 172.21.15.111:0/3103925259 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d400fee0 con 0x55da943236e0 2023-03-29T10:57:34.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.702+0000 7ff2f4789b80 1 -- 172.21.15.111:0/3103925259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da943236e0 msgr2=0x55da943486e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:34.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.702+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/3103925259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da943236e0 0x55da943486e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4002210 tx=0x7ff2d4001a70 comp rx=0 tx=0).stop 2023-03-29T10:57:34.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.702+0000 7ff2f4789b80 1 -- 172.21.15.111:0/3103925259 shutdown_connections 2023-03-29T10:57:34.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.702+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/3103925259 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da943236e0 0x55da943486e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.702+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/3103925259 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55da94322060 0x55da94322430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.702+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/3103925259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94361c50 0x55da943481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.702+0000 7ff2f4789b80 1 -- 172.21.15.111:0/3103925259 >> 172.21.15.111:0/3103925259 conn(0x55da9431aac0 msgr2=0x55da94320030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:34.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.702+0000 7ff2f4789b80 1 -- 172.21.15.111:0/3103925259 shutdown_connections 2023-03-29T10:57:34.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.702+0000 7ff2f4789b80 1 -- 172.21.15.111:0/3103925259 wait complete. 2023-03-29T10:57:34.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.703+0000 7ff2f4789b80 1 -- start start 2023-03-29T10:57:34.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.703+0000 7ff2f4789b80 1 Processor -- start 2023-03-29T10:57:34.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.704+0000 7ff2f4789b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94322060 0x55da94353a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.704+0000 7ff2f4789b80 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55da943236e0 0x55da943510f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.704+0000 7ff2f4789b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 0x55da9433e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.704+0000 7ff2f4789b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da94278960 con 0x55da94361c50 2023-03-29T10:57:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.704+0000 7ff2f4789b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da9406d140 con 0x55da94322060 2023-03-29T10:57:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.704+0000 7ff2f4789b80 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da9434ec00 con 0x55da943236e0 2023-03-29T10:57:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.704+0000 7ff2f4789b80 10 client.0 _ll_register_callbacks cb 0x55da942161c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-03-29T10:57:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.704+0000 7ff2dad9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 0x55da9433e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.704+0000 7ff2da59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94322060 0x55da94353a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.704+0000 7ff2dad9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 0x55da9433e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38930/0 (socket says 172.21.15.111:38930) 2023-03-29T10:57:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.704+0000 7ff2dad9d700 1 -- 172.21.15.111:0/1900064143 learned_addr learned my addr 172.21.15.111:0/1900064143 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.704+0000 7ff2d9d9b700 1 --2- 172.21.15.111:0/1900064143 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55da943236e0 0x55da943510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.704+0000 7ff2dad9d700 1 -- 172.21.15.111:0/1900064143 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55da943236e0 msgr2=0x55da943510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.704+0000 7ff2dad9d700 1 --2- 172.21.15.111:0/1900064143 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55da943236e0 0x55da943510f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.704+0000 7ff2dad9d700 1 -- 172.21.15.111:0/1900064143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94322060 msgr2=0x55da94353a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.704+0000 7ff2dad9d700 1 --2- 172.21.15.111:0/1900064143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94322060 0x55da94353a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.704+0000 7ff2dad9d700 1 -- 172.21.15.111:0/1900064143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2dc0cdc60 con 0x55da94361c50 2023-03-29T10:57:34.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.705+0000 7ff2dad9d700 1 --2- 172.21.15.111:0/1900064143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 0x55da9433e3a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009010 tx=0x7ff2d4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ff61c736df2696 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.705+0000 7ff2b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-03-29T10:57:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.705+0000 7ff2b97fa700 1 -- 172.21.15.111:0/1900064143 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40017b0 con 0x55da94361c50 2023-03-29T10:57:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.705+0000 7ff2b97fa700 1 -- 172.21.15.111:0/1900064143 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff2d40055c0 con 0x55da94361c50 2023-03-29T10:57:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.705+0000 7ff2f4789b80 1 -- 172.21.15.111:0/1900064143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da943180d0 con 0x55da94361c50 2023-03-29T10:57:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.705+0000 7ff2b97fa700 1 -- 172.21.15.111:0/1900064143 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2d40012f0 con 0x55da94361c50 2023-03-29T10:57:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.705+0000 7ff2f4789b80 10 client.4452 fetch_fsmap learned FSMap version 7 2023-03-29T10:57:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.705+0000 7ff2f4789b80 1 -- 172.21.15.111:0/1900064143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55da940a6b10 con 0x55da94361c50 2023-03-29T10:57:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.705+0000 7ff2b97fa700 1 -- 172.21.15.111:0/1900064143 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55da943180d0 con 0x55da94361c50 2023-03-29T10:57:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.705+0000 7ff2b97fa700 1 -- 172.21.15.111:0/1900064143 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7ff2d4012070 con 0x55da94361c50 2023-03-29T10:57:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.705+0000 7ff2f4789b80 10 client.4452 fetch_fsmap finished waiting for FSMap version 7 2023-03-29T10:57:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.705+0000 7ff2f4789b80 10 client.4452 resolve_mds: resolved name 'd' to daemon mds.d 2023-03-29T10:57:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.705+0000 7ff2f4789b80 20 client.4452 populate_metadata read hostname 'smithi111' 2023-03-29T10:57:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.705+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/1900064143 >> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] conn(0x55da9437e090 0x55da9437e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.706+0000 7ff2f4789b80 4 client.4452 mds_command: new command op to 4296 tid=0[{"prefix": "get_command_descriptions"}] 2023-03-29T10:57:34.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.706+0000 7ff2f4789b80 1 -- 172.21.15.111:0/1900064143 --> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da942f54c0 con 0x55da9437e090 2023-03-29T10:57:34.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.706+0000 7ff2da59c700 1 --2- 172.21.15.111:0/1900064143 >> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] conn(0x55da9437e090 0x55da9437e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.706+0000 7ff2da59c700 1 --2- 172.21.15.111:0/1900064143 >> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] conn(0x55da9437e090 0x55da9437e460 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=360768bb68bcf36b server_cookie=247e046688ae61dc in_seq=0 out_seq=0 2023-03-29T10:57:34.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.706+0000 7ff2b97fa700 10 client.4452 ms_handle_connect on v2:172.21.15.119:6833/3160800581 2023-03-29T10:57:34.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.707+0000 7ff2b97fa700 1 -- 172.21.15.111:0/1900064143 <== mds.0 v2:172.21.15.119:6833/3160800581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18278 (crc 0 0 0) 0x55da942f54c0 con 0x55da9437e090 2023-03-29T10:57:34.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.707+0000 7ff2b97fa700 10 client.4452 handle_command_reply: tid=0 2023-03-29T10:57:34.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.708+0000 7ff2f4789b80 1 client.4452 shutdown 2023-03-29T10:57:34.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.708+0000 7ff2f4789b80 1 -- 172.21.15.111:0/1900064143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 msgr2=0x55da9433e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:34.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.708+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/1900064143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 0x55da9433e3a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009010 tx=0x7ff2d4001250 comp rx=0 tx=0).stop 2023-03-29T10:57:34.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.708+0000 7ff2f4789b80 1 -- 172.21.15.111:0/1900064143 shutdown_connections 2023-03-29T10:57:34.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.708+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/1900064143 >> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] conn(0x55da9437e090 0x55da9437e460 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.708+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/1900064143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 0x55da9433e3a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.708+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/1900064143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94322060 0x55da94353a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.708+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/1900064143 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55da943236e0 0x55da943510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.708+0000 7ff2b97fa700 0 client.4452 ms_handle_reset on v2:172.21.15.119:6833/3160800581 2023-03-29T10:57:34.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.708+0000 7ff2f4789b80 1 -- 172.21.15.111:0/1900064143 >> 172.21.15.111:0/1900064143 conn(0x55da9431aac0 msgr2=0x55da9431f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:34.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.708+0000 7ff2f4789b80 1 -- 172.21.15.111:0/1900064143 shutdown_connections 2023-03-29T10:57:34.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.708+0000 7ff2f4789b80 1 -- 172.21.15.111:0/1900064143 wait complete. 2023-03-29T10:57:34.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.708+0000 7ff2f4789b80 20 client.4452 trim_cache size 0 max 16384 2023-03-29T10:57:34.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.726+0000 7ff2f4789b80 1 Processor -- start 2023-03-29T10:57:34.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.726+0000 7ff2f4789b80 1 -- start start 2023-03-29T10:57:34.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.727+0000 7ff2f4789b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94322060 0x55da9431c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.727+0000 7ff2f4789b80 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55da943236e0 0x55da9433e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.727+0000 7ff2f4789b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 0x55da94379670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.727+0000 7ff2f4789b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da9437d8a0 con 0x55da94361c50 2023-03-29T10:57:34.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.727+0000 7ff2f4789b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da94322680 con 0x55da94322060 2023-03-29T10:57:34.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.727+0000 7ff2f4789b80 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da94325f60 con 0x55da943236e0 2023-03-29T10:57:34.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.727+0000 7ff2dad9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 0x55da94379670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.727+0000 7ff2dad9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 0x55da94379670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38932/0 (socket says 172.21.15.111:38932) 2023-03-29T10:57:34.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.727+0000 7ff2dad9d700 1 -- 172.21.15.111:0/3641979203 learned_addr learned my addr 172.21.15.111:0/3641979203 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:34.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.727+0000 7ff2dad9d700 1 -- 172.21.15.111:0/3641979203 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55da943236e0 msgr2=0x55da9433e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:34.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.727+0000 7ff2dad9d700 1 --2- 172.21.15.111:0/3641979203 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55da943236e0 0x55da9433e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.727+0000 7ff2da59c700 1 --2- 172.21.15.111:0/3641979203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94322060 0x55da9431c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.727+0000 7ff2dad9d700 1 -- 172.21.15.111:0/3641979203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94322060 msgr2=0x55da9431c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:34.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.727+0000 7ff2dad9d700 1 --2- 172.21.15.111:0/3641979203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94322060 0x55da9431c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.727+0000 7ff2dad9d700 1 -- 172.21.15.111:0/3641979203 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da9423f230 con 0x55da94361c50 2023-03-29T10:57:34.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.727+0000 7ff2da59c700 1 --2- 172.21.15.111:0/3641979203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94322060 0x55da9431c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:34.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.728+0000 7ff2dad9d700 1 --2- 172.21.15.111:0/3641979203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 0x55da94379670 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4011010 tx=0x7ff2d401f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8507e88d804bf567 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:34.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.728+0000 7ff2c8ff9700 1 -- 172.21.15.111:0/3641979203 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40054e0 con 0x55da94361c50 2023-03-29T10:57:34.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.728+0000 7ff2c8ff9700 1 -- 172.21.15.111:0/3641979203 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d4021070 con 0x55da94361c50 2023-03-29T10:57:34.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.728+0000 7ff2c8ff9700 1 -- 172.21.15.111:0/3641979203 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4026710 con 0x55da94361c50 2023-03-29T10:57:34.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.728+0000 7ff2f4789b80 1 -- 172.21.15.111:0/3641979203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 msgr2=0x55da94379670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:34.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.728+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/3641979203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 0x55da94379670 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4011010 tx=0x7ff2d401f070 comp rx=0 tx=0).stop 2023-03-29T10:57:34.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.728+0000 7ff2f4789b80 1 -- 172.21.15.111:0/3641979203 shutdown_connections 2023-03-29T10:57:34.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.728+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/3641979203 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 0x55da94379670 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.728+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/3641979203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94322060 0x55da9431c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.728+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/3641979203 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55da943236e0 0x55da9433e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.728+0000 7ff2f4789b80 1 -- 172.21.15.111:0/3641979203 >> 172.21.15.111:0/3641979203 conn(0x55da9431aac0 msgr2=0x55da94324fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:34.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.728+0000 7ff2f4789b80 1 -- 172.21.15.111:0/3641979203 shutdown_connections 2023-03-29T10:57:34.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.729+0000 7ff2f4789b80 1 -- 172.21.15.111:0/3641979203 wait complete. 2023-03-29T10:57:34.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.729+0000 7ff2f4789b80 1 -- start start 2023-03-29T10:57:34.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.729+0000 7ff2f4789b80 1 Processor -- start 2023-03-29T10:57:34.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.730+0000 7ff2f4789b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94322060 0x55da94350f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.730+0000 7ff2f4789b80 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55da943236e0 0x55da943bb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.730+0000 7ff2f4789b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 0x55da94348f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.730+0000 7ff2f4789b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da943514b0 con 0x55da94361c50 2023-03-29T10:57:34.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.730+0000 7ff2f4789b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da9434af90 con 0x55da94322060 2023-03-29T10:57:34.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.730+0000 7ff2f4789b80 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da9437c800 con 0x55da943236e0 2023-03-29T10:57:34.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.730+0000 7ff2f4789b80 10 client.0 _ll_register_callbacks cb 0x7ff2d4002080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-03-29T10:57:34.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.730+0000 7ff2dad9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 0x55da94348f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.731+0000 7ff2dad9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 0x55da94348f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38934/0 (socket says 172.21.15.111:38934) 2023-03-29T10:57:34.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.731+0000 7ff2dad9d700 1 -- 172.21.15.111:0/2288321525 learned_addr learned my addr 172.21.15.111:0/2288321525 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:34.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.731+0000 7ff2da59c700 1 --2- 172.21.15.111:0/2288321525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94322060 0x55da94350f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.731+0000 7ff2d9d9b700 1 --2- 172.21.15.111:0/2288321525 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55da943236e0 0x55da943bb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.731+0000 7ff2dad9d700 1 -- 172.21.15.111:0/2288321525 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55da943236e0 msgr2=0x55da943bb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:34.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.731+0000 7ff2dad9d700 1 --2- 172.21.15.111:0/2288321525 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55da943236e0 0x55da943bb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.731+0000 7ff2dad9d700 1 -- 172.21.15.111:0/2288321525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94322060 msgr2=0x55da94350f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:34.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.731+0000 7ff2dad9d700 1 --2- 172.21.15.111:0/2288321525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94322060 0x55da94350f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.731+0000 7ff2dad9d700 1 -- 172.21.15.111:0/2288321525 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d400f040 con 0x55da94361c50 2023-03-29T10:57:34.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.731+0000 7ff2da59c700 1 --2- 172.21.15.111:0/2288321525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94322060 0x55da94350f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:57:34.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.731+0000 7ff2d9d9b700 1 --2- 172.21.15.111:0/2288321525 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55da943236e0 0x55da943bb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:57:34.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.731+0000 7ff2dad9d700 1 --2- 172.21.15.111:0/2288321525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 0x55da94348f90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009010 tx=0x7ff2d4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c005e756878468 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:34.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.731+0000 7ff2b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-03-29T10:57:34.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.731+0000 7ff2b97fa700 1 -- 172.21.15.111:0/2288321525 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d40012f0 con 0x55da94361c50 2023-03-29T10:57:34.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.731+0000 7ff2b97fa700 1 -- 172.21.15.111:0/2288321525 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff2d4005070 con 0x55da94361c50 2023-03-29T10:57:34.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.731+0000 7ff2f4789b80 1 -- 172.21.15.111:0/2288321525 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2d4003a00 con 0x55da94361c50 2023-03-29T10:57:34.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.732+0000 7ff2b97fa700 1 -- 172.21.15.111:0/2288321525 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2d4005270 con 0x55da94361c50 2023-03-29T10:57:34.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.732+0000 7ff2f4789b80 10 client.4458 fetch_fsmap learned FSMap version 7 2023-03-29T10:57:34.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.732+0000 7ff2f4789b80 1 -- 172.21.15.111:0/2288321525 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55da9408f990 con 0x55da94361c50 2023-03-29T10:57:34.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.732+0000 7ff2b97fa700 1 -- 172.21.15.111:0/2288321525 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d4003a00 con 0x55da94361c50 2023-03-29T10:57:34.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.732+0000 7ff2b97fa700 1 -- 172.21.15.111:0/2288321525 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 7) v1 ==== 3122+0+0 (secure 0 0 0) 0x7ff2d400f1d0 con 0x55da94361c50 2023-03-29T10:57:34.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.732+0000 7ff2f4789b80 10 client.4458 fetch_fsmap finished waiting for FSMap version 7 2023-03-29T10:57:34.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.732+0000 7ff2f4789b80 10 client.4458 resolve_mds: resolved name 'd' to daemon mds.d 2023-03-29T10:57:34.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.732+0000 7ff2f4789b80 20 client.4458 populate_metadata read hostname 'smithi111' 2023-03-29T10:57:34.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.732+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/2288321525 >> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] conn(0x55da943494d0 0x55da9434a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.732+0000 7ff2f4789b80 4 client.4458 mds_command: new command op to 4296 tid=0[{"prefix": "status"}] 2023-03-29T10:57:34.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.732+0000 7ff2f4789b80 1 -- 172.21.15.111:0/2288321525 --> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55da94271aa0 con 0x55da943494d0 2023-03-29T10:57:34.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.732+0000 7ff2da59c700 1 --2- 172.21.15.111:0/2288321525 >> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] conn(0x55da943494d0 0x55da9434a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.733+0000 7ff2da59c700 1 --2- 172.21.15.111:0/2288321525 >> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] conn(0x55da943494d0 0x55da9434a1b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe3903dba9788880 server_cookie=6b56a6775e3cfb64 in_seq=0 out_seq=0 2023-03-29T10:57:34.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.733+0000 7ff2b97fa700 10 client.4458 ms_handle_connect on v2:172.21.15.119:6833/3160800581 2023-03-29T10:57:34.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.733+0000 7ff2b97fa700 1 -- 172.21.15.111:0/2288321525 <== mds.0 v2:172.21.15.119:6833/3160800581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55da94271aa0 con 0x55da943494d0 2023-03-29T10:57:34.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.733+0000 7ff2b97fa700 10 client.4458 handle_command_reply: tid=0 2023-03-29T10:57:34.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.733+0000 7ff2f4789b80 1 client.4458 shutdown 2023-03-29T10:57:34.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.733+0000 7ff2f4789b80 1 -- 172.21.15.111:0/2288321525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 msgr2=0x55da94348f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:34.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.733+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/2288321525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 0x55da94348f90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009010 tx=0x7ff2d4009040 comp rx=0 tx=0).stop 2023-03-29T10:57:34.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.733+0000 7ff2f4789b80 1 -- 172.21.15.111:0/2288321525 shutdown_connections 2023-03-29T10:57:34.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.733+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/2288321525 >> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] conn(0x55da943494d0 0x55da9434a1b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.733+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/2288321525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55da94361c50 0x55da94348f90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.733+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/2288321525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55da94322060 0x55da94350f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.733+0000 7ff2b97fa700 0 client.4458 ms_handle_reset on v2:172.21.15.119:6833/3160800581 2023-03-29T10:57:34.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.733+0000 7ff2f4789b80 1 --2- 172.21.15.111:0/2288321525 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55da943236e0 0x55da943bb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.733+0000 7ff2f4789b80 1 -- 172.21.15.111:0/2288321525 >> 172.21.15.111:0/2288321525 conn(0x55da9431aac0 msgr2=0x55da94379670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:34.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.733+0000 7ff2f4789b80 1 -- 172.21.15.111:0/2288321525 shutdown_connections 2023-03-29T10:57:34.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.733+0000 7ff2f4789b80 1 -- 172.21.15.111:0/2288321525 wait complete. 2023-03-29T10:57:34.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.734+0000 7ff2f4789b80 20 client.4458 trim_cache size 0 max 16384 2023-03-29T10:57:34.740 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-03-29T10:57:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "c8526f71-9b58-4b2e-978f-708199ec8aef", 2023-03-29T10:57:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 0, 2023-03-29T10:57:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 4296, 2023-03-29T10:57:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2023-03-29T10:57:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2023-03-29T10:57:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2023-03-29T10:57:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 6.9069171709999999, 2023-03-29T10:57:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 7, 2023-03-29T10:57:34.740 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 28, 2023-03-29T10:57:34.741 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 27, 2023-03-29T10:57:34.741 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 12.198013162000001 2023-03-29T10:57:34.741 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-03-29T10:57:34.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.734+0000 7ff2e0dff700 1 -- 172.21.15.111:0/3744308070 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff2bc069de0 msgr2=0x7ff2bc06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:34.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.734+0000 7ff2e0dff700 1 --2- 172.21.15.111:0/3744308070 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff2bc069de0 0x7ff2bc06c290 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0090b0 tx=0x7ff2cc008030 comp rx=0 tx=0).stop 2023-03-29T10:57:34.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.734+0000 7ff2e0dff700 1 -- 172.21.15.111:0/3744308070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2dc1601e0 msgr2=0x7ff2dc1f5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:34.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.734+0000 7ff2e0dff700 1 --2- 172.21.15.111:0/3744308070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2dc1601e0 0x7ff2dc1f5700 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff2d000c750 tx=0x7ff2d000ca60 comp rx=0 tx=0).stop 2023-03-29T10:57:34.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.734+0000 7ff2e0dff700 1 -- 172.21.15.111:0/3744308070 shutdown_connections 2023-03-29T10:57:34.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.734+0000 7ff2e0dff700 1 --2- 172.21.15.111:0/3744308070 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff2bc069de0 0x7ff2bc06c290 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.734+0000 7ff2e0dff700 1 --2- 172.21.15.111:0/3744308070 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff2dc160d40 0x7ff2dc0cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.734+0000 7ff2e0dff700 1 --2- 172.21.15.111:0/3744308070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2dc0b87a0 0x7ff2dc14d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.734+0000 7ff2e0dff700 1 --2- 172.21.15.111:0/3744308070 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff2dc1601e0 0x7ff2dc1f5700 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.734+0000 7ff2e0dff700 1 -- 172.21.15.111:0/3744308070 >> 172.21.15.111:0/3744308070 conn(0x7ff2dc071b10 msgr2=0x7ff2dc0cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:34.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.734+0000 7ff2e0dff700 1 -- 172.21.15.111:0/3744308070 shutdown_connections 2023-03-29T10:57:34.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.734+0000 7ff2e0dff700 1 -- 172.21.15.111:0/3744308070 wait complete. 2023-03-29T10:57:34.750 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-03-29T10:57:34.750 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-03-29T10:57:34.750 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-03-29T10:57:34.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.934+0000 7fa3863ff700 1 Processor -- start 2023-03-29T10:57:34.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.935+0000 7fa3863ff700 1 -- start start 2023-03-29T10:57:34.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.936+0000 7fa3863ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3800cf970 0x7fa3800cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.936+0000 7fa3863ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa3800b70f0 0x7fa3800b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.936+0000 7fa3863ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3800cdc10 con 0x7fa3800cfd40 2023-03-29T10:57:34.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.936+0000 7fa3863ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3800d08c0 con 0x7fa3800b70f0 2023-03-29T10:57:34.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.936+0000 7fa3863ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3800b6c60 con 0x7fa3800cf970 2023-03-29T10:57:34.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.936+0000 7fa38499c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa3800b70f0 0x7fa3800b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.936+0000 7fa38499c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa3800b70f0 0x7fa3800b7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38936/0 (socket says 172.21.15.111:38936) 2023-03-29T10:57:34.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.936+0000 7fa38499c700 1 -- 172.21.15.111:0/3797654409 learned_addr learned my addr 172.21.15.111:0/3797654409 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:34.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.936+0000 7fa37ffff700 1 --2- 172.21.15.111:0/3797654409 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3800cf970 0x7fa3800cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.936+0000 7fa38499c700 1 -- 172.21.15.111:0/3797654409 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3800cf970 msgr2=0x7fa3800cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:34.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.936+0000 7fa38499c700 1 --2- 172.21.15.111:0/3797654409 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3800cf970 0x7fa3800cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.936+0000 7fa38499c700 1 -- 172.21.15.111:0/3797654409 >> v1:172.21.15.119:6789/0 conn(0x7fa3800cfd40 legacy=0x7fa3800d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:34.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.936+0000 7fa38499c700 1 -- 172.21.15.111:0/3797654409 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3800cd650 con 0x7fa3800b70f0 2023-03-29T10:57:34.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.936+0000 7fa37ffff700 1 --2- 172.21.15.111:0/3797654409 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3800cf970 0x7fa3800cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:34.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.937+0000 7fa38499c700 1 --2- 172.21.15.111:0/3797654409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3800b70f0 0x7fa3800b7530 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa378009e50 tx=0x7fa37800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a90cc616672ae3d server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:34.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.937+0000 7fa37f7fe700 1 -- 172.21.15.111:0/3797654409 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37800e070 con 0x7fa3800b70f0 2023-03-29T10:57:34.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.937+0000 7fa37f7fe700 1 -- 172.21.15.111:0/3797654409 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37800b9c0 con 0x7fa3800b70f0 2023-03-29T10:57:34.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.937+0000 7fa37f7fe700 1 -- 172.21.15.111:0/3797654409 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37800bd70 con 0x7fa3800b70f0 2023-03-29T10:57:34.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.937+0000 7fa3863ff700 1 -- 172.21.15.111:0/3797654409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3800b70f0 msgr2=0x7fa3800b7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:34.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.937+0000 7fa3863ff700 1 --2- 172.21.15.111:0/3797654409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3800b70f0 0x7fa3800b7530 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa378009e50 tx=0x7fa37800b330 comp rx=0 tx=0).stop 2023-03-29T10:57:34.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.937+0000 7fa3863ff700 1 -- 172.21.15.111:0/3797654409 shutdown_connections 2023-03-29T10:57:34.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.937+0000 7fa3863ff700 1 --2- 172.21.15.111:0/3797654409 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3800b70f0 0x7fa3800b7530 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.937+0000 7fa3863ff700 1 --2- 172.21.15.111:0/3797654409 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3800cf970 0x7fa3800cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.937+0000 7fa3863ff700 1 -- 172.21.15.111:0/3797654409 >> 172.21.15.111:0/3797654409 conn(0x7fa380071b10 msgr2=0x7fa380071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:34.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.938+0000 7fa3863ff700 1 -- 172.21.15.111:0/3797654409 shutdown_connections 2023-03-29T10:57:34.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.938+0000 7fa3863ff700 1 -- 172.21.15.111:0/3797654409 wait complete. 2023-03-29T10:57:34.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.939+0000 7fa3863ff700 1 Processor -- start 2023-03-29T10:57:34.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.939+0000 7fa3863ff700 1 -- start start 2023-03-29T10:57:34.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.940+0000 7fa3863ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3800b70f0 0x7fa3801543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.940+0000 7fa3863ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3800cf970 0x7fa3800cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.940+0000 7fa3863ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3800cfd40 0x7fa38015af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.940+0000 7fa3863ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3801548e0 con 0x7fa3800cfd40 2023-03-29T10:57:34.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.940+0000 7fa3863ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3800cce20 con 0x7fa3800cf970 2023-03-29T10:57:34.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.940+0000 7fa3863ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3800c7d30 con 0x7fa3800b70f0 2023-03-29T10:57:34.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.940+0000 7fa38499c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3800cfd40 0x7fa38015af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.940+0000 7fa38499c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3800cfd40 0x7fa38015af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38952/0 (socket says 172.21.15.111:38952) 2023-03-29T10:57:34.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.940+0000 7fa38499c700 1 -- 172.21.15.111:0/448592838 learned_addr learned my addr 172.21.15.111:0/448592838 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:34.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.941+0000 7fa37ffff700 1 --2- 172.21.15.111:0/448592838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3800b70f0 0x7fa3801543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.941+0000 7fa38499c700 1 -- 172.21.15.111:0/448592838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3800b70f0 msgr2=0x7fa3801543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:34.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.941+0000 7fa377fff700 1 --2- 172.21.15.111:0/448592838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3800cf970 0x7fa3800cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.941+0000 7fa38499c700 1 --2- 172.21.15.111:0/448592838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3800b70f0 0x7fa3801543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.941+0000 7fa38499c700 1 -- 172.21.15.111:0/448592838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3800cf970 msgr2=0x7fa3800cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:34.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.941+0000 7fa38499c700 1 --2- 172.21.15.111:0/448592838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3800cf970 0x7fa3800cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:34.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.941+0000 7fa38499c700 1 -- 172.21.15.111:0/448592838 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa378009b30 con 0x7fa3800cfd40 2023-03-29T10:57:34.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.941+0000 7fa37ffff700 1 --2- 172.21.15.111:0/448592838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3800b70f0 0x7fa3801543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:34.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.941+0000 7fa38499c700 1 --2- 172.21.15.111:0/448592838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3800cfd40 0x7fa38015af30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa380072670 tx=0x7fa378004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:34.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.941+0000 7fa37d7fa700 1 -- 172.21.15.111:0/448592838 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3780092e0 con 0x7fa3800cfd40 2023-03-29T10:57:34.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.941+0000 7fa37d7fa700 1 -- 172.21.15.111:0/448592838 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa378004960 con 0x7fa3800cfd40 2023-03-29T10:57:34.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.941+0000 7fa3863ff700 1 -- 172.21.15.111:0/448592838 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3800b68f0 con 0x7fa3800cfd40 2023-03-29T10:57:34.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.941+0000 7fa37d7fa700 1 -- 172.21.15.111:0/448592838 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa37801b670 con 0x7fa3800cfd40 2023-03-29T10:57:34.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.941+0000 7fa3863ff700 1 -- 172.21.15.111:0/448592838 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3800cd930 con 0x7fa3800cfd40 2023-03-29T10:57:34.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.942+0000 7fa3863ff700 1 -- 172.21.15.111:0/448592838 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3800cf650 con 0x7fa3800cfd40 2023-03-29T10:57:34.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.943+0000 7fa37d7fa700 1 -- 172.21.15.111:0/448592838 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fa37801b7d0 con 0x7fa3800cfd40 2023-03-29T10:57:34.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.944+0000 7fa37d7fa700 1 --2- 172.21.15.111:0/448592838 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa360069de0 0x7fa36006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:34.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.944+0000 7fa37d7fa700 1 -- 172.21.15.111:0/448592838 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa378089c60 con 0x7fa3800cfd40 2023-03-29T10:57:34.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.944+0000 7fa37ffff700 1 --2- 172.21.15.111:0/448592838 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa360069de0 0x7fa36006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:34.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.945+0000 7fa37ffff700 1 --2- 172.21.15.111:0/448592838 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa360069de0 0x7fa36006c290 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa370008910 tx=0x7fa370008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:34.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:34.948+0000 7fa37d7fa700 1 -- 172.21.15.111:0/448592838 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fa3780552b0 con 0x7fa3800cfd40 2023-03-29T10:57:35.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:35.184+0000 7fa3863ff700 1 -- 172.21.15.111:0/448592838 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa380072720 con 0x7fa3800cfd40 2023-03-29T10:57:35.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:35.185+0000 7fa37d7fa700 1 -- 172.21.15.111:0/448592838 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7fa3780588d0 con 0x7fa3800cfd40 2023-03-29T10:57:35.185 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-03-29T10:57:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:35.187+0000 7fa3863ff700 1 -- 172.21.15.111:0/448592838 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa360069de0 msgr2=0x7fa36006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:35.187+0000 7fa3863ff700 1 --2- 172.21.15.111:0/448592838 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa360069de0 0x7fa36006c290 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa370008910 tx=0x7fa370008040 comp rx=0 tx=0).stop 2023-03-29T10:57:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:35.187+0000 7fa3863ff700 1 -- 172.21.15.111:0/448592838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3800cfd40 msgr2=0x7fa38015af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:35.187+0000 7fa3863ff700 1 --2- 172.21.15.111:0/448592838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3800cfd40 0x7fa38015af30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa380072670 tx=0x7fa378004ad0 comp rx=0 tx=0).stop 2023-03-29T10:57:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:35.187+0000 7fa3863ff700 1 -- 172.21.15.111:0/448592838 shutdown_connections 2023-03-29T10:57:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:35.187+0000 7fa3863ff700 1 --2- 172.21.15.111:0/448592838 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa360069de0 0x7fa36006c290 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:35.187+0000 7fa3863ff700 1 --2- 172.21.15.111:0/448592838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa3800cfd40 0x7fa38015af30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:35.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:35.187+0000 7fa3863ff700 1 --2- 172.21.15.111:0/448592838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa3800b70f0 0x7fa3801543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:35.187+0000 7fa3863ff700 1 --2- 172.21.15.111:0/448592838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3800cf970 0x7fa3800cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:35.187+0000 7fa3863ff700 1 -- 172.21.15.111:0/448592838 >> 172.21.15.111:0/448592838 conn(0x7fa380071b10 msgr2=0x7fa3800d01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:35.187+0000 7fa3863ff700 1 -- 172.21.15.111:0/448592838 shutdown_connections 2023-03-29T10:57:35.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:35.187+0000 7fa3863ff700 1 -- 172.21.15.111:0/448592838 wait complete. 2023-03-29T10:57:35.204 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-03-29T10:57:35.204 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-03-29T10:57:35.204 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-03-29T10:57:35.204 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-03-29T10:57:35.204 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-03-29T10:57:35.204 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-03-29T10:57:35.204 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-03-29T10:57:35.204 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-03-29T10:57:35.204 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-03-29T10:57:35.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:57:35.205 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-03-29T10:57:35.245 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-29T10:57:35.245 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-29T10:57:35.245 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-03-29T10:57:35.245 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:57:35.245 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-03-29T10:57:35.245 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:57:35.245 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-29T10:57:35.246 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-03-29T10:57:35.246 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-03-29T10:57:35.246 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-29T10:57:35.246 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-03-29T10:57:35.246 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-03-29T10:57:35.246 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-29T10:57:35.246 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-03-29T10:57:35.246 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-03-29T10:57:35.246 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-03-29T10:57:35.246 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-03-29T10:57:35.246 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42362sec preferred_lft 42362sec 2023-03-29T10:57:35.246 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-03-29T10:57:35.246 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:57:35.247 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-03-29T10:57:35.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:57:35.247 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-03-29T10:57:35.248 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-03-29T10:57:35.248 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add name ceph-brx type bridge 2023-03-29T10:57:35.248 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr flush dev ceph-brx 2023-03-29T10:57:35.248 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set ceph-brx up 2023-03-29T10:57:35.248 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-03-29T10:57:35.248 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-03-29T10:57:35.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:57:35.707 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-03-29T10:57:35.808 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-03-29T10:57:35.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:57:35.811 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-03-29T10:57:35.861 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-03-29T10:57:35.861 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-03-29T10:57:35.861 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-03-29T10:57:35.861 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-03-29T10:57:35.861 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-03-29T10:57:35.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:57:35.862 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-03-29T10:57:35.862 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-03-29T10:57:35.862 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-03-29T10:57:35.862 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-03-29T10:57:35.862 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-03-29T10:57:35.862 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-03-29T10:57:36.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:57:36.521 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-03-29T10:57:36.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:57:36.563 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list-id 2023-03-29T10:57:36.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:57:36.645 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-03-29T10:57:36.645 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-03-29T10:57:36.645 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-03-29T10:57:36.645 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-03-29T10:57:36.645 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-03-29T10:57:36.812 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-03-29T10:57:36.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:57:36.812 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-03-29T10:57:36.812 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-03-29T10:57:36.812 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-03-29T10:57:36.812 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-03-29T10:57:36.812 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-03-29T10:57:36.812 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-03-29T10:57:36.812 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-03-29T10:57:36.812 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-03-29T10:57:37.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:57:37.061 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-03-29T10:57:37.061 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-03-29T10:57:37.061 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-03-29T10:57:37.061 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set dev brx.0 master ceph-brx 2023-03-29T10:57:37.061 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-03-29T10:57:37.225 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-03-29T10:57:37.226 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-29T10:57:37.226 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-03-29T10:57:37.267 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-03-29T10:57:37.267 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-29T10:57:37.267 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-03-29T10:57:37.347 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2023-03-29T10:57:37.450 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-03-29T10:57:37.450 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-03-29T10:57:37.514 INFO:teuthology.orchestra.run.smithi111.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-03-29T10:57:37.517 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-03-29T10:57:37.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:57:37.518 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2023-03-29T10:57:37.558 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-03-29T10:57:37.558 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-03-29T10:57:37.602 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2023-03-29T10:57:37.665 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-03-29T10:57:37.665 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-03-29T10:57:37.728 INFO:teuthology.orchestra.run.smithi111.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-03-29T10:57:37.730 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-03-29T10:57:37.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:57:37.730 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2023-03-29T10:57:37.759 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi111.stderr:2023-03-29T10:57:37.759+0000 7febacdc15c0 -1 init, newargv = 0x55a460e61650 newargc=15 2023-03-29T10:57:37.759 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi111.stderr:ceph-fuse[118142]: starting ceph client 2023-03-29T10:57:37.769 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi111.stderr:ceph-fuse[118142]: starting fuse 2023-03-29T10:57:37.775 INFO:teuthology.orchestra.run.smithi111.stdout:42 2023-03-29T10:57:37.775 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-03-29T10:57:37.776 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-03-29T10:57:37.776 DEBUG:teuthology.orchestra.run.smithi111:> import glob 2023-03-29T10:57:37.776 DEBUG:teuthology.orchestra.run.smithi111:> import re 2023-03-29T10:57:37.776 DEBUG:teuthology.orchestra.run.smithi111:> import os 2023-03-29T10:57:37.776 DEBUG:teuthology.orchestra.run.smithi111:> import subprocess 2023-03-29T10:57:37.776 DEBUG:teuthology.orchestra.run.smithi111:> 2023-03-29T10:57:37.776 DEBUG:teuthology.orchestra.run.smithi111:> def _find_admin_socket(client_name): 2023-03-29T10:57:37.776 DEBUG:teuthology.orchestra.run.smithi111:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-03-29T10:57:37.776 DEBUG:teuthology.orchestra.run.smithi111:> files = glob.glob(asok_path) 2023-03-29T10:57:37.776 DEBUG:teuthology.orchestra.run.smithi111:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-03-29T10:57:37.776 DEBUG:teuthology.orchestra.run.smithi111:> 2023-03-29T10:57:37.776 DEBUG:teuthology.orchestra.run.smithi111:> # Given a non-glob path, it better be there 2023-03-29T10:57:37.776 DEBUG:teuthology.orchestra.run.smithi111:> if "*" not in asok_path: 2023-03-29T10:57:37.776 DEBUG:teuthology.orchestra.run.smithi111:> assert(len(files) == 1) 2023-03-29T10:57:37.776 DEBUG:teuthology.orchestra.run.smithi111:> return files[0] 2023-03-29T10:57:37.777 DEBUG:teuthology.orchestra.run.smithi111:> 2023-03-29T10:57:37.777 DEBUG:teuthology.orchestra.run.smithi111:> for f in files: 2023-03-29T10:57:37.777 DEBUG:teuthology.orchestra.run.smithi111:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-03-29T10:57:37.777 DEBUG:teuthology.orchestra.run.smithi111:> if os.path.exists("/proc/{0}".format(pid)): 2023-03-29T10:57:37.777 DEBUG:teuthology.orchestra.run.smithi111:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-03-29T10:57:37.777 DEBUG:teuthology.orchestra.run.smithi111:> contents = proc_f.read() 2023-03-29T10:57:37.777 DEBUG:teuthology.orchestra.run.smithi111:> if mountpoint in contents: 2023-03-29T10:57:37.777 DEBUG:teuthology.orchestra.run.smithi111:> return f 2023-03-29T10:57:37.777 DEBUG:teuthology.orchestra.run.smithi111:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-03-29T10:57:37.777 DEBUG:teuthology.orchestra.run.smithi111:> 2023-03-29T10:57:37.777 DEBUG:teuthology.orchestra.run.smithi111:> print(_find_admin_socket("client.admin")) 2023-03-29T10:57:37.777 DEBUG:teuthology.orchestra.run.smithi111:> ' 2023-03-29T10:57:37.950 INFO:teuthology.orchestra.run.smithi111.stdout:/var/run/ceph/ceph-client.admin.118142.asok 2023-03-29T10:57:38.128 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.118142.asok 2023-03-29T10:57:38.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:57:38.128 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.118142.asok status 2023-03-29T10:57:38.345 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-03-29T10:57:38.345 INFO:teuthology.orchestra.run.smithi111.stdout: "metadata": { 2023-03-29T10:57:38.345 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_sha1": "f03d69a8741c605ae6711afe2c9991ac46095f7d", 2023-03-29T10:57:38.345 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 18.0.0-3066-gf03d69a8 (f03d69a8741c605ae6711afe2c9991ac46095f7d) reef (dev)", 2023-03-29T10:57:38.345 INFO:teuthology.orchestra.run.smithi111.stdout: "entity_id": "admin", 2023-03-29T10:57:38.345 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": "smithi111", 2023-03-29T10:57:38.346 INFO:teuthology.orchestra.run.smithi111.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-03-29T10:57:38.346 INFO:teuthology.orchestra.run.smithi111.stdout: "pid": "118142", 2023-03-29T10:57:38.346 INFO:teuthology.orchestra.run.smithi111.stdout: "root": "/" 2023-03-29T10:57:38.346 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:57:38.346 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_count": 0, 2023-03-29T10:57:38.346 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_pinned_count": 0, 2023-03-29T10:57:38.346 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 4473, 2023-03-29T10:57:38.346 INFO:teuthology.orchestra.run.smithi111.stdout: "inst": { 2023-03-29T10:57:38.346 INFO:teuthology.orchestra.run.smithi111.stdout: "name": { 2023-03-29T10:57:38.346 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "client", 2023-03-29T10:57:38.346 INFO:teuthology.orchestra.run.smithi111.stdout: "num": 4473 2023-03-29T10:57:38.346 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:57:38.346 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2023-03-29T10:57:38.346 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2023-03-29T10:57:38.346 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2023-03-29T10:57:38.346 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 283212172 2023-03-29T10:57:38.346 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:57:38.346 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:57:38.347 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2023-03-29T10:57:38.347 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2023-03-29T10:57:38.347 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2023-03-29T10:57:38.347 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 283212172 2023-03-29T10:57:38.347 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:57:38.347 INFO:teuthology.orchestra.run.smithi111.stdout: "inst_str": "client.4473 192.168.0.1:0/283212172", 2023-03-29T10:57:38.347 INFO:teuthology.orchestra.run.smithi111.stdout: "addr_str": "192.168.0.1:0/283212172", 2023-03-29T10:57:38.347 INFO:teuthology.orchestra.run.smithi111.stdout: "inode_count": 1, 2023-03-29T10:57:38.347 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_epoch": 7, 2023-03-29T10:57:38.347 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch": 28, 2023-03-29T10:57:38.347 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch_barrier": 0, 2023-03-29T10:57:38.347 INFO:teuthology.orchestra.run.smithi111.stdout: "blocklisted": false, 2023-03-29T10:57:38.347 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs" 2023-03-29T10:57:38.347 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-03-29T10:57:38.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:57:38.358 DEBUG:teuthology.orchestra.run.smithi111:> stat --file-system '--printf=%T 2023-03-29T10:57:38.358 DEBUG:teuthology.orchestra.run.smithi111:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-03-29T10:57:38.401 INFO:teuthology.orchestra.run.smithi111.stdout:fuseblk 2023-03-29T10:57:38.401 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-03-29T10:57:38.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:57:38.402 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-03-29T10:57:38.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:57:38.506 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-03-29T10:57:38.767 INFO:teuthology.orchestra.run.smithi111.stdout:# file: . 2023-03-29T10:57:38.768 INFO:teuthology.orchestra.run.smithi111.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-03-29T10:57:38.768 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:38.768 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:57:38.769 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-29T10:57:38.809 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:57:39.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.031+0000 7f3126bff700 1 Processor -- start 2023-03-29T10:57:39.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.031+0000 7f3126bff700 1 -- start start 2023-03-29T10:57:39.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.032+0000 7f3126bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f312015e1f0 0x7f31200cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:39.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.032+0000 7f3126bff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f312006aaf0 0x7f31200cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:39.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.033+0000 7f3126bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312015d020 con 0x7f3120146250 2023-03-29T10:57:39.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.033+0000 7f3126bff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312015dca0 con 0x7f312006aaf0 2023-03-29T10:57:39.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.033+0000 7f3126bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312015d300 con 0x7f312015e1f0 2023-03-29T10:57:39.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.033+0000 7f312519c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f312006aaf0 0x7f31200cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:39.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.033+0000 7f312519c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f312006aaf0 0x7f31200cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58004/0 (socket says 172.21.15.111:58004) 2023-03-29T10:57:39.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.033+0000 7f312519c700 1 -- 172.21.15.111:0/2900131657 learned_addr learned my addr 172.21.15.111:0/2900131657 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:39.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.033+0000 7f311ffff700 1 --2- 172.21.15.111:0/2900131657 >> v2:172.21.15.119:3300/0 conn(0x7f312015e1f0 0x7f31200cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:39.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.033+0000 7f312519c700 1 -- 172.21.15.111:0/2900131657 >> v2:172.21.15.119:3300/0 conn(0x7f312015e1f0 msgr2=0x7f31200cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:39.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.033+0000 7f312519c700 1 --2- 172.21.15.111:0/2900131657 >> v2:172.21.15.119:3300/0 conn(0x7f312015e1f0 0x7f31200cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.033+0000 7f312519c700 1 -- 172.21.15.111:0/2900131657 >> v1:172.21.15.119:6789/0 conn(0x7f3120146250 legacy=0x7f312006a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:39.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.033+0000 7f312519c700 1 -- 172.21.15.111:0/2900131657 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312015d5e0 con 0x7f312006aaf0 2023-03-29T10:57:39.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.033+0000 7f312519c700 1 --2- 172.21.15.111:0/2900131657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f312006aaf0 0x7f31200cddf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f312006d970 tx=0x7f311800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2e65c28d7f54ad server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:39.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.034+0000 7f311f7fe700 1 -- 172.21.15.111:0/2900131657 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311800e070 con 0x7f312006aaf0 2023-03-29T10:57:39.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.034+0000 7f311f7fe700 1 -- 172.21.15.111:0/2900131657 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3118003a20 con 0x7f312006aaf0 2023-03-29T10:57:39.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.034+0000 7f311f7fe700 1 -- 172.21.15.111:0/2900131657 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3118003dd0 con 0x7f312006aaf0 2023-03-29T10:57:39.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.034+0000 7f3126bff700 1 -- 172.21.15.111:0/2900131657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f312006aaf0 msgr2=0x7f31200cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:39.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.034+0000 7f3126bff700 1 --2- 172.21.15.111:0/2900131657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f312006aaf0 0x7f31200cddf0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f312006d970 tx=0x7f311800b330 comp rx=0 tx=0).stop 2023-03-29T10:57:39.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.034+0000 7f3126bff700 1 -- 172.21.15.111:0/2900131657 shutdown_connections 2023-03-29T10:57:39.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.034+0000 7f3126bff700 1 --2- 172.21.15.111:0/2900131657 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f312006aaf0 0x7f31200cddf0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.034+0000 7f3126bff700 1 --2- 172.21.15.111:0/2900131657 >> v2:172.21.15.119:3300/0 conn(0x7f312015e1f0 0x7f31200cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.034+0000 7f3126bff700 1 -- 172.21.15.111:0/2900131657 >> 172.21.15.111:0/2900131657 conn(0x7f312006dde0 msgr2=0x7f312006e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:39.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.034+0000 7f3126bff700 1 -- 172.21.15.111:0/2900131657 shutdown_connections 2023-03-29T10:57:39.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.035+0000 7f3126bff700 1 -- 172.21.15.111:0/2900131657 wait complete. 2023-03-29T10:57:39.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.036+0000 7f3126bff700 1 Processor -- start 2023-03-29T10:57:39.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.036+0000 7f3126bff700 1 -- start start 2023-03-29T10:57:39.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.037+0000 7f3126bff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f312006aaf0 0x7f31200c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:39.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.037+0000 7f3126bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3120146250 0x7f31201566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:39.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.037+0000 7f3126bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f312015e1f0 0x7f31200af620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:39.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.037+0000 7f3126bff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3120156be0 con 0x7f312006aaf0 2023-03-29T10:57:39.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.037+0000 7f3126bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31200c5050 con 0x7f312015e1f0 2023-03-29T10:57:39.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.037+0000 7f3126bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3120146010 con 0x7f3120146250 2023-03-29T10:57:39.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.037+0000 7f312499b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f312006aaf0 0x7f31200c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:39.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.038+0000 7f312499b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f312006aaf0 0x7f31200c62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58020/0 (socket says 172.21.15.111:58020) 2023-03-29T10:57:39.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.038+0000 7f312499b700 1 -- 172.21.15.111:0/108787267 learned_addr learned my addr 172.21.15.111:0/108787267 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:39.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.038+0000 7f312519c700 1 --2- 172.21.15.111:0/108787267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f312015e1f0 0x7f31200af620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:39.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.038+0000 7f311ffff700 1 --2- 172.21.15.111:0/108787267 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3120146250 0x7f31201566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:39.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.038+0000 7f312499b700 1 -- 172.21.15.111:0/108787267 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3120146250 msgr2=0x7f31201566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:39.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.038+0000 7f312499b700 1 --2- 172.21.15.111:0/108787267 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3120146250 0x7f31201566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.038+0000 7f312499b700 1 -- 172.21.15.111:0/108787267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f312015e1f0 msgr2=0x7f31200af620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:39.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.038+0000 7f312499b700 1 --2- 172.21.15.111:0/108787267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f312015e1f0 0x7f31200af620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.038+0000 7f312499b700 1 -- 172.21.15.111:0/108787267 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3118009b30 con 0x7f312006aaf0 2023-03-29T10:57:39.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.038+0000 7f312519c700 1 --2- 172.21.15.111:0/108787267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f312015e1f0 0x7f31200af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:39.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.038+0000 7f311ffff700 1 --2- 172.21.15.111:0/108787267 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3120146250 0x7f31201566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:39.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.038+0000 7f312499b700 1 --2- 172.21.15.111:0/108787267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f312006aaf0 0x7f31200c62a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f311400c4d0 tx=0x7f311400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:39.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.038+0000 7f311d7fa700 1 -- 172.21.15.111:0/108787267 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3114011790 con 0x7f312006aaf0 2023-03-29T10:57:39.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.038+0000 7f311d7fa700 1 -- 172.21.15.111:0/108787267 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31140118f0 con 0x7f312006aaf0 2023-03-29T10:57:39.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.038+0000 7f3126bff700 1 -- 172.21.15.111:0/108787267 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31200ce330 con 0x7f312006aaf0 2023-03-29T10:57:39.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.038+0000 7f311d7fa700 1 -- 172.21.15.111:0/108787267 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3114011ca0 con 0x7f312006aaf0 2023-03-29T10:57:39.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.039+0000 7f3126bff700 1 -- 172.21.15.111:0/108787267 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312015d8c0 con 0x7f312006aaf0 2023-03-29T10:57:39.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.039+0000 7f3126bff700 1 -- 172.21.15.111:0/108787267 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312006f930 con 0x7f312006aaf0 2023-03-29T10:57:39.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.040+0000 7f311d7fa700 1 -- 172.21.15.111:0/108787267 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f311401c460 con 0x7f312006aaf0 2023-03-29T10:57:39.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.042+0000 7f311d7fa700 1 --2- 172.21.15.111:0/108787267 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3104069de0 0x7f310406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:39.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.042+0000 7f311d7fa700 1 -- 172.21.15.111:0/108787267 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f31140881a0 con 0x7f312006aaf0 2023-03-29T10:57:39.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.042+0000 7f311ffff700 1 --2- 172.21.15.111:0/108787267 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3104069de0 0x7f310406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:39.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.043+0000 7f311ffff700 1 --2- 172.21.15.111:0/108787267 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3104069de0 0x7f310406c290 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f31201f5cc0 tx=0x7f310c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:39.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.047+0000 7f311d7fa700 1 -- 172.21.15.111:0/108787267 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f3114053660 con 0x7f312006aaf0 2023-03-29T10:57:39.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.241+0000 7f3126bff700 1 -- 172.21.15.111:0/108787267 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3120074430 con 0x7f312006aaf0 2023-03-29T10:57:39.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.242+0000 7f311d7fa700 1 -- 172.21.15.111:0/108787267 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19796 (secure 0 0 0) 0x7f3114056c80 con 0x7f312006aaf0 2023-03-29T10:57:39.243 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:39.243 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":28,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:57:32.331880+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:57:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.245+0000 7f3126bff700 1 -- 172.21.15.111:0/108787267 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3104069de0 msgr2=0x7f310406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.245+0000 7f3126bff700 1 --2- 172.21.15.111:0/108787267 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3104069de0 0x7f310406c290 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f31201f5cc0 tx=0x7f310c008040 comp rx=0 tx=0).stop 2023-03-29T10:57:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.245+0000 7f3126bff700 1 -- 172.21.15.111:0/108787267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f312006aaf0 msgr2=0x7f31200c62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.245+0000 7f3126bff700 1 --2- 172.21.15.111:0/108787267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f312006aaf0 0x7f31200c62a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f311400c4d0 tx=0x7f311400c890 comp rx=0 tx=0).stop 2023-03-29T10:57:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.245+0000 7f3126bff700 1 -- 172.21.15.111:0/108787267 shutdown_connections 2023-03-29T10:57:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.245+0000 7f3126bff700 1 --2- 172.21.15.111:0/108787267 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3104069de0 0x7f310406c290 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.245+0000 7f3126bff700 1 --2- 172.21.15.111:0/108787267 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f312006aaf0 0x7f31200c62a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.245+0000 7f3126bff700 1 --2- 172.21.15.111:0/108787267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f312015e1f0 0x7f31200af620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.245+0000 7f3126bff700 1 --2- 172.21.15.111:0/108787267 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3120146250 0x7f31201566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.245+0000 7f3126bff700 1 -- 172.21.15.111:0/108787267 >> 172.21.15.111:0/108787267 conn(0x7f312006dde0 msgr2=0x7f312006fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:39.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.245+0000 7f3126bff700 1 -- 172.21.15.111:0/108787267 shutdown_connections 2023-03-29T10:57:39.246 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.245+0000 7f3126bff700 1 -- 172.21.15.111:0/108787267 wait complete. 2023-03-29T10:57:39.262 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-29T10:57:16.183979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-29T10:57:21.239809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-29T10:57:26.192124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-29T10:57:26.796777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-29T10:57:29.874795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-29T10:57:39.263 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-29T10:57:39.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.449+0000 7efd133ff700 1 Processor -- start 2023-03-29T10:57:39.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.450+0000 7efd133ff700 1 -- start start 2023-03-29T10:57:39.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.451+0000 7efd133ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd0c0b87a0 0x7efd0c0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:39.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.451+0000 7efd133ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7efd0c1601e0 0x7efd0c160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:39.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.451+0000 7efd133ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efd0c160b70 0x7efd0c0d2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:39.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.451+0000 7efd133ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd0c0d02d0 con 0x7efd0c1601e0 2023-03-29T10:57:39.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.451+0000 7efd133ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd0c0d0cb0 con 0x7efd0c160b70 2023-03-29T10:57:39.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.451+0000 7efd133ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c0b2730 con 0x7efd0c0b87a0 2023-03-29T10:57:39.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.451+0000 7efd1099a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7efd0c1601e0 0x7efd0c160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:39.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.451+0000 7efd1099a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7efd0c1601e0 0x7efd0c160630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58034/0 (socket says 172.21.15.111:58034) 2023-03-29T10:57:39.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.451+0000 7efd1099a700 1 -- 172.21.15.111:0/3972949786 learned_addr learned my addr 172.21.15.111:0/3972949786 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:39.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.451+0000 7efd1199c700 1 --2- 172.21.15.111:0/3972949786 >> v2:172.21.15.119:3300/0 conn(0x7efd0c160b70 0x7efd0c0d2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:39.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.451+0000 7efd1119b700 1 --2- 172.21.15.111:0/3972949786 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd0c0b87a0 0x7efd0c0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:39.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.452+0000 7efd1099a700 1 -- 172.21.15.111:0/3972949786 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd0c0b87a0 msgr2=0x7efd0c0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:39.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.452+0000 7efd1099a700 1 --2- 172.21.15.111:0/3972949786 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd0c0b87a0 0x7efd0c0cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.452+0000 7efd1099a700 1 -- 172.21.15.111:0/3972949786 >> v2:172.21.15.119:3300/0 conn(0x7efd0c160b70 msgr2=0x7efd0c0d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:39.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.452+0000 7efd1099a700 1 --2- 172.21.15.111:0/3972949786 >> v2:172.21.15.119:3300/0 conn(0x7efd0c160b70 0x7efd0c0d2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.452+0000 7efd1099a700 1 -- 172.21.15.111:0/3972949786 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c160ff0 con 0x7efd0c1601e0 2023-03-29T10:57:39.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.452+0000 7efd1099a700 1 --2- 172.21.15.111:0/3972949786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd0c1601e0 0x7efd0c160630 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efd0c08be30 tx=0x7efcfc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d958f52b5945dff server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:39.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.452+0000 7efd03fff700 1 -- 172.21.15.111:0/3972949786 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc011830 con 0x7efd0c1601e0 2023-03-29T10:57:39.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.452+0000 7efd03fff700 1 -- 172.21.15.111:0/3972949786 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcfc011990 con 0x7efd0c1601e0 2023-03-29T10:57:39.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.452+0000 7efd03fff700 1 -- 172.21.15.111:0/3972949786 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc011d20 con 0x7efd0c1601e0 2023-03-29T10:57:39.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.453+0000 7efd133ff700 1 -- 172.21.15.111:0/3972949786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd0c1601e0 msgr2=0x7efd0c160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:39.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.453+0000 7efd133ff700 1 --2- 172.21.15.111:0/3972949786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd0c1601e0 0x7efd0c160630 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efd0c08be30 tx=0x7efcfc009bd0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.453+0000 7efd133ff700 1 -- 172.21.15.111:0/3972949786 shutdown_connections 2023-03-29T10:57:39.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.453+0000 7efd133ff700 1 --2- 172.21.15.111:0/3972949786 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd0c0b87a0 0x7efd0c0cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.453+0000 7efd133ff700 1 --2- 172.21.15.111:0/3972949786 >> v2:172.21.15.119:3300/0 conn(0x7efd0c160b70 0x7efd0c0d2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.453+0000 7efd133ff700 1 --2- 172.21.15.111:0/3972949786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd0c1601e0 0x7efd0c160630 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.453+0000 7efd133ff700 1 -- 172.21.15.111:0/3972949786 >> 172.21.15.111:0/3972949786 conn(0x7efd0c071b10 msgr2=0x7efd0c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:39.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.453+0000 7efd133ff700 1 -- 172.21.15.111:0/3972949786 shutdown_connections 2023-03-29T10:57:39.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.453+0000 7efd133ff700 1 -- 172.21.15.111:0/3972949786 wait complete. 2023-03-29T10:57:39.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.454+0000 7efd133ff700 1 Processor -- start 2023-03-29T10:57:39.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.455+0000 7efd133ff700 1 -- start start 2023-03-29T10:57:39.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.455+0000 7efd133ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd0c0b87a0 0x7efd0c1e5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:39.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.455+0000 7efd133ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd0c1601e0 0x7efd0c1e7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:39.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.455+0000 7efd133ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd0c160b70 0x7efd0c15a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:39.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.455+0000 7efd133ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c1e6fa0 con 0x7efd0c0b87a0 2023-03-29T10:57:39.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.455+0000 7efd133ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c15fba0 con 0x7efd0c1601e0 2023-03-29T10:57:39.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.456+0000 7efd133ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c1e69e0 con 0x7efd0c160b70 2023-03-29T10:57:39.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.456+0000 7efd1119b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd0c0b87a0 0x7efd0c1e5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:39.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.456+0000 7efd1119b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd0c0b87a0 0x7efd0c1e5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58046/0 (socket says 172.21.15.111:58046) 2023-03-29T10:57:39.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.456+0000 7efd1119b700 1 -- 172.21.15.111:0/2346763744 learned_addr learned my addr 172.21.15.111:0/2346763744 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:39.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.456+0000 7efd1119b700 1 -- 172.21.15.111:0/2346763744 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd0c160b70 msgr2=0x7efd0c15a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:39.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.456+0000 7efd1119b700 1 --2- 172.21.15.111:0/2346763744 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd0c160b70 0x7efd0c15a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.456+0000 7efd1099a700 1 --2- 172.21.15.111:0/2346763744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd0c1601e0 0x7efd0c1e7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:39.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.456+0000 7efd1119b700 1 -- 172.21.15.111:0/2346763744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd0c1601e0 msgr2=0x7efd0c1e7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:39.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.456+0000 7efd1119b700 1 --2- 172.21.15.111:0/2346763744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd0c1601e0 0x7efd0c1e7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.456+0000 7efd1119b700 1 -- 172.21.15.111:0/2346763744 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc009550 con 0x7efd0c0b87a0 2023-03-29T10:57:39.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.456+0000 7efd1119b700 1 --2- 172.21.15.111:0/2346763744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd0c0b87a0 0x7efd0c1e5f90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efd0800e7f0 tx=0x7efd0800ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:39.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.456+0000 7efd01ffb700 1 -- 172.21.15.111:0/2346763744 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08013680 con 0x7efd0c0b87a0 2023-03-29T10:57:39.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.457+0000 7efd01ffb700 1 -- 172.21.15.111:0/2346763744 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd080137e0 con 0x7efd0c0b87a0 2023-03-29T10:57:39.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.457+0000 7efd01ffb700 1 -- 172.21.15.111:0/2346763744 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08013b90 con 0x7efd0c0b87a0 2023-03-29T10:57:39.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.457+0000 7efd133ff700 1 -- 172.21.15.111:0/2346763744 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd0c0cd6a0 con 0x7efd0c0b87a0 2023-03-29T10:57:39.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.457+0000 7efd133ff700 1 -- 172.21.15.111:0/2346763744 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd0c0cdc60 con 0x7efd0c0b87a0 2023-03-29T10:57:39.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.458+0000 7efd133ff700 1 -- 172.21.15.111:0/2346763744 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd0c0cd980 con 0x7efd0c0b87a0 2023-03-29T10:57:39.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.458+0000 7efd01ffb700 1 -- 172.21.15.111:0/2346763744 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7efd08013cf0 con 0x7efd0c0b87a0 2023-03-29T10:57:39.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.460+0000 7efd01ffb700 1 --2- 172.21.15.111:0/2346763744 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7efcf0069de0 0x7efcf006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:39.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.460+0000 7efd01ffb700 1 -- 172.21.15.111:0/2346763744 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7efd08088190 con 0x7efd0c0b87a0 2023-03-29T10:57:39.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.460+0000 7efd1099a700 1 --2- 172.21.15.111:0/2346763744 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7efcf0069de0 0x7efcf006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:39.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.461+0000 7efd1099a700 1 --2- 172.21.15.111:0/2346763744 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7efcf0069de0 0x7efcf006c290 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0c6fc0 tx=0x7efcfc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:39.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.464+0000 7efd01ffb700 1 -- 172.21.15.111:0/2346763744 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7efd08053740 con 0x7efd0c0b87a0 2023-03-29T10:57:39.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.656+0000 7efd133ff700 1 -- 172.21.15.111:0/2346763744 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd0c074430 con 0x7efd0c0b87a0 2023-03-29T10:57:39.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.656+0000 7efd01ffb700 1 -- 172.21.15.111:0/2346763744 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd08056d60 con 0x7efd0c0b87a0 2023-03-29T10:57:39.656 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-03-29T10:57:39.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.658+0000 7efd133ff700 1 -- 172.21.15.111:0/2346763744 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7efcf0069de0 msgr2=0x7efcf006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:39.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.658+0000 7efd133ff700 1 --2- 172.21.15.111:0/2346763744 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7efcf0069de0 0x7efcf006c290 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0c6fc0 tx=0x7efcfc008030 comp rx=0 tx=0).stop 2023-03-29T10:57:39.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.658+0000 7efd133ff700 1 -- 172.21.15.111:0/2346763744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd0c0b87a0 msgr2=0x7efd0c1e5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:39.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.658+0000 7efd133ff700 1 --2- 172.21.15.111:0/2346763744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd0c0b87a0 0x7efd0c1e5f90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efd0800e7f0 tx=0x7efd0800ebb0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.659+0000 7efd133ff700 1 -- 172.21.15.111:0/2346763744 shutdown_connections 2023-03-29T10:57:39.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.659+0000 7efd133ff700 1 --2- 172.21.15.111:0/2346763744 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7efcf0069de0 0x7efcf006c290 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.659+0000 7efd133ff700 1 --2- 172.21.15.111:0/2346763744 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efd0c0b87a0 0x7efd0c1e5f90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.659+0000 7efd133ff700 1 --2- 172.21.15.111:0/2346763744 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efd0c160b70 0x7efd0c15a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.659+0000 7efd133ff700 1 --2- 172.21.15.111:0/2346763744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd0c1601e0 0x7efd0c1e7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.659+0000 7efd133ff700 1 -- 172.21.15.111:0/2346763744 >> 172.21.15.111:0/2346763744 conn(0x7efd0c071b10 msgr2=0x7efd0c15ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:39.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.659+0000 7efd133ff700 1 -- 172.21.15.111:0/2346763744 shutdown_connections 2023-03-29T10:57:39.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.659+0000 7efd133ff700 1 -- 172.21.15.111:0/2346763744 wait complete. 2023-03-29T10:57:39.675 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-29T10:57:39.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.860+0000 7fdaf73ff700 1 Processor -- start 2023-03-29T10:57:39.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.860+0000 7fdaf73ff700 1 -- start start 2023-03-29T10:57:39.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.861+0000 7fdaf73ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdaf00b87a0 0x7fdaf00cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:39.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.862+0000 7fdaf73ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf00cdc60 con 0x7fdaf0160d40 2023-03-29T10:57:39.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.862+0000 7fdaf73ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaf00cd6a0 con 0x7fdaf01601e0 2023-03-29T10:57:39.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.862+0000 7fdaf73ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaf00d02d0 con 0x7fdaf00b87a0 2023-03-29T10:57:39.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.862+0000 7fdaf519b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdaf00b87a0 0x7fdaf00cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:39.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.862+0000 7fdaf599c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fdaf0160d40 0x7fdaf00d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47432/0 (socket says 172.21.15.111:47432) 2023-03-29T10:57:39.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.862+0000 7fdaf599c700 1 -- 172.21.15.111:0/1687168207 learned_addr learned my addr 172.21.15.111:0/1687168207 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:39.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.862+0000 7fdaf519b700 1 -- 172.21.15.111:0/1687168207 >> v1:172.21.15.119:6789/0 conn(0x7fdaf01601e0 legacy=0x7fdaf0160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:39.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.862+0000 7fdaf519b700 1 -- 172.21.15.111:0/1687168207 >> v1:172.21.15.111:6789/0 conn(0x7fdaf0160d40 legacy=0x7fdaf00d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:39.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.862+0000 7fdaf519b700 1 -- 172.21.15.111:0/1687168207 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf00cd980 con 0x7fdaf00b87a0 2023-03-29T10:57:39.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.863+0000 7fdaf519b700 1 --2- 172.21.15.111:0/1687168207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdaf00b87a0 0x7fdaf00cfbf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009840 tx=0x7fdaec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3454c3ae818db6 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:39.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.863+0000 7fdae7fff700 1 -- 172.21.15.111:0/1687168207 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec011830 con 0x7fdaf00b87a0 2023-03-29T10:57:39.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.863+0000 7fdae7fff700 1 -- 172.21.15.111:0/1687168207 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaec011990 con 0x7fdaf00b87a0 2023-03-29T10:57:39.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.863+0000 7fdae7fff700 1 -- 172.21.15.111:0/1687168207 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaec011d20 con 0x7fdaf00b87a0 2023-03-29T10:57:39.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.864+0000 7fdaf73ff700 1 -- 172.21.15.111:0/1687168207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdaf00b87a0 msgr2=0x7fdaf00cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:39.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.864+0000 7fdaf73ff700 1 --2- 172.21.15.111:0/1687168207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdaf00b87a0 0x7fdaf00cfbf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009840 tx=0x7fdaec009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:39.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.864+0000 7fdaf73ff700 1 -- 172.21.15.111:0/1687168207 shutdown_connections 2023-03-29T10:57:39.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.864+0000 7fdaf73ff700 1 --2- 172.21.15.111:0/1687168207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdaf00b87a0 0x7fdaf00cfbf0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.864+0000 7fdaf73ff700 1 -- 172.21.15.111:0/1687168207 >> 172.21.15.111:0/1687168207 conn(0x7fdaf0071b10 msgr2=0x7fdaf0071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:39.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.864+0000 7fdaf73ff700 1 -- 172.21.15.111:0/1687168207 shutdown_connections 2023-03-29T10:57:39.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.864+0000 7fdaf73ff700 1 -- 172.21.15.111:0/1687168207 wait complete. 2023-03-29T10:57:39.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.866+0000 7fdaf73ff700 1 Processor -- start 2023-03-29T10:57:39.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.866+0000 7fdaf73ff700 1 -- start start 2023-03-29T10:57:39.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.867+0000 7fdaf73ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdaf00b87a0 0x7fdaf014d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:39.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.867+0000 7fdaf73ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdaf01601e0 0x7fdaf00cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:39.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.867+0000 7fdaf73ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdaf0160d40 0x7fdaf00c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:39.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.867+0000 7fdaf73ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf00cc730 con 0x7fdaf00b87a0 2023-03-29T10:57:39.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.867+0000 7fdaf73ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf00cbab0 con 0x7fdaf01601e0 2023-03-29T10:57:39.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.867+0000 7fdaf73ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf00cbd90 con 0x7fdaf0160d40 2023-03-29T10:57:39.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.867+0000 7fdaf519b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdaf00b87a0 0x7fdaf014d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:39.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.867+0000 7fdaf599c700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdaf0160d40 0x7fdaf00c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:39.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.868+0000 7fdaf519b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdaf00b87a0 0x7fdaf014d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58072/0 (socket says 172.21.15.111:58072) 2023-03-29T10:57:39.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.868+0000 7fdaf519b700 1 -- 172.21.15.111:0/196276351 learned_addr learned my addr 172.21.15.111:0/196276351 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:39.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.868+0000 7fdaf499a700 1 --2- 172.21.15.111:0/196276351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdaf01601e0 0x7fdaf00cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:39.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.868+0000 7fdaf499a700 1 -- 172.21.15.111:0/196276351 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdaf0160d40 msgr2=0x7fdaf00c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:39.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.868+0000 7fdaf499a700 1 --2- 172.21.15.111:0/196276351 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdaf0160d40 0x7fdaf00c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.868+0000 7fdaf499a700 1 -- 172.21.15.111:0/196276351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdaf00b87a0 msgr2=0x7fdaf014d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:39.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.868+0000 7fdaf499a700 1 --2- 172.21.15.111:0/196276351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdaf00b87a0 0x7fdaf014d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:39.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.868+0000 7fdaf499a700 1 -- 172.21.15.111:0/196276351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaec0094a0 con 0x7fdaf01601e0 2023-03-29T10:57:39.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.868+0000 7fdaf519b700 1 --2- 172.21.15.111:0/196276351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdaf00b87a0 0x7fdaf014d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:39.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.869+0000 7fdaf499a700 1 --2- 172.21.15.111:0/196276351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdaf01601e0 0x7fdaf00cb260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdae0000f60 tx=0x7fdae000c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:39.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.869+0000 7fdae5ffb700 1 -- 172.21.15.111:0/196276351 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae00052d0 con 0x7fdaf01601e0 2023-03-29T10:57:39.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.869+0000 7fdae5ffb700 1 -- 172.21.15.111:0/196276351 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae0005430 con 0x7fdaf01601e0 2023-03-29T10:57:39.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.869+0000 7fdae5ffb700 1 -- 172.21.15.111:0/196276351 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae00038c0 con 0x7fdaf01601e0 2023-03-29T10:57:39.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.869+0000 7fdaf73ff700 1 -- 172.21.15.111:0/196276351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf00d19a0 con 0x7fdaf01601e0 2023-03-29T10:57:39.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.869+0000 7fdaf73ff700 1 -- 172.21.15.111:0/196276351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf00d1810 con 0x7fdaf01601e0 2023-03-29T10:57:39.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.870+0000 7fdaf73ff700 1 -- 172.21.15.111:0/196276351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf00d1b30 con 0x7fdaf01601e0 2023-03-29T10:57:39.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.871+0000 7fdae5ffb700 1 -- 172.21.15.111:0/196276351 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fdae0003a20 con 0x7fdaf01601e0 2023-03-29T10:57:39.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.872+0000 7fdae5ffb700 1 --2- 172.21.15.111:0/196276351 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdad4069de0 0x7fdad406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:39.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.872+0000 7fdae5ffb700 1 -- 172.21.15.111:0/196276351 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdae0087590 con 0x7fdaf01601e0 2023-03-29T10:57:39.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.872+0000 7fdaf519b700 1 --2- 172.21.15.111:0/196276351 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdad4069de0 0x7fdad406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:39.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.873+0000 7fdaf519b700 1 --2- 172.21.15.111:0/196276351 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdad4069de0 0x7fdad406c290 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009b90 tx=0x7fdaec008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:39.879 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:39.880+0000 7fdae5ffb700 1 -- 172.21.15.111:0/196276351 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fdae0052b40 con 0x7fdaf01601e0 2023-03-29T10:57:40.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.069+0000 7fdaf73ff700 1 -- 172.21.15.111:0/196276351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdaf00cc870 con 0x7fdaf01601e0 2023-03-29T10:57:40.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.070+0000 7fdae5ffb700 1 -- 172.21.15.111:0/196276351 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdaf00cc870 con 0x7fdaf01601e0 2023-03-29T10:57:40.070 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 8 2023-03-29T10:57:40.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.072+0000 7fdaf73ff700 1 -- 172.21.15.111:0/196276351 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdad4069de0 msgr2=0x7fdad406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:40.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.072+0000 7fdaf73ff700 1 --2- 172.21.15.111:0/196276351 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdad4069de0 0x7fdad406c290 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009b90 tx=0x7fdaec008030 comp rx=0 tx=0).stop 2023-03-29T10:57:40.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.072+0000 7fdaf73ff700 1 -- 172.21.15.111:0/196276351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdaf01601e0 msgr2=0x7fdaf00cb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:40.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.072+0000 7fdaf73ff700 1 --2- 172.21.15.111:0/196276351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdaf01601e0 0x7fdaf00cb260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdae0000f60 tx=0x7fdae000c730 comp rx=0 tx=0).stop 2023-03-29T10:57:40.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.072+0000 7fdaf73ff700 1 -- 172.21.15.111:0/196276351 shutdown_connections 2023-03-29T10:57:40.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.072+0000 7fdaf73ff700 1 --2- 172.21.15.111:0/196276351 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdad4069de0 0x7fdad406c290 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.072+0000 7fdaf73ff700 1 --2- 172.21.15.111:0/196276351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdaf00b87a0 0x7fdaf014d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.072+0000 7fdaf73ff700 1 --2- 172.21.15.111:0/196276351 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdaf0160d40 0x7fdaf00c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.072+0000 7fdaf73ff700 1 --2- 172.21.15.111:0/196276351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdaf01601e0 0x7fdaf00cb260 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.072+0000 7fdaf73ff700 1 -- 172.21.15.111:0/196276351 >> 172.21.15.111:0/196276351 conn(0x7fdaf0071b10 msgr2=0x7fdaf00b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:40.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.072+0000 7fdaf73ff700 1 -- 172.21.15.111:0/196276351 shutdown_connections 2023-03-29T10:57:40.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.072+0000 7fdaf73ff700 1 -- 172.21.15.111:0/196276351 wait complete. 2023-03-29T10:57:40.090 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-29T10:57:40.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.278+0000 7f1cc3dff700 1 Processor -- start 2023-03-29T10:57:40.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.278+0000 7f1cc3dff700 1 -- start start 2023-03-29T10:57:40.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.279+0000 7f1cc3dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1cbc0b87a0 0x7f1cbc0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:40.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.280+0000 7f1cc3dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cbc160d40 0x7f1cbc0d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:40.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.280+0000 7f1cc3dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cbc0cdc60 con 0x7f1cbc1601e0 2023-03-29T10:57:40.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.280+0000 7f1cc3dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cbc0d02d0 con 0x7f1cbc0b87a0 2023-03-29T10:57:40.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.280+0000 7f1cc3dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc0d0cb0 con 0x7f1cbc160d40 2023-03-29T10:57:40.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.280+0000 7f1cc239c700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cbc160d40 0x7f1cbc0d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:40.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.280+0000 7f1cc239c700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cbc160d40 0x7f1cbc0d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:55942/0 (socket says 172.21.15.111:55942) 2023-03-29T10:57:40.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.280+0000 7f1cc239c700 1 -- 172.21.15.111:0/508261865 learned_addr learned my addr 172.21.15.111:0/508261865 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:40.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.280+0000 7f1cc1b9b700 1 --2- 172.21.15.111:0/508261865 >> v2:172.21.15.119:3300/0 conn(0x7f1cbc0b87a0 0x7f1cbc0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:40.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.280+0000 7f1cc239c700 1 -- 172.21.15.111:0/508261865 >> v2:172.21.15.119:3300/0 conn(0x7f1cbc0b87a0 msgr2=0x7f1cbc0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:40.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.280+0000 7f1cc239c700 1 --2- 172.21.15.111:0/508261865 >> v2:172.21.15.119:3300/0 conn(0x7f1cbc0b87a0 0x7f1cbc0cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.280+0000 7f1cc239c700 1 -- 172.21.15.111:0/508261865 >> v1:172.21.15.119:6789/0 conn(0x7f1cbc1601e0 legacy=0x7f1cbc160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:40.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.280+0000 7f1cc239c700 1 -- 172.21.15.111:0/508261865 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc0cd6a0 con 0x7f1cbc160d40 2023-03-29T10:57:40.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.281+0000 7f1cc239c700 1 --2- 172.21.15.111:0/508261865 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cbc160d40 0x7f1cbc0d1100 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009e50 tx=0x7f1cb400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c249296d99709c3 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:40.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.281+0000 7f1cc0b99700 1 -- 172.21.15.111:0/508261865 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb400e070 con 0x7f1cbc160d40 2023-03-29T10:57:40.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.281+0000 7f1cc0b99700 1 -- 172.21.15.111:0/508261865 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb400b910 con 0x7f1cbc160d40 2023-03-29T10:57:40.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.281+0000 7f1cc0b99700 1 -- 172.21.15.111:0/508261865 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb400bcc0 con 0x7f1cbc160d40 2023-03-29T10:57:40.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.282+0000 7f1cc3dff700 1 -- 172.21.15.111:0/508261865 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cbc160d40 msgr2=0x7f1cbc0d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:40.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.282+0000 7f1cc3dff700 1 --2- 172.21.15.111:0/508261865 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cbc160d40 0x7f1cbc0d1100 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009e50 tx=0x7f1cb400b330 comp rx=0 tx=0).stop 2023-03-29T10:57:40.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.282+0000 7f1cc3dff700 1 -- 172.21.15.111:0/508261865 shutdown_connections 2023-03-29T10:57:40.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.282+0000 7f1cc3dff700 1 --2- 172.21.15.111:0/508261865 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cbc160d40 0x7f1cbc0d1100 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.282+0000 7f1cc3dff700 1 --2- 172.21.15.111:0/508261865 >> v2:172.21.15.119:3300/0 conn(0x7f1cbc0b87a0 0x7f1cbc0cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.282+0000 7f1cc3dff700 1 -- 172.21.15.111:0/508261865 >> 172.21.15.111:0/508261865 conn(0x7f1cbc071b10 msgr2=0x7f1cbc071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:40.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.282+0000 7f1cc3dff700 1 -- 172.21.15.111:0/508261865 shutdown_connections 2023-03-29T10:57:40.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.282+0000 7f1cc3dff700 1 -- 172.21.15.111:0/508261865 wait complete. 2023-03-29T10:57:40.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.283+0000 7f1cc3dff700 1 Processor -- start 2023-03-29T10:57:40.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.283+0000 7f1cc3dff700 1 -- start start 2023-03-29T10:57:40.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.284+0000 7f1cc3dff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cbc0b87a0 0x7f1cbc1507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:40.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.284+0000 7f1cc3dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cbc1601e0 0x7f1cbc0c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:40.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.284+0000 7f1cc3dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cbc160d40 0x7f1cbc1511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:40.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.284+0000 7f1cc3dff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc150d20 con 0x7f1cbc0b87a0 2023-03-29T10:57:40.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.284+0000 7f1cc3dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc0c7850 con 0x7f1cbc160d40 2023-03-29T10:57:40.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.284+0000 7f1cc3dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc151700 con 0x7f1cbc1601e0 2023-03-29T10:57:40.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.285+0000 7f1cc1b9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cbc0b87a0 0x7f1cbc1507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:40.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.285+0000 7f1cc1b9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cbc0b87a0 0x7f1cbc1507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58078/0 (socket says 172.21.15.111:58078) 2023-03-29T10:57:40.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.285+0000 7f1cc1b9b700 1 -- 172.21.15.111:0/992836629 learned_addr learned my addr 172.21.15.111:0/992836629 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:40.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.285+0000 7f1cc139a700 1 --2- 172.21.15.111:0/992836629 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cbc1601e0 0x7f1cbc0c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:40.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.285+0000 7f1cc239c700 1 --2- 172.21.15.111:0/992836629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cbc160d40 0x7f1cbc1511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:40.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.285+0000 7f1cc1b9b700 1 -- 172.21.15.111:0/992836629 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cbc1601e0 msgr2=0x7f1cbc0c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:40.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.285+0000 7f1cc1b9b700 1 --2- 172.21.15.111:0/992836629 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cbc1601e0 0x7f1cbc0c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.285+0000 7f1cc1b9b700 1 -- 172.21.15.111:0/992836629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cbc160d40 msgr2=0x7f1cbc1511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:40.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.285+0000 7f1cc1b9b700 1 --2- 172.21.15.111:0/992836629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cbc160d40 0x7f1cbc1511c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.285+0000 7f1cc1b9b700 1 -- 172.21.15.111:0/992836629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb4009b30 con 0x7f1cbc0b87a0 2023-03-29T10:57:40.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.285+0000 7f1cc1b9b700 1 --2- 172.21.15.111:0/992836629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cbc0b87a0 0x7f1cbc1507e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800e9b0 tx=0x7f1cb800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:40.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.286+0000 7f1cb27fc700 1 -- 172.21.15.111:0/992836629 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8009710 con 0x7f1cbc0b87a0 2023-03-29T10:57:40.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.286+0000 7f1cb27fc700 1 -- 172.21.15.111:0/992836629 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb800ee50 con 0x7f1cbc0b87a0 2023-03-29T10:57:40.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.286+0000 7f1cb27fc700 1 -- 172.21.15.111:0/992836629 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb800ca50 con 0x7f1cbc0b87a0 2023-03-29T10:57:40.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.286+0000 7f1cc3dff700 1 -- 172.21.15.111:0/992836629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cbc0d1640 con 0x7f1cbc0b87a0 2023-03-29T10:57:40.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.286+0000 7f1cc3dff700 1 -- 172.21.15.111:0/992836629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cbc0cd980 con 0x7f1cbc0b87a0 2023-03-29T10:57:40.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.287+0000 7f1cc3dff700 1 -- 172.21.15.111:0/992836629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cbc0d17d0 con 0x7f1cbc0b87a0 2023-03-29T10:57:40.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.288+0000 7f1cb27fc700 1 -- 172.21.15.111:0/992836629 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f1cb800cbb0 con 0x7f1cbc0b87a0 2023-03-29T10:57:40.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.288+0000 7f1cb27fc700 1 --2- 172.21.15.111:0/992836629 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1ca0069de0 0x7f1ca006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:40.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.289+0000 7f1cb27fc700 1 -- 172.21.15.111:0/992836629 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1cb8088300 con 0x7f1cbc0b87a0 2023-03-29T10:57:40.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.289+0000 7f1cc139a700 1 --2- 172.21.15.111:0/992836629 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1ca0069de0 0x7f1ca006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:40.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.289+0000 7f1cc139a700 1 --2- 172.21.15.111:0/992836629 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1ca0069de0 0x7f1ca006c290 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc1f5a60 tx=0x7f1cac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:40.292 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.292+0000 7f1cb27fc700 1 -- 172.21.15.111:0/992836629 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f1cb80572e0 con 0x7f1cbc0b87a0 2023-03-29T10:57:40.483 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.484+0000 7f1cc3dff700 1 -- 172.21.15.111:0/992836629 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1cbc074430 con 0x7f1cbc0b87a0 2023-03-29T10:57:40.484 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.484+0000 7f1cb27fc700 1 -- 172.21.15.111:0/992836629 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1cb800ce60 con 0x7f1cbc0b87a0 2023-03-29T10:57:40.484 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:57:40.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.486+0000 7f1cc3dff700 1 -- 172.21.15.111:0/992836629 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1ca0069de0 msgr2=0x7f1ca006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:40.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.486+0000 7f1cc3dff700 1 --2- 172.21.15.111:0/992836629 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1ca0069de0 0x7f1ca006c290 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc1f5a60 tx=0x7f1cac009040 comp rx=0 tx=0).stop 2023-03-29T10:57:40.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.486+0000 7f1cc3dff700 1 -- 172.21.15.111:0/992836629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cbc0b87a0 msgr2=0x7f1cbc1507e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:40.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.486+0000 7f1cc3dff700 1 --2- 172.21.15.111:0/992836629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cbc0b87a0 0x7f1cbc1507e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1cb800e9b0 tx=0x7f1cb800ecc0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.486+0000 7f1cc3dff700 1 -- 172.21.15.111:0/992836629 shutdown_connections 2023-03-29T10:57:40.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.486+0000 7f1cc3dff700 1 --2- 172.21.15.111:0/992836629 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1ca0069de0 0x7f1ca006c290 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.486+0000 7f1cc3dff700 1 --2- 172.21.15.111:0/992836629 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cbc0b87a0 0x7f1cbc1507e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.487 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.486+0000 7f1cc3dff700 1 --2- 172.21.15.111:0/992836629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cbc160d40 0x7f1cbc1511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.487 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.486+0000 7f1cc3dff700 1 --2- 172.21.15.111:0/992836629 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cbc1601e0 0x7f1cbc0c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.487 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.486+0000 7f1cc3dff700 1 -- 172.21.15.111:0/992836629 >> 172.21.15.111:0/992836629 conn(0x7f1cbc071b10 msgr2=0x7f1cbc160610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:40.487 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.487+0000 7f1cc3dff700 1 -- 172.21.15.111:0/992836629 shutdown_connections 2023-03-29T10:57:40.487 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.487+0000 7f1cc3dff700 1 -- 172.21.15.111:0/992836629 wait complete. 2023-03-29T10:57:40.503 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-29T10:57:40.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.690+0000 7f5fd9bff700 1 Processor -- start 2023-03-29T10:57:40.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.690+0000 7f5fd9bff700 1 -- start start 2023-03-29T10:57:40.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.691+0000 7f5fd9bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5fd4071200 0x7f5fd4165340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:40.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.691+0000 7f5fd9bff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd40cb280 con 0x7f5fd415a5a0 2023-03-29T10:57:40.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.691+0000 7f5fd9bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd4071700 con 0x7f5fd406aaf0 2023-03-29T10:57:40.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.691+0000 7f5fd9bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd415a970 con 0x7f5fd4071200 2023-03-29T10:57:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.692+0000 7f5fd37fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f5fd415a5a0 0x7f5fd406a3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47438/0 (socket says 172.21.15.111:47438) 2023-03-29T10:57:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.692+0000 7f5fd37fe700 1 -- 172.21.15.111:0/3853378363 learned_addr learned my addr 172.21.15.111:0/3853378363 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.692+0000 7f5fd3fff700 1 --2- 172.21.15.111:0/3853378363 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5fd4071200 0x7f5fd4165340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.692+0000 7f5fd2ffd700 1 -- 172.21.15.111:0/3853378363 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646078267 0 0) 0x7f5fc4001110 con 0x7f5fd415a5a0 2023-03-29T10:57:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.692+0000 7f5fd2ffd700 1 -- 172.21.15.111:0/3853378363 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fc00035b0 con 0x7f5fd415a5a0 2023-03-29T10:57:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.692+0000 7f5fd2ffd700 1 -- 172.21.15.111:0/3853378363 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408467919 0 0) 0x7f5fd4071700 con 0x7f5fd406aaf0 2023-03-29T10:57:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.692+0000 7f5fd2ffd700 1 -- 172.21.15.111:0/3853378363 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fc4001110 con 0x7f5fd406aaf0 2023-03-29T10:57:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.692+0000 7f5fd3fff700 1 -- 172.21.15.111:0/3853378363 >> v1:172.21.15.119:6789/0 conn(0x7f5fd406aaf0 legacy=0x7f5fd4070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.692+0000 7f5fd2ffd700 1 -- 172.21.15.111:0/3853378363 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1240061208 0 0) 0x7f5fc00035b0 con 0x7f5fd415a5a0 2023-03-29T10:57:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.692+0000 7f5fd3fff700 1 -- 172.21.15.111:0/3853378363 >> v1:172.21.15.111:6789/0 conn(0x7f5fd415a5a0 legacy=0x7f5fd406a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:40.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.692+0000 7f5fd3fff700 1 -- 172.21.15.111:0/3853378363 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd40719e0 con 0x7f5fd4071200 2023-03-29T10:57:40.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.693+0000 7f5fd3fff700 1 --2- 172.21.15.111:0/3853378363 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5fd4071200 0x7f5fd4165340 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009dc0 tx=0x7f5fcc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=205f1d10075e3b21 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:40.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.693+0000 7f5fd2ffd700 1 -- 172.21.15.111:0/3853378363 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc00e070 con 0x7f5fd4071200 2023-03-29T10:57:40.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.693+0000 7f5fd2ffd700 1 -- 172.21.15.111:0/3853378363 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fcc003a20 con 0x7f5fd4071200 2023-03-29T10:57:40.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.693+0000 7f5fd2ffd700 1 -- 172.21.15.111:0/3853378363 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4001800 con 0x7f5fd4071200 2023-03-29T10:57:40.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.693+0000 7f5fd9bff700 1 -- 172.21.15.111:0/3853378363 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5fd4071200 msgr2=0x7f5fd4165340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:40.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.693+0000 7f5fd9bff700 1 --2- 172.21.15.111:0/3853378363 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5fd4071200 0x7f5fd4165340 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc009dc0 tx=0x7f5fcc00b330 comp rx=0 tx=0).stop 2023-03-29T10:57:40.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.693+0000 7f5fd9bff700 1 -- 172.21.15.111:0/3853378363 shutdown_connections 2023-03-29T10:57:40.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.693+0000 7f5fd9bff700 1 --2- 172.21.15.111:0/3853378363 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5fd4071200 0x7f5fd4165340 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.693+0000 7f5fd9bff700 1 -- 172.21.15.111:0/3853378363 >> 172.21.15.111:0/3853378363 conn(0x7f5fd40cf720 msgr2=0x7f5fd40cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:40.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.694+0000 7f5fd9bff700 1 -- 172.21.15.111:0/3853378363 shutdown_connections 2023-03-29T10:57:40.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.694+0000 7f5fd9bff700 1 -- 172.21.15.111:0/3853378363 wait complete. 2023-03-29T10:57:40.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.696+0000 7f5fd9bff700 1 Processor -- start 2023-03-29T10:57:40.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.696+0000 7f5fd9bff700 1 -- start start 2023-03-29T10:57:40.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.697+0000 7f5fd9bff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fd406aaf0 0x7f5fd4150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:40.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.697+0000 7f5fd9bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5fd4071200 0x7f5fd4150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:40.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.697+0000 7f5fd9bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fd415a5a0 0x7f5fd4151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:40.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.697+0000 7f5fd9bff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd40b4650 con 0x7f5fd406aaf0 2023-03-29T10:57:40.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.697+0000 7f5fd9bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd40b22d0 con 0x7f5fd415a5a0 2023-03-29T10:57:40.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.697+0000 7f5fd9bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd40b25b0 con 0x7f5fd4071200 2023-03-29T10:57:40.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.697+0000 7f5fd37fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fd406aaf0 0x7f5fd4150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:40.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.697+0000 7f5fd37fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fd406aaf0 0x7f5fd4150590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58092/0 (socket says 172.21.15.111:58092) 2023-03-29T10:57:40.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.697+0000 7f5fd37fe700 1 -- 172.21.15.111:0/3106809348 learned_addr learned my addr 172.21.15.111:0/3106809348 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:40.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.698+0000 7f5fcbfff700 1 --2- 172.21.15.111:0/3106809348 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5fd4071200 0x7f5fd4150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:40.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.698+0000 7f5fd3fff700 1 --2- 172.21.15.111:0/3106809348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fd415a5a0 0x7f5fd4151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:40.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.698+0000 7f5fd37fe700 1 -- 172.21.15.111:0/3106809348 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5fd4071200 msgr2=0x7f5fd4150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:40.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.698+0000 7f5fd37fe700 1 --2- 172.21.15.111:0/3106809348 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5fd4071200 0x7f5fd4150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.698+0000 7f5fd37fe700 1 -- 172.21.15.111:0/3106809348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fd415a5a0 msgr2=0x7f5fd4151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:40.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.698+0000 7f5fd37fe700 1 --2- 172.21.15.111:0/3106809348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fd415a5a0 0x7f5fd4151380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.698+0000 7f5fd37fe700 1 -- 172.21.15.111:0/3106809348 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc009a70 con 0x7f5fd406aaf0 2023-03-29T10:57:40.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.698+0000 7f5fd3fff700 1 --2- 172.21.15.111:0/3106809348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fd415a5a0 0x7f5fd4151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:57:40.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.698+0000 7f5fd37fe700 1 --2- 172.21.15.111:0/3106809348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fd406aaf0 0x7f5fd4150590 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400c780 tx=0x7f5fc400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:40.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.698+0000 7f5fd0ff9700 1 -- 172.21.15.111:0/3106809348 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4004c80 con 0x7f5fd406aaf0 2023-03-29T10:57:40.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.698+0000 7f5fd0ff9700 1 -- 172.21.15.111:0/3106809348 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc4004de0 con 0x7f5fd406aaf0 2023-03-29T10:57:40.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.698+0000 7f5fd9bff700 1 -- 172.21.15.111:0/3106809348 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd406c9a0 con 0x7f5fd406aaf0 2023-03-29T10:57:40.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.698+0000 7f5fd0ff9700 1 -- 172.21.15.111:0/3106809348 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc40072d0 con 0x7f5fd406aaf0 2023-03-29T10:57:40.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.698+0000 7f5fd9bff700 1 -- 172.21.15.111:0/3106809348 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd406c6c0 con 0x7f5fd406aaf0 2023-03-29T10:57:40.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.700+0000 7f5fd9bff700 1 -- 172.21.15.111:0/3106809348 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd406cc80 con 0x7f5fd406aaf0 2023-03-29T10:57:40.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.700+0000 7f5fd0ff9700 1 -- 172.21.15.111:0/3106809348 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f5fc4007430 con 0x7f5fd406aaf0 2023-03-29T10:57:40.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.701+0000 7f5fd0ff9700 1 --2- 172.21.15.111:0/3106809348 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5fb4069de0 0x7f5fb406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:40.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.701+0000 7f5fd0ff9700 1 -- 172.21.15.111:0/3106809348 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5fc4087720 con 0x7f5fd406aaf0 2023-03-29T10:57:40.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.702+0000 7f5fcbfff700 1 --2- 172.21.15.111:0/3106809348 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5fb4069de0 0x7f5fb406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:40.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.702+0000 7f5fcbfff700 1 --2- 172.21.15.111:0/3106809348 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5fb4069de0 0x7f5fb406c290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40ad440 tx=0x7f5fbc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:40.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.705+0000 7f5fd0ff9700 1 -- 172.21.15.111:0/3106809348 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f5fc4052cd0 con 0x7f5fd406aaf0 2023-03-29T10:57:40.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.898+0000 7f5fd9bff700 1 -- 172.21.15.111:0/3106809348 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5fd40cc5d0 con 0x7f5fd406aaf0 2023-03-29T10:57:40.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.899+0000 7f5fd0ff9700 1 -- 172.21.15.111:0/3106809348 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5fc40562f0 con 0x7f5fd406aaf0 2023-03-29T10:57:40.898 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-03-29T10:57:40.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.901+0000 7f5fd9bff700 1 -- 172.21.15.111:0/3106809348 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5fb4069de0 msgr2=0x7f5fb406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:40.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.901+0000 7f5fd9bff700 1 --2- 172.21.15.111:0/3106809348 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5fb4069de0 0x7f5fb406c290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40ad440 tx=0x7f5fbc009040 comp rx=0 tx=0).stop 2023-03-29T10:57:40.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.901+0000 7f5fd9bff700 1 -- 172.21.15.111:0/3106809348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fd406aaf0 msgr2=0x7f5fd4150590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:40.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.901+0000 7f5fd9bff700 1 --2- 172.21.15.111:0/3106809348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fd406aaf0 0x7f5fd4150590 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400c780 tx=0x7f5fc400ca90 comp rx=0 tx=0).stop 2023-03-29T10:57:40.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.901+0000 7f5fd9bff700 1 -- 172.21.15.111:0/3106809348 shutdown_connections 2023-03-29T10:57:40.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.901+0000 7f5fd9bff700 1 --2- 172.21.15.111:0/3106809348 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5fb4069de0 0x7f5fb406c290 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.901+0000 7f5fd9bff700 1 --2- 172.21.15.111:0/3106809348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5fd406aaf0 0x7f5fd4150590 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.901+0000 7f5fd9bff700 1 --2- 172.21.15.111:0/3106809348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fd415a5a0 0x7f5fd4151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.901+0000 7f5fd9bff700 1 --2- 172.21.15.111:0/3106809348 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5fd4071200 0x7f5fd4150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:40.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.901+0000 7f5fd9bff700 1 -- 172.21.15.111:0/3106809348 >> 172.21.15.111:0/3106809348 conn(0x7f5fd40cf720 msgr2=0x7f5fd406a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:40.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.901+0000 7f5fd9bff700 1 -- 172.21.15.111:0/3106809348 shutdown_connections 2023-03-29T10:57:40.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:40.901+0000 7f5fd9bff700 1 -- 172.21.15.111:0/3106809348 wait complete. 2023-03-29T10:57:40.918 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-29T10:57:41.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.105+0000 7f8fc67ff700 1 Processor -- start 2023-03-29T10:57:41.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.105+0000 7f8fc67ff700 1 -- start start 2023-03-29T10:57:41.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.106+0000 7f8fc67ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8fc00cff50 0x7f8fc0160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:41.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.106+0000 7f8fc67ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc00cd520 con 0x7f8fc015c590 2023-03-29T10:57:41.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.106+0000 7f8fc67ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc00d2640 con 0x7f8fc00cf010 2023-03-29T10:57:41.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.106+0000 7f8fc67ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc01610d0 con 0x7f8fc00cff50 2023-03-29T10:57:41.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.106+0000 7f8fbffff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f8fc015c590 0x7f8fc00cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47446/0 (socket says 172.21.15.111:47446) 2023-03-29T10:57:41.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.106+0000 7f8fbffff700 1 -- 172.21.15.111:0/4185415728 learned_addr learned my addr 172.21.15.111:0/4185415728 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:41.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.106+0000 7f8fc4d9c700 1 --2- 172.21.15.111:0/4185415728 >> v2:172.21.15.111:3300/0 conn(0x7f8fc00cff50 0x7f8fc0160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:41.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.107+0000 7f8fc4d9c700 1 -- 172.21.15.111:0/4185415728 >> v1:172.21.15.119:6789/0 conn(0x7f8fc00cf010 legacy=0x7f8fc00d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:41.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.107+0000 7f8fc4d9c700 1 -- 172.21.15.111:0/4185415728 >> v1:172.21.15.111:6789/0 conn(0x7f8fc015c590 legacy=0x7f8fc00cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:41.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.107+0000 7f8fc4d9c700 1 -- 172.21.15.111:0/4185415728 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc00b23c0 con 0x7f8fc00cff50 2023-03-29T10:57:41.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.107+0000 7f8fc4d9c700 1 --2- 172.21.15.111:0/4185415728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fc00cff50 0x7f8fc0160b90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009d90 tx=0x7f8fb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2386be4f38561793 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:41.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.107+0000 7f8fbeffd700 1 -- 172.21.15.111:0/4185415728 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb800e070 con 0x7f8fc00cff50 2023-03-29T10:57:41.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.107+0000 7f8fbeffd700 1 -- 172.21.15.111:0/4185415728 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb800bbd0 con 0x7f8fc00cff50 2023-03-29T10:57:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.107+0000 7f8fbeffd700 1 -- 172.21.15.111:0/4185415728 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8012430 con 0x7f8fc00cff50 2023-03-29T10:57:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.108+0000 7f8fc67ff700 1 -- 172.21.15.111:0/4185415728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fc00cff50 msgr2=0x7f8fc0160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.108+0000 7f8fc67ff700 1 --2- 172.21.15.111:0/4185415728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fc00cff50 0x7f8fc0160b90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009d90 tx=0x7f8fb800b330 comp rx=0 tx=0).stop 2023-03-29T10:57:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.108+0000 7f8fc67ff700 1 -- 172.21.15.111:0/4185415728 shutdown_connections 2023-03-29T10:57:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.108+0000 7f8fc67ff700 1 --2- 172.21.15.111:0/4185415728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fc00cff50 0x7f8fc0160b90 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.108+0000 7f8fc67ff700 1 -- 172.21.15.111:0/4185415728 >> 172.21.15.111:0/4185415728 conn(0x7f8fc0071b10 msgr2=0x7f8fc0071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.108+0000 7f8fc67ff700 1 -- 172.21.15.111:0/4185415728 shutdown_connections 2023-03-29T10:57:41.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.108+0000 7f8fc67ff700 1 -- 172.21.15.111:0/4185415728 wait complete. 2023-03-29T10:57:41.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.110+0000 7f8fc67ff700 1 Processor -- start 2023-03-29T10:57:41.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.110+0000 7f8fc67ff700 1 -- start start 2023-03-29T10:57:41.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.111+0000 7f8fc67ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fc00cf010 0x7f8fc01f5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:41.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.111+0000 7f8fc67ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fc00cff50 0x7f8fc00b89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.111+0000 7f8fc67ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc015c590 0x7f8fc01e6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.111+0000 7f8fc67ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00b8ee0 con 0x7f8fc00cf010 2023-03-29T10:57:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.111+0000 7f8fc67ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc01e7090 con 0x7f8fc015c590 2023-03-29T10:57:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.111+0000 7f8fc67ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc01e6e80 con 0x7f8fc00cff50 2023-03-29T10:57:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.111+0000 7f8fbffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fc00cf010 0x7f8fc01f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.111+0000 7f8fbffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fc00cf010 0x7f8fc01f5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58110/0 (socket says 172.21.15.111:58110) 2023-03-29T10:57:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.111+0000 7f8fbffff700 1 -- 172.21.15.111:0/1306390948 learned_addr learned my addr 172.21.15.111:0/1306390948 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.111+0000 7f8fbf7fe700 1 --2- 172.21.15.111:0/1306390948 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fc00cff50 0x7f8fc00b89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.111+0000 7f8fc4d9c700 1 --2- 172.21.15.111:0/1306390948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc015c590 0x7f8fc01e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.111+0000 7f8fbffff700 1 -- 172.21.15.111:0/1306390948 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fc00cff50 msgr2=0x7f8fc00b89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.111+0000 7f8fbffff700 1 --2- 172.21.15.111:0/1306390948 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fc00cff50 0x7f8fc00b89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.111+0000 7f8fbffff700 1 -- 172.21.15.111:0/1306390948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc015c590 msgr2=0x7f8fc01e6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.111+0000 7f8fbffff700 1 --2- 172.21.15.111:0/1306390948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc015c590 0x7f8fc01e6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.111+0000 7f8fbffff700 1 -- 172.21.15.111:0/1306390948 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb8009a70 con 0x7f8fc00cf010 2023-03-29T10:57:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.112+0000 7f8fbffff700 1 --2- 172.21.15.111:0/1306390948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fc00cf010 0x7f8fc01f5240 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000c480 tx=0x7f8fb000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.112+0000 7f8fbcff9700 1 -- 172.21.15.111:0/1306390948 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0005d70 con 0x7f8fc00cf010 2023-03-29T10:57:41.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.112+0000 7f8fbcff9700 1 -- 172.21.15.111:0/1306390948 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb000ce90 con 0x7f8fc00cf010 2023-03-29T10:57:41.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.112+0000 7f8fc67ff700 1 -- 172.21.15.111:0/1306390948 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc00cd770 con 0x7f8fc00cf010 2023-03-29T10:57:41.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.112+0000 7f8fbcff9700 1 -- 172.21.15.111:0/1306390948 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00072d0 con 0x7f8fc00cf010 2023-03-29T10:57:41.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.112+0000 7f8fc67ff700 1 -- 172.21.15.111:0/1306390948 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc00cf560 con 0x7f8fc00cf010 2023-03-29T10:57:41.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.113+0000 7f8fc67ff700 1 -- 172.21.15.111:0/1306390948 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc015c960 con 0x7f8fc00cf010 2023-03-29T10:57:41.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.114+0000 7f8fbcff9700 1 -- 172.21.15.111:0/1306390948 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f8fb0007430 con 0x7f8fc00cf010 2023-03-29T10:57:41.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.115+0000 7f8fbcff9700 1 --2- 172.21.15.111:0/1306390948 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8fac069de0 0x7f8fac06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:41.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.115+0000 7f8fbcff9700 1 -- 172.21.15.111:0/1306390948 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8fb00874e0 con 0x7f8fc00cf010 2023-03-29T10:57:41.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.115+0000 7f8fbf7fe700 1 --2- 172.21.15.111:0/1306390948 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8fac069de0 0x7f8fac06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:41.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.115+0000 7f8fbf7fe700 1 --2- 172.21.15.111:0/1306390948 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8fac069de0 0x7f8fac06c290 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00ccfa0 tx=0x7f8fb4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:41.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.119+0000 7f8fbcff9700 1 -- 172.21.15.111:0/1306390948 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f8fb004c160 con 0x7f8fc00cf010 2023-03-29T10:57:41.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.311+0000 7f8fc67ff700 1 -- 172.21.15.111:0/1306390948 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8fc0074430 con 0x7f8fc00cf010 2023-03-29T10:57:41.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.312+0000 7f8fbcff9700 1 -- 172.21.15.111:0/1306390948 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8fb0052ab0 con 0x7f8fc00cf010 2023-03-29T10:57:41.311 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:57:41.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.314+0000 7f8fc67ff700 1 -- 172.21.15.111:0/1306390948 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8fac069de0 msgr2=0x7f8fac06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:41.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.314+0000 7f8fc67ff700 1 --2- 172.21.15.111:0/1306390948 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8fac069de0 0x7f8fac06c290 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00ccfa0 tx=0x7f8fb4009040 comp rx=0 tx=0).stop 2023-03-29T10:57:41.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.314+0000 7f8fc67ff700 1 -- 172.21.15.111:0/1306390948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fc00cf010 msgr2=0x7f8fc01f5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:41.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.314+0000 7f8fc67ff700 1 --2- 172.21.15.111:0/1306390948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fc00cf010 0x7f8fc01f5240 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000c480 tx=0x7f8fb000c790 comp rx=0 tx=0).stop 2023-03-29T10:57:41.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.314+0000 7f8fc67ff700 1 -- 172.21.15.111:0/1306390948 shutdown_connections 2023-03-29T10:57:41.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.314+0000 7f8fc67ff700 1 --2- 172.21.15.111:0/1306390948 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8fac069de0 0x7f8fac06c290 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:41.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.314+0000 7f8fc67ff700 1 --2- 172.21.15.111:0/1306390948 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8fc00cf010 0x7f8fc01f5240 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:41.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.314+0000 7f8fc67ff700 1 --2- 172.21.15.111:0/1306390948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc015c590 0x7f8fc01e6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:41.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.314+0000 7f8fc67ff700 1 --2- 172.21.15.111:0/1306390948 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8fc00cff50 0x7f8fc00b89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:41.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.314+0000 7f8fc67ff700 1 -- 172.21.15.111:0/1306390948 >> 172.21.15.111:0/1306390948 conn(0x7f8fc0071b10 msgr2=0x7f8fc00c9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:41.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.314+0000 7f8fc67ff700 1 -- 172.21.15.111:0/1306390948 shutdown_connections 2023-03-29T10:57:41.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.314+0000 7f8fc67ff700 1 -- 172.21.15.111:0/1306390948 wait complete. 2023-03-29T10:57:41.332 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-03-29T10:57:41.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.515+0000 7f68399ff700 1 Processor -- start 2023-03-29T10:57:41.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.515+0000 7f68399ff700 1 -- start start 2023-03-29T10:57:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.516+0000 7f68399ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f68340b87a0 0x7f68340cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.516+0000 7f68399ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f68341601e0 0x7f6834160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.516+0000 7f68399ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68340cdc60 con 0x7f6834160b70 2023-03-29T10:57:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.516+0000 7f68399ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68340d02d0 con 0x7f68341601e0 2023-03-29T10:57:41.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.516+0000 7f68399ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68340d0cb0 con 0x7f68340b87a0 2023-03-29T10:57:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.517+0000 7f682bfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f68341601e0 0x7f6834160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.517+0000 7f68337fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f6834160b70 0x7f68340d2de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47460/0 (socket says 172.21.15.111:47460) 2023-03-29T10:57:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.517+0000 7f68337fe700 1 -- 172.21.15.111:0/706274942 learned_addr learned my addr 172.21.15.111:0/706274942 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.517+0000 7f6832ffd700 1 --2- 172.21.15.111:0/706274942 >> v2:172.21.15.119:3300/0 conn(0x7f68340b87a0 0x7f68340cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.517+0000 7f682bfff700 1 -- 172.21.15.111:0/706274942 >> v2:172.21.15.119:3300/0 conn(0x7f68340b87a0 msgr2=0x7f68340cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.517+0000 7f682bfff700 1 --2- 172.21.15.111:0/706274942 >> v2:172.21.15.119:3300/0 conn(0x7f68340b87a0 0x7f68340cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.517+0000 7f682bfff700 1 -- 172.21.15.111:0/706274942 >> v1:172.21.15.111:6789/0 conn(0x7f6834160b70 legacy=0x7f68340d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.517+0000 7f682bfff700 1 -- 172.21.15.111:0/706274942 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68340cd6a0 con 0x7f68341601e0 2023-03-29T10:57:41.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.517+0000 7f68327fc700 1 -- 172.21.15.111:0/706274942 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2508975956 0 0) 0x7f68340cdc60 con 0x7f6834160b70 2023-03-29T10:57:41.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.517+0000 7f682bfff700 1 --2- 172.21.15.111:0/706274942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68341601e0 0x7f6834160630 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f68340b4ac0 tx=0x7f681c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1ac3d0f1f4937 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:41.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.518+0000 7f68327fc700 1 -- 172.21.15.111:0/706274942 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c011830 con 0x7f68341601e0 2023-03-29T10:57:41.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.518+0000 7f68327fc700 1 -- 172.21.15.111:0/706274942 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c011990 con 0x7f68341601e0 2023-03-29T10:57:41.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.518+0000 7f68327fc700 1 -- 172.21.15.111:0/706274942 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c011d20 con 0x7f68341601e0 2023-03-29T10:57:41.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.518+0000 7f68399ff700 1 -- 172.21.15.111:0/706274942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68341601e0 msgr2=0x7f6834160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:41.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.518+0000 7f68399ff700 1 --2- 172.21.15.111:0/706274942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68341601e0 0x7f6834160630 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f68340b4ac0 tx=0x7f681c009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:41.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.518+0000 7f68399ff700 1 -- 172.21.15.111:0/706274942 shutdown_connections 2023-03-29T10:57:41.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.518+0000 7f68399ff700 1 --2- 172.21.15.111:0/706274942 >> v2:172.21.15.119:3300/0 conn(0x7f68340b87a0 0x7f68340cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:41.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.518+0000 7f68399ff700 1 --2- 172.21.15.111:0/706274942 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68341601e0 0x7f6834160630 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:41.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.518+0000 7f68399ff700 1 -- 172.21.15.111:0/706274942 >> 172.21.15.111:0/706274942 conn(0x7f6834071b10 msgr2=0x7f6834071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:41.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.518+0000 7f68399ff700 1 -- 172.21.15.111:0/706274942 shutdown_connections 2023-03-29T10:57:41.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.518+0000 7f68399ff700 1 -- 172.21.15.111:0/706274942 wait complete. 2023-03-29T10:57:41.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.521+0000 7f68399ff700 1 Processor -- start 2023-03-29T10:57:41.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.521+0000 7f68399ff700 1 -- start start 2023-03-29T10:57:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.522+0000 7f68399ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68340b87a0 0x7f683415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.522+0000 7f68399ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68341601e0 0x7f68341f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.522+0000 7f68399ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6834160b70 0x7f68340b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.522+0000 7f68399ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683415e280 con 0x7f68340b87a0 2023-03-29T10:57:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.522+0000 7f68399ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683415e560 con 0x7f68341601e0 2023-03-29T10:57:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.522+0000 7f68399ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683415e840 con 0x7f6834160b70 2023-03-29T10:57:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.522+0000 7f6832ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68340b87a0 0x7f683415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:41.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.522+0000 7f6832ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68340b87a0 0x7f683415d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58134/0 (socket says 172.21.15.111:58134) 2023-03-29T10:57:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.522+0000 7f6832ffd700 1 -- 172.21.15.111:0/4195679089 learned_addr learned my addr 172.21.15.111:0/4195679089 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.523+0000 7f682bfff700 1 --2- 172.21.15.111:0/4195679089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68341601e0 0x7f68341f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.523+0000 7f68337fe700 1 --2- 172.21.15.111:0/4195679089 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6834160b70 0x7f68340b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.523+0000 7f6832ffd700 1 -- 172.21.15.111:0/4195679089 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6834160b70 msgr2=0x7f68340b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.523+0000 7f6832ffd700 1 --2- 172.21.15.111:0/4195679089 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6834160b70 0x7f68340b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.523+0000 7f6832ffd700 1 -- 172.21.15.111:0/4195679089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68341601e0 msgr2=0x7f68341f1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.523+0000 7f6832ffd700 1 --2- 172.21.15.111:0/4195679089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68341601e0 0x7f68341f1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.523+0000 7f6832ffd700 1 -- 172.21.15.111:0/4195679089 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f681c0094a0 con 0x7f68340b87a0 2023-03-29T10:57:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.523+0000 7f682bfff700 1 --2- 172.21.15.111:0/4195679089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68341601e0 0x7f68341f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.523+0000 7f68337fe700 1 --2- 172.21.15.111:0/4195679089 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6834160b70 0x7f68340b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.523+0000 7f6832ffd700 1 --2- 172.21.15.111:0/4195679089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68340b87a0 0x7f683415d8f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f682400e9b0 tx=0x7f682400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.523+0000 7f6830ff9700 1 -- 172.21.15.111:0/4195679089 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6824009710 con 0x7f68340b87a0 2023-03-29T10:57:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.523+0000 7f6830ff9700 1 -- 172.21.15.111:0/4195679089 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682400c850 con 0x7f68340b87a0 2023-03-29T10:57:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.523+0000 7f68399ff700 1 -- 172.21.15.111:0/4195679089 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6834160f90 con 0x7f68340b87a0 2023-03-29T10:57:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.523+0000 7f6830ff9700 1 -- 172.21.15.111:0/4195679089 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682400cbc0 con 0x7f68340b87a0 2023-03-29T10:57:41.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.523+0000 7f68399ff700 1 -- 172.21.15.111:0/4195679089 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68340cd980 con 0x7f68340b87a0 2023-03-29T10:57:41.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.524+0000 7f68399ff700 1 -- 172.21.15.111:0/4195679089 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68340d2110 con 0x7f68340b87a0 2023-03-29T10:57:41.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.525+0000 7f6830ff9700 1 -- 172.21.15.111:0/4195679089 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f682400cd20 con 0x7f68340b87a0 2023-03-29T10:57:41.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.526+0000 7f6830ff9700 1 --2- 172.21.15.111:0/4195679089 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6814069de0 0x7f681406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:41.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.526+0000 7f6830ff9700 1 -- 172.21.15.111:0/4195679089 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6824088a50 con 0x7f68340b87a0 2023-03-29T10:57:41.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.526+0000 7f682bfff700 1 --2- 172.21.15.111:0/4195679089 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6814069de0 0x7f681406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:41.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.527+0000 7f682bfff700 1 --2- 172.21.15.111:0/4195679089 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6814069de0 0x7f681406c290 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f683415d890 tx=0x7f681c010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:41.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.530+0000 7f6830ff9700 1 -- 172.21.15.111:0/4195679089 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f6824057bb0 con 0x7f68340b87a0 2023-03-29T10:57:41.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.738+0000 7f68399ff700 1 -- 172.21.15.111:0/4195679089 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6834074430 con 0x7f68340b87a0 2023-03-29T10:57:41.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.739+0000 7f6830ff9700 1 -- 172.21.15.111:0/4195679089 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7f68240577a0 con 0x7f68340b87a0 2023-03-29T10:57:41.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.741+0000 7f68399ff700 1 -- 172.21.15.111:0/4195679089 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6814069de0 msgr2=0x7f681406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:41.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.741+0000 7f68399ff700 1 --2- 172.21.15.111:0/4195679089 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6814069de0 0x7f681406c290 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f683415d890 tx=0x7f681c010000 comp rx=0 tx=0).stop 2023-03-29T10:57:41.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.741+0000 7f68399ff700 1 -- 172.21.15.111:0/4195679089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68340b87a0 msgr2=0x7f683415d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:41.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.741+0000 7f68399ff700 1 --2- 172.21.15.111:0/4195679089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68340b87a0 0x7f683415d8f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f682400e9b0 tx=0x7f682400ed70 comp rx=0 tx=0).stop 2023-03-29T10:57:41.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.741+0000 7f68399ff700 1 -- 172.21.15.111:0/4195679089 shutdown_connections 2023-03-29T10:57:41.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.741+0000 7f68399ff700 1 --2- 172.21.15.111:0/4195679089 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6814069de0 0x7f681406c290 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:41.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.741+0000 7f68399ff700 1 --2- 172.21.15.111:0/4195679089 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68340b87a0 0x7f683415d8f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:41.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.741+0000 7f68399ff700 1 --2- 172.21.15.111:0/4195679089 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6834160b70 0x7f68340b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:41.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.741+0000 7f68399ff700 1 --2- 172.21.15.111:0/4195679089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68341601e0 0x7f68341f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:41.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.741+0000 7f68399ff700 1 -- 172.21.15.111:0/4195679089 >> 172.21.15.111:0/4195679089 conn(0x7f6834071b10 msgr2=0x7f683415f300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:41.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.741+0000 7f68399ff700 1 -- 172.21.15.111:0/4195679089 shutdown_connections 2023-03-29T10:57:41.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:41.741+0000 7f68399ff700 1 -- 172.21.15.111:0/4195679089 wait complete. 2023-03-29T10:57:41.742 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-03-29T10:57:41.757 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:57:41.758 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-29T10:57:41.798 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:57:42.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.020+0000 7fc5dabff700 1 Processor -- start 2023-03-29T10:57:42.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.020+0000 7fc5dabff700 1 -- start start 2023-03-29T10:57:42.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.021+0000 7fc5dabff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc5d40cfd40 0x7fc5d40d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:42.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.021+0000 7fc5dabff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc5d40b70f0 0x7fc5d40b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:42.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.021+0000 7fc5dabff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d40cdc10 con 0x7fc5d40cf970 2023-03-29T10:57:42.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.021+0000 7fc5dabff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d40b6c60 con 0x7fc5d40b70f0 2023-03-29T10:57:42.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.021+0000 7fc5dabff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d40cd650 con 0x7fc5d40cfd40 2023-03-29T10:57:42.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.021+0000 7fc5d899b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc5d40cf970 0x7fc5d40cf110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47474/0 (socket says 172.21.15.111:47474) 2023-03-29T10:57:42.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.021+0000 7fc5d899b700 1 -- 172.21.15.111:0/1915003971 learned_addr learned my addr 172.21.15.111:0/1915003971 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:42.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.021+0000 7fc5d919c700 1 --2- 172.21.15.111:0/1915003971 >> v2:172.21.15.111:3300/0 conn(0x7fc5d40b70f0 0x7fc5d40b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:42.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.022+0000 7fc5d3fff700 1 --2- 172.21.15.111:0/1915003971 >> v2:172.21.15.119:3300/0 conn(0x7fc5d40cfd40 0x7fc5d40d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:42.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.022+0000 7fc5d37fe700 1 -- 172.21.15.111:0/1915003971 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 190621230 0 0) 0x7fc5d40cdc10 con 0x7fc5d40cf970 2023-03-29T10:57:42.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.022+0000 7fc5d37fe700 1 -- 172.21.15.111:0/1915003971 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5c40035b0 con 0x7fc5d40cf970 2023-03-29T10:57:42.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.022+0000 7fc5d919c700 1 -- 172.21.15.111:0/1915003971 >> v2:172.21.15.119:3300/0 conn(0x7fc5d40cfd40 msgr2=0x7fc5d40d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:42.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.022+0000 7fc5d919c700 1 --2- 172.21.15.111:0/1915003971 >> v2:172.21.15.119:3300/0 conn(0x7fc5d40cfd40 0x7fc5d40d01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.022+0000 7fc5d919c700 1 -- 172.21.15.111:0/1915003971 >> v1:172.21.15.111:6789/0 conn(0x7fc5d40cf970 legacy=0x7fc5d40cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:42.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.022+0000 7fc5d919c700 1 -- 172.21.15.111:0/1915003971 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d40cd930 con 0x7fc5d40b70f0 2023-03-29T10:57:42.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.022+0000 7fc5d3fff700 1 --2- 172.21.15.111:0/1915003971 >> v2:172.21.15.119:3300/0 conn(0x7fc5d40cfd40 0x7fc5d40d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:57:42.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.022+0000 7fc5d919c700 1 --2- 172.21.15.111:0/1915003971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5d40b70f0 0x7fc5d40b74f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc5d40b8c40 tx=0x7fc5cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b5053609fe7873 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:42.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.022+0000 7fc5d37fe700 1 -- 172.21.15.111:0/1915003971 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc00e070 con 0x7fc5d40b70f0 2023-03-29T10:57:42.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.022+0000 7fc5d37fe700 1 -- 172.21.15.111:0/1915003971 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5cc00bb20 con 0x7fc5d40b70f0 2023-03-29T10:57:42.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.022+0000 7fc5d37fe700 1 -- 172.21.15.111:0/1915003971 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc012430 con 0x7fc5d40b70f0 2023-03-29T10:57:42.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.023+0000 7fc5dabff700 1 -- 172.21.15.111:0/1915003971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5d40b70f0 msgr2=0x7fc5d40b74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:42.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.023+0000 7fc5dabff700 1 --2- 172.21.15.111:0/1915003971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5d40b70f0 0x7fc5d40b74f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc5d40b8c40 tx=0x7fc5cc00b330 comp rx=0 tx=0).stop 2023-03-29T10:57:42.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.023+0000 7fc5dabff700 1 -- 172.21.15.111:0/1915003971 shutdown_connections 2023-03-29T10:57:42.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.023+0000 7fc5dabff700 1 --2- 172.21.15.111:0/1915003971 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5d40b70f0 0x7fc5d40b74f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.023+0000 7fc5dabff700 1 --2- 172.21.15.111:0/1915003971 >> v2:172.21.15.119:3300/0 conn(0x7fc5d40cfd40 0x7fc5d40d01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.023+0000 7fc5dabff700 1 -- 172.21.15.111:0/1915003971 >> 172.21.15.111:0/1915003971 conn(0x7fc5d4071b10 msgr2=0x7fc5d4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:42.024 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.023+0000 7fc5dabff700 1 -- 172.21.15.111:0/1915003971 shutdown_connections 2023-03-29T10:57:42.024 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.023+0000 7fc5dabff700 1 -- 172.21.15.111:0/1915003971 wait complete. 2023-03-29T10:57:42.024 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.025+0000 7fc5dabff700 1 Processor -- start 2023-03-29T10:57:42.024 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.025+0000 7fc5dabff700 1 -- start start 2023-03-29T10:57:42.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.026+0000 7fc5dabff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5d40b70f0 0x7fc5d40ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:42.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.026+0000 7fc5dabff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5d40cf970 0x7fc5d41493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:42.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.026+0000 7fc5dabff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5d40cfd40 0x7fc5d40c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:42.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.026+0000 7fc5dabff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d40c8fa0 con 0x7fc5d40cfd40 2023-03-29T10:57:42.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.026+0000 7fc5dabff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d4149930 con 0x7fc5d40cf970 2023-03-29T10:57:42.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.026+0000 7fc5dabff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d40cd2b0 con 0x7fc5d40b70f0 2023-03-29T10:57:42.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.026+0000 7fc5d3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5d40cf970 0x7fc5d41493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:42.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.026+0000 7fc5d899b700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5d40b70f0 0x7fc5d40ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:42.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.026+0000 7fc5d919c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5d40cfd40 0x7fc5d40c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:42.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.026+0000 7fc5d3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5d40cf970 0x7fc5d41493f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:44590/0 (socket says 172.21.15.111:44590) 2023-03-29T10:57:42.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.026+0000 7fc5d3fff700 1 -- 172.21.15.111:0/2174161 learned_addr learned my addr 172.21.15.111:0/2174161 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:42.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.026+0000 7fc5d919c700 1 -- 172.21.15.111:0/2174161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5d40b70f0 msgr2=0x7fc5d40ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:42.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.026+0000 7fc5d919c700 1 --2- 172.21.15.111:0/2174161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5d40b70f0 0x7fc5d40ca030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.026+0000 7fc5d919c700 1 -- 172.21.15.111:0/2174161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5d40cf970 msgr2=0x7fc5d41493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:42.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.026+0000 7fc5d919c700 1 --2- 172.21.15.111:0/2174161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5d40cf970 0x7fc5d41493f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.026+0000 7fc5d919c700 1 -- 172.21.15.111:0/2174161 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc009a70 con 0x7fc5d40cfd40 2023-03-29T10:57:42.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.027+0000 7fc5d919c700 1 --2- 172.21.15.111:0/2174161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5d40cfd40 0x7fc5d40c9430 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc016000 tx=0x7fc5cc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:42.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.027+0000 7fc5d17fa700 1 -- 172.21.15.111:0/2174161 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc012430 con 0x7fc5d40cfd40 2023-03-29T10:57:42.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.027+0000 7fc5d17fa700 1 -- 172.21.15.111:0/2174161 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5cc020070 con 0x7fc5d40cfd40 2023-03-29T10:57:42.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.027+0000 7fc5dabff700 1 -- 172.21.15.111:0/2174161 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d40d0700 con 0x7fc5d40cfd40 2023-03-29T10:57:42.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.027+0000 7fc5d17fa700 1 -- 172.21.15.111:0/2174161 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc0098f0 con 0x7fc5d40cfd40 2023-03-29T10:57:42.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.027+0000 7fc5dabff700 1 -- 172.21.15.111:0/2174161 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d40b68f0 con 0x7fc5d40cfd40 2023-03-29T10:57:42.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.028+0000 7fc5dabff700 1 -- 172.21.15.111:0/2174161 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d40d36d0 con 0x7fc5d40cfd40 2023-03-29T10:57:42.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.029+0000 7fc5d17fa700 1 -- 172.21.15.111:0/2174161 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fc5cc004040 con 0x7fc5d40cfd40 2023-03-29T10:57:42.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.030+0000 7fc5d17fa700 1 --2- 172.21.15.111:0/2174161 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc5b8069de0 0x7fc5b806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:42.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.030+0000 7fc5d17fa700 1 -- 172.21.15.111:0/2174161 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc5cc022070 con 0x7fc5d40cfd40 2023-03-29T10:57:42.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.030+0000 7fc5d899b700 1 --2- 172.21.15.111:0/2174161 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc5b8069de0 0x7fc5b806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:42.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.031+0000 7fc5d899b700 1 --2- 172.21.15.111:0/2174161 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc5b8069de0 0x7fc5b806c290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8007fd0 tx=0x7fc5c8007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:42.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.034+0000 7fc5d17fa700 1 -- 172.21.15.111:0/2174161 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fc5cc055520 con 0x7fc5d40cfd40 2023-03-29T10:57:42.230 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.231+0000 7fc5dabff700 1 -- 172.21.15.111:0/2174161 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5d4074430 con 0x7fc5d40cfd40 2023-03-29T10:57:42.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.232+0000 7fc5d17fa700 1 -- 172.21.15.111:0/2174161 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19796 (secure 0 0 0) 0x7fc5cc02c020 con 0x7fc5d40cfd40 2023-03-29T10:57:42.232 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:42.232 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":28,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:57:32.331880+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:57:42.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.234+0000 7fc5dabff700 1 -- 172.21.15.111:0/2174161 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc5b8069de0 msgr2=0x7fc5b806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:42.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.234+0000 7fc5dabff700 1 --2- 172.21.15.111:0/2174161 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc5b8069de0 0x7fc5b806c290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8007fd0 tx=0x7fc5c8007dc0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.234+0000 7fc5dabff700 1 -- 172.21.15.111:0/2174161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5d40cfd40 msgr2=0x7fc5d40c9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:42.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.234+0000 7fc5dabff700 1 --2- 172.21.15.111:0/2174161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5d40cfd40 0x7fc5d40c9430 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc016000 tx=0x7fc5cc004ad0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.234+0000 7fc5dabff700 1 -- 172.21.15.111:0/2174161 shutdown_connections 2023-03-29T10:57:42.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.234+0000 7fc5dabff700 1 --2- 172.21.15.111:0/2174161 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc5b8069de0 0x7fc5b806c290 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.234+0000 7fc5dabff700 1 --2- 172.21.15.111:0/2174161 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5d40cfd40 0x7fc5d40c9430 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.234+0000 7fc5dabff700 1 --2- 172.21.15.111:0/2174161 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5d40b70f0 0x7fc5d40ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.234+0000 7fc5dabff700 1 --2- 172.21.15.111:0/2174161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5d40cf970 0x7fc5d41493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.234+0000 7fc5dabff700 1 -- 172.21.15.111:0/2174161 >> 172.21.15.111:0/2174161 conn(0x7fc5d4071b10 msgr2=0x7fc5d40b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:42.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.234+0000 7fc5dabff700 1 -- 172.21.15.111:0/2174161 shutdown_connections 2023-03-29T10:57:42.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.234+0000 7fc5dabff700 1 -- 172.21.15.111:0/2174161 wait complete. 2023-03-29T10:57:42.251 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-29T10:57:16.183979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-29T10:57:21.239809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-29T10:57:26.192124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-29T10:57:26.796777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-29T10:57:29.874795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-29T10:57:42.252 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-29T10:57:42.438 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.439+0000 7f11819ff700 1 Processor -- start 2023-03-29T10:57:42.439 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.439+0000 7f11819ff700 1 -- start start 2023-03-29T10:57:42.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.440+0000 7f11819ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f117c0cd790 0x7f117c0ca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:42.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.440+0000 7f11819ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f117c0b1d50 0x7f117c06b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:42.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.440+0000 7f11819ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117c06a5f0 con 0x7f117c0cdb60 2023-03-29T10:57:42.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.440+0000 7f11819ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f117c06a840 con 0x7f117c0cd790 2023-03-29T10:57:42.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.440+0000 7f11819ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117c06ab20 con 0x7f117c0b1d50 2023-03-29T10:57:42.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.440+0000 7f117b7fe700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f117c0b1d50 0x7f117c06b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:42.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.440+0000 7f117affd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f117c0cd790 0x7f117c0ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:42.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.440+0000 7f1173fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f117c0cdb60 0x7f117c0b1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.111:51958/0 (socket says 172.21.15.111:51958) 2023-03-29T10:57:42.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.440+0000 7f1173fff700 1 -- 172.21.15.111:0/3682564909 learned_addr learned my addr 172.21.15.111:0/3682564909 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:42.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.441+0000 7f117b7fe700 1 -- 172.21.15.111:0/3682564909 >> v2:172.21.15.119:3300/0 conn(0x7f117c0cd790 msgr2=0x7f117c0ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:42.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.441+0000 7f117b7fe700 1 --2- 172.21.15.111:0/3682564909 >> v2:172.21.15.119:3300/0 conn(0x7f117c0cd790 0x7f117c0ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.441+0000 7f117a7fc700 1 -- 172.21.15.111:0/3682564909 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3814445950 0 0) 0x7f117c06a5f0 con 0x7f117c0cdb60 2023-03-29T10:57:42.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.441+0000 7f117b7fe700 1 -- 172.21.15.111:0/3682564909 >> v1:172.21.15.119:6789/0 conn(0x7f117c0cdb60 legacy=0x7f117c0b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:42.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.441+0000 7f117b7fe700 1 -- 172.21.15.111:0/3682564909 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c06ae00 con 0x7f117c0b1d50 2023-03-29T10:57:42.442 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.441+0000 7f117affd700 1 --2- 172.21.15.111:0/3682564909 >> v2:172.21.15.119:3300/0 conn(0x7f117c0cd790 0x7f117c0ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:42.442 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.441+0000 7f117b7fe700 1 --2- 172.21.15.111:0/3682564909 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f117c0b1d50 0x7f117c06b600 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1174009dc0 tx=0x7f117400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d75d1ed4e7c9d9e server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:42.442 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.441+0000 7f117a7fc700 1 -- 172.21.15.111:0/3682564909 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117400e070 con 0x7f117c0b1d50 2023-03-29T10:57:42.442 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.441+0000 7f117a7fc700 1 -- 172.21.15.111:0/3682564909 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117400bae0 con 0x7f117c0b1d50 2023-03-29T10:57:42.442 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.441+0000 7f117a7fc700 1 -- 172.21.15.111:0/3682564909 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117400be90 con 0x7f117c0b1d50 2023-03-29T10:57:42.442 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.442+0000 7f11819ff700 1 -- 172.21.15.111:0/3682564909 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f117c0b1d50 msgr2=0x7f117c06b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:42.442 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.442+0000 7f11819ff700 1 --2- 172.21.15.111:0/3682564909 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f117c0b1d50 0x7f117c06b600 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1174009dc0 tx=0x7f117400b330 comp rx=0 tx=0).stop 2023-03-29T10:57:42.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.442+0000 7f11819ff700 1 -- 172.21.15.111:0/3682564909 shutdown_connections 2023-03-29T10:57:42.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.442+0000 7f11819ff700 1 --2- 172.21.15.111:0/3682564909 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f117c0b1d50 0x7f117c06b600 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.442+0000 7f11819ff700 1 --2- 172.21.15.111:0/3682564909 >> v2:172.21.15.119:3300/0 conn(0x7f117c0cd790 0x7f117c0ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.442+0000 7f11819ff700 1 -- 172.21.15.111:0/3682564909 >> 172.21.15.111:0/3682564909 conn(0x7f117c06e980 msgr2=0x7f117c0cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:42.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.442+0000 7f11819ff700 1 -- 172.21.15.111:0/3682564909 shutdown_connections 2023-03-29T10:57:42.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.442+0000 7f11819ff700 1 -- 172.21.15.111:0/3682564909 wait complete. 2023-03-29T10:57:42.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.444+0000 7f11819ff700 1 Processor -- start 2023-03-29T10:57:42.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.444+0000 7f11819ff700 1 -- start start 2023-03-29T10:57:42.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.445+0000 7f11819ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f117c0b1d50 0x7f117c1525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:42.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.445+0000 7f11819ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f117c0cd790 0x7f117c0b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:42.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.445+0000 7f11819ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f117c0cdb60 0x7f117c151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:42.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.445+0000 7f11819ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c0b3f30 con 0x7f117c0b1d50 2023-03-29T10:57:42.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.445+0000 7f11819ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c151f40 con 0x7f117c0cdb60 2023-03-29T10:57:42.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.445+0000 7f11819ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117c0c81d0 con 0x7f117c0cd790 2023-03-29T10:57:42.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.445+0000 7f117affd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f117c0b1d50 0x7f117c1525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:42.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.445+0000 7f117affd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f117c0b1d50 0x7f117c1525d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58162/0 (socket says 172.21.15.111:58162) 2023-03-29T10:57:42.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.445+0000 7f117affd700 1 -- 172.21.15.111:0/1173915348 learned_addr learned my addr 172.21.15.111:0/1173915348 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:42.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.445+0000 7f1173fff700 1 --2- 172.21.15.111:0/1173915348 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f117c0cd790 0x7f117c0b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:42.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.445+0000 7f117b7fe700 1 --2- 172.21.15.111:0/1173915348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f117c0cdb60 0x7f117c151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:42.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.445+0000 7f117affd700 1 -- 172.21.15.111:0/1173915348 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f117c0cd790 msgr2=0x7f117c0b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:42.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.445+0000 7f117affd700 1 --2- 172.21.15.111:0/1173915348 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f117c0cd790 0x7f117c0b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.445+0000 7f117affd700 1 -- 172.21.15.111:0/1173915348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f117c0cdb60 msgr2=0x7f117c151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:42.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.445+0000 7f117affd700 1 --2- 172.21.15.111:0/1173915348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f117c0cdb60 0x7f117c151a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.446+0000 7f117affd700 1 -- 172.21.15.111:0/1173915348 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c0094a0 con 0x7f117c0b1d50 2023-03-29T10:57:42.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.446+0000 7f117affd700 1 --2- 172.21.15.111:0/1173915348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f117c0b1d50 0x7f117c1525d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f116c00ebe0 tx=0x7f116c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:42.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.446+0000 7f1178ff9700 1 -- 172.21.15.111:0/1173915348 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c009760 con 0x7f117c0b1d50 2023-03-29T10:57:42.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.446+0000 7f1178ff9700 1 -- 172.21.15.111:0/1173915348 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116c00c850 con 0x7f117c0b1d50 2023-03-29T10:57:42.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.446+0000 7f11819ff700 1 -- 172.21.15.111:0/1173915348 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1174009a70 con 0x7f117c0b1d50 2023-03-29T10:57:42.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.446+0000 7f1178ff9700 1 -- 172.21.15.111:0/1173915348 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c00cbe0 con 0x7f117c0b1d50 2023-03-29T10:57:42.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.446+0000 7f11819ff700 1 -- 172.21.15.111:0/1173915348 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117c0ce000 con 0x7f117c0b1d50 2023-03-29T10:57:42.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.447+0000 7f11819ff700 1 -- 172.21.15.111:0/1173915348 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f117c06b0e0 con 0x7f117c0b1d50 2023-03-29T10:57:42.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.448+0000 7f1178ff9700 1 -- 172.21.15.111:0/1173915348 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f116c00cd40 con 0x7f117c0b1d50 2023-03-29T10:57:42.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.449+0000 7f1178ff9700 1 --2- 172.21.15.111:0/1173915348 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f115c069de0 0x7f115c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:42.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.449+0000 7f1178ff9700 1 -- 172.21.15.111:0/1173915348 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f116c088250 con 0x7f117c0b1d50 2023-03-29T10:57:42.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.449+0000 7f1173fff700 1 --2- 172.21.15.111:0/1173915348 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f115c069de0 0x7f115c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:42.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.450+0000 7f1173fff700 1 --2- 172.21.15.111:0/1173915348 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f115c069de0 0x7f115c06c290 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f117c0cb940 tx=0x7f1164007e70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:42.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.453+0000 7f1178ff9700 1 -- 172.21.15.111:0/1173915348 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f116c057230 con 0x7f117c0b1d50 2023-03-29T10:57:42.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.651+0000 7f11819ff700 1 -- 172.21.15.111:0/1173915348 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f117c15ca20 con 0x7f117c0b1d50 2023-03-29T10:57:42.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.652+0000 7f1178ff9700 1 -- 172.21.15.111:0/1173915348 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f116c056e20 con 0x7f117c0b1d50 2023-03-29T10:57:42.652 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-03-29T10:57:42.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.654+0000 7f11819ff700 1 -- 172.21.15.111:0/1173915348 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f115c069de0 msgr2=0x7f115c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:42.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.654+0000 7f11819ff700 1 --2- 172.21.15.111:0/1173915348 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f115c069de0 0x7f115c06c290 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f117c0cb940 tx=0x7f1164007e70 comp rx=0 tx=0).stop 2023-03-29T10:57:42.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.654+0000 7f11819ff700 1 -- 172.21.15.111:0/1173915348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f117c0b1d50 msgr2=0x7f117c1525d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:42.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.654+0000 7f11819ff700 1 --2- 172.21.15.111:0/1173915348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f117c0b1d50 0x7f117c1525d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f116c00ebe0 tx=0x7f116c00eef0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.654+0000 7f11819ff700 1 -- 172.21.15.111:0/1173915348 shutdown_connections 2023-03-29T10:57:42.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.654+0000 7f11819ff700 1 --2- 172.21.15.111:0/1173915348 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f115c069de0 0x7f115c06c290 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.654+0000 7f11819ff700 1 --2- 172.21.15.111:0/1173915348 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f117c0b1d50 0x7f117c1525d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.654+0000 7f11819ff700 1 --2- 172.21.15.111:0/1173915348 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f117c0cdb60 0x7f117c151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.654+0000 7f11819ff700 1 --2- 172.21.15.111:0/1173915348 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f117c0cd790 0x7f117c0b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.654+0000 7f11819ff700 1 -- 172.21.15.111:0/1173915348 >> 172.21.15.111:0/1173915348 conn(0x7f117c06e980 msgr2=0x7f117c0cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:42.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.654+0000 7f11819ff700 1 -- 172.21.15.111:0/1173915348 shutdown_connections 2023-03-29T10:57:42.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.654+0000 7f11819ff700 1 -- 172.21.15.111:0/1173915348 wait complete. 2023-03-29T10:57:42.671 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-29T10:57:42.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.856+0000 7f490c3ff700 1 Processor -- start 2023-03-29T10:57:42.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.856+0000 7f490c3ff700 1 -- start start 2023-03-29T10:57:42.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.857+0000 7f490c3ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f49040b87a0 0x7f49040cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:42.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.857+0000 7f490c3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f49041601e0 0x7f4904160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:42.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.857+0000 7f490c3ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4904160ff0 con 0x7f4904160b70 2023-03-29T10:57:42.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.857+0000 7f490c3ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49040d02d0 con 0x7f49040b87a0 2023-03-29T10:57:42.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.857+0000 7f490c3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49040d0cb0 con 0x7f49041601e0 2023-03-29T10:57:42.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.858+0000 7f490a99c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f4904160b70 0x7f49040d2de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47480/0 (socket says 172.21.15.111:47480) 2023-03-29T10:57:42.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.858+0000 7f490a99c700 1 -- 172.21.15.111:0/1321184485 learned_addr learned my addr 172.21.15.111:0/1321184485 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:42.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.858+0000 7f490a19b700 1 --2- 172.21.15.111:0/1321184485 >> v2:172.21.15.119:3300/0 conn(0x7f49040b87a0 0x7f49040cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:42.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.858+0000 7f490999a700 1 --2- 172.21.15.111:0/1321184485 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f49041601e0 0x7f4904160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:42.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.858+0000 7f4909199700 1 -- 172.21.15.111:0/1321184485 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2698243350 0 0) 0x7f4904160ff0 con 0x7f4904160b70 2023-03-29T10:57:42.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.858+0000 7f4909199700 1 -- 172.21.15.111:0/1321184485 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48f00035b0 con 0x7f4904160b70 2023-03-29T10:57:42.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.858+0000 7f490a19b700 1 -- 172.21.15.111:0/1321184485 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f49041601e0 msgr2=0x7f4904160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:42.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.858+0000 7f490a19b700 1 --2- 172.21.15.111:0/1321184485 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f49041601e0 0x7f4904160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.858+0000 7f490a19b700 1 -- 172.21.15.111:0/1321184485 >> v1:172.21.15.111:6789/0 conn(0x7f4904160b70 legacy=0x7f49040d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:42.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.858+0000 7f490a19b700 1 -- 172.21.15.111:0/1321184485 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49040cdc60 con 0x7f49040b87a0 2023-03-29T10:57:42.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.858+0000 7f490999a700 1 --2- 172.21.15.111:0/1321184485 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f49041601e0 0x7f4904160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:42.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.859+0000 7f490a19b700 1 --2- 172.21.15.111:0/1321184485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49040b87a0 0x7f49040cfbf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4900009840 tx=0x7f4900009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed10fe7b392645dc server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:42.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.859+0000 7f4909199700 1 -- 172.21.15.111:0/1321184485 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900011830 con 0x7f49040b87a0 2023-03-29T10:57:42.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.859+0000 7f4909199700 1 -- 172.21.15.111:0/1321184485 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4900011990 con 0x7f49040b87a0 2023-03-29T10:57:42.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.859+0000 7f4909199700 1 -- 172.21.15.111:0/1321184485 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4900011d20 con 0x7f49040b87a0 2023-03-29T10:57:42.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.859+0000 7f490c3ff700 1 -- 172.21.15.111:0/1321184485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49040b87a0 msgr2=0x7f49040cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:42.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.859+0000 7f490c3ff700 1 --2- 172.21.15.111:0/1321184485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49040b87a0 0x7f49040cfbf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4900009840 tx=0x7f4900009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:42.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.859+0000 7f490c3ff700 1 -- 172.21.15.111:0/1321184485 shutdown_connections 2023-03-29T10:57:42.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.859+0000 7f490c3ff700 1 --2- 172.21.15.111:0/1321184485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49040b87a0 0x7f49040cfbf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.859+0000 7f490c3ff700 1 --2- 172.21.15.111:0/1321184485 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f49041601e0 0x7f4904160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.859+0000 7f490c3ff700 1 -- 172.21.15.111:0/1321184485 >> 172.21.15.111:0/1321184485 conn(0x7f4904071b10 msgr2=0x7f4904071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:42.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.859+0000 7f490c3ff700 1 -- 172.21.15.111:0/1321184485 shutdown_connections 2023-03-29T10:57:42.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.859+0000 7f490c3ff700 1 -- 172.21.15.111:0/1321184485 wait complete. 2023-03-29T10:57:42.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.861+0000 7f490c3ff700 1 Processor -- start 2023-03-29T10:57:42.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.861+0000 7f490c3ff700 1 -- start start 2023-03-29T10:57:42.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.862+0000 7f490c3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f49040b87a0 0x7f49041f2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:42.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.862+0000 7f490c3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49041601e0 0x7f49041f1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:42.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.862+0000 7f490c3ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4904160b70 0x7f490415ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:42.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.862+0000 7f490c3ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49040c7700 con 0x7f4904160b70 2023-03-29T10:57:42.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.862+0000 7f490c3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49041f2ec0 con 0x7f49041601e0 2023-03-29T10:57:42.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.862+0000 7f490c3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49040c6af0 con 0x7f49040b87a0 2023-03-29T10:57:42.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.862+0000 7f490a99c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4904160b70 0x7f490415ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:42.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.862+0000 7f490a19b700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f49040b87a0 0x7f49041f2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:42.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.862+0000 7f490a99c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4904160b70 0x7f490415ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58174/0 (socket says 172.21.15.111:58174) 2023-03-29T10:57:42.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.862+0000 7f490a99c700 1 -- 172.21.15.111:0/2005475093 learned_addr learned my addr 172.21.15.111:0/2005475093 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:42.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.863+0000 7f490999a700 1 --2- 172.21.15.111:0/2005475093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49041601e0 0x7f49041f1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:42.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.863+0000 7f490a99c700 1 -- 172.21.15.111:0/2005475093 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f49040b87a0 msgr2=0x7f49041f2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:42.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.863+0000 7f490a99c700 1 --2- 172.21.15.111:0/2005475093 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f49040b87a0 0x7f49041f2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.863+0000 7f490a99c700 1 -- 172.21.15.111:0/2005475093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49041601e0 msgr2=0x7f49041f1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:42.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.863+0000 7f490a99c700 1 --2- 172.21.15.111:0/2005475093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49041601e0 0x7f49041f1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:42.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.863+0000 7f490a99c700 1 -- 172.21.15.111:0/2005475093 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f40094a0 con 0x7f4904160b70 2023-03-29T10:57:42.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.863+0000 7f490a99c700 1 --2- 172.21.15.111:0/2005475093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4904160b70 0x7f490415ee70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00cad0 tx=0x7f48fc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:42.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.863+0000 7f48faffd700 1 -- 172.21.15.111:0/2005475093 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc00a9a0 con 0x7f4904160b70 2023-03-29T10:57:42.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.863+0000 7f48faffd700 1 -- 172.21.15.111:0/2005475093 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc00ab00 con 0x7f4904160b70 2023-03-29T10:57:42.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.863+0000 7f48faffd700 1 -- 172.21.15.111:0/2005475093 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc00ae90 con 0x7f4904160b70 2023-03-29T10:57:42.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.863+0000 7f490c3ff700 1 -- 172.21.15.111:0/2005475093 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49000094a0 con 0x7f4904160b70 2023-03-29T10:57:42.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.864+0000 7f490c3ff700 1 -- 172.21.15.111:0/2005475093 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49040cd980 con 0x7f4904160b70 2023-03-29T10:57:42.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.865+0000 7f490c3ff700 1 -- 172.21.15.111:0/2005475093 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49040cd6a0 con 0x7f4904160b70 2023-03-29T10:57:42.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.865+0000 7f48faffd700 1 -- 172.21.15.111:0/2005475093 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f48fc01b3b0 con 0x7f4904160b70 2023-03-29T10:57:42.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.866+0000 7f48faffd700 1 --2- 172.21.15.111:0/2005475093 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f48e8069de0 0x7f48e806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:42.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.866+0000 7f48faffd700 1 -- 172.21.15.111:0/2005475093 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f48fc087f20 con 0x7f4904160b70 2023-03-29T10:57:42.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.866+0000 7f490a19b700 1 --2- 172.21.15.111:0/2005475093 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f48e8069de0 0x7f48e806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:42.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.867+0000 7f490a19b700 1 --2- 172.21.15.111:0/2005475093 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f48e8069de0 0x7f48e806c290 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4900000c00 tx=0x7f4900008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:42.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:42.870+0000 7f48faffd700 1 -- 172.21.15.111:0/2005475093 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f48fc053470 con 0x7f4904160b70 2023-03-29T10:57:43.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.061+0000 7f490c3ff700 1 -- 172.21.15.111:0/2005475093 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f49040d2110 con 0x7f4904160b70 2023-03-29T10:57:43.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.062+0000 7f48faffd700 1 -- 172.21.15.111:0/2005475093 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7f48fc0170a0 con 0x7f4904160b70 2023-03-29T10:57:43.062 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 8 2023-03-29T10:57:43.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.064+0000 7f490c3ff700 1 -- 172.21.15.111:0/2005475093 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f48e8069de0 msgr2=0x7f48e806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:43.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.064+0000 7f490c3ff700 1 --2- 172.21.15.111:0/2005475093 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f48e8069de0 0x7f48e806c290 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4900000c00 tx=0x7f4900008030 comp rx=0 tx=0).stop 2023-03-29T10:57:43.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.064+0000 7f490c3ff700 1 -- 172.21.15.111:0/2005475093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4904160b70 msgr2=0x7f490415ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:43.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.064+0000 7f490c3ff700 1 --2- 172.21.15.111:0/2005475093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4904160b70 0x7f490415ee70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00cad0 tx=0x7f48fc00cde0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.064+0000 7f490c3ff700 1 -- 172.21.15.111:0/2005475093 shutdown_connections 2023-03-29T10:57:43.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.064+0000 7f490c3ff700 1 --2- 172.21.15.111:0/2005475093 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f48e8069de0 0x7f48e806c290 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.064+0000 7f490c3ff700 1 --2- 172.21.15.111:0/2005475093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4904160b70 0x7f490415ee70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.064+0000 7f490c3ff700 1 --2- 172.21.15.111:0/2005475093 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f49040b87a0 0x7f49041f2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.064+0000 7f490c3ff700 1 --2- 172.21.15.111:0/2005475093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49041601e0 0x7f49041f1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.064+0000 7f490c3ff700 1 -- 172.21.15.111:0/2005475093 >> 172.21.15.111:0/2005475093 conn(0x7f4904071b10 msgr2=0x7f49040b6360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:43.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.064+0000 7f490c3ff700 1 -- 172.21.15.111:0/2005475093 shutdown_connections 2023-03-29T10:57:43.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.064+0000 7f490c3ff700 1 -- 172.21.15.111:0/2005475093 wait complete. 2023-03-29T10:57:43.081 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-29T10:57:43.266 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.266+0000 7fda0d9ff700 1 Processor -- start 2023-03-29T10:57:43.266 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.266+0000 7fda0d9ff700 1 -- start start 2023-03-29T10:57:43.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.268+0000 7fda0d9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fda080b87a0 0x7fda080cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:43.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.268+0000 7fda0d9ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda081601e0 0x7fda08160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:43.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.268+0000 7fda0d9ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda08160ff0 con 0x7fda08160b70 2023-03-29T10:57:43.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.268+0000 7fda0d9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda080d02d0 con 0x7fda080b87a0 2023-03-29T10:57:43.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.268+0000 7fda0d9ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda080d0cb0 con 0x7fda081601e0 2023-03-29T10:57:43.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.268+0000 7fda077fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fda08160b70 0x7fda080d2de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47496/0 (socket says 172.21.15.111:47496) 2023-03-29T10:57:43.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.268+0000 7fda077fe700 1 -- 172.21.15.111:0/705675532 learned_addr learned my addr 172.21.15.111:0/705675532 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:43.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.268+0000 7fd9fffff700 1 --2- 172.21.15.111:0/705675532 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda081601e0 0x7fda08160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:43.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.268+0000 7fda06ffd700 1 --2- 172.21.15.111:0/705675532 >> v2:172.21.15.119:3300/0 conn(0x7fda080b87a0 0x7fda080cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:43.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.268+0000 7fda067fc700 1 -- 172.21.15.111:0/705675532 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803252016 0 0) 0x7fda08160ff0 con 0x7fda08160b70 2023-03-29T10:57:43.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.268+0000 7fda067fc700 1 -- 172.21.15.111:0/705675532 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9f40035b0 con 0x7fda08160b70 2023-03-29T10:57:43.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.268+0000 7fd9fffff700 1 -- 172.21.15.111:0/705675532 >> v2:172.21.15.119:3300/0 conn(0x7fda080b87a0 msgr2=0x7fda080cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:43.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.268+0000 7fd9fffff700 1 --2- 172.21.15.111:0/705675532 >> v2:172.21.15.119:3300/0 conn(0x7fda080b87a0 0x7fda080cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.268+0000 7fd9fffff700 1 -- 172.21.15.111:0/705675532 >> v1:172.21.15.111:6789/0 conn(0x7fda08160b70 legacy=0x7fda080d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:43.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.268+0000 7fd9fffff700 1 -- 172.21.15.111:0/705675532 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda080cdc60 con 0x7fda081601e0 2023-03-29T10:57:43.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.269+0000 7fd9fffff700 1 --2- 172.21.15.111:0/705675532 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda081601e0 0x7fda08160630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009840 tx=0x7fd9f0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=393836484e0addf server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:43.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.269+0000 7fda067fc700 1 -- 172.21.15.111:0/705675532 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0011830 con 0x7fda081601e0 2023-03-29T10:57:43.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.269+0000 7fda067fc700 1 -- 172.21.15.111:0/705675532 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f0011990 con 0x7fda081601e0 2023-03-29T10:57:43.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.269+0000 7fda067fc700 1 -- 172.21.15.111:0/705675532 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0011d20 con 0x7fda081601e0 2023-03-29T10:57:43.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.269+0000 7fda0d9ff700 1 -- 172.21.15.111:0/705675532 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda081601e0 msgr2=0x7fda08160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:43.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.269+0000 7fda0d9ff700 1 --2- 172.21.15.111:0/705675532 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda081601e0 0x7fda08160630 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0009840 tx=0x7fd9f0009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:43.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.270+0000 7fda0d9ff700 1 -- 172.21.15.111:0/705675532 shutdown_connections 2023-03-29T10:57:43.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.270+0000 7fda0d9ff700 1 --2- 172.21.15.111:0/705675532 >> v2:172.21.15.119:3300/0 conn(0x7fda080b87a0 0x7fda080cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.270+0000 7fda0d9ff700 1 --2- 172.21.15.111:0/705675532 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda081601e0 0x7fda08160630 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.270+0000 7fda0d9ff700 1 -- 172.21.15.111:0/705675532 >> 172.21.15.111:0/705675532 conn(0x7fda08071b10 msgr2=0x7fda08071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:43.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.270+0000 7fda0d9ff700 1 -- 172.21.15.111:0/705675532 shutdown_connections 2023-03-29T10:57:43.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.270+0000 7fda0d9ff700 1 -- 172.21.15.111:0/705675532 wait complete. 2023-03-29T10:57:43.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.271+0000 7fda0d9ff700 1 Processor -- start 2023-03-29T10:57:43.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.272+0000 7fda0d9ff700 1 -- start start 2023-03-29T10:57:43.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.272+0000 7fda0d9ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda080b87a0 0x7fda080cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:43.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.272+0000 7fda0d9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda081601e0 0x7fda0815ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:43.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.272+0000 7fda0d9ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda08160b70 0x7fda080d2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:43.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.272+0000 7fda0d9ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda080c6ff0 con 0x7fda08160b70 2023-03-29T10:57:43.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.272+0000 7fda0d9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda080ca410 con 0x7fda081601e0 2023-03-29T10:57:43.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.272+0000 7fda0d9ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda080ca6f0 con 0x7fda080b87a0 2023-03-29T10:57:43.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.273+0000 7fda077fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda08160b70 0x7fda080d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:43.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.273+0000 7fda06ffd700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda080b87a0 0x7fda080cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:43.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.273+0000 7fda077fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda08160b70 0x7fda080d2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58182/0 (socket says 172.21.15.111:58182) 2023-03-29T10:57:43.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.273+0000 7fda077fe700 1 -- 172.21.15.111:0/308243300 learned_addr learned my addr 172.21.15.111:0/308243300 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:43.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.273+0000 7fd9fffff700 1 --2- 172.21.15.111:0/308243300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda081601e0 0x7fda0815ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:43.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.273+0000 7fda077fe700 1 -- 172.21.15.111:0/308243300 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda080b87a0 msgr2=0x7fda080cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:43.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.273+0000 7fda077fe700 1 --2- 172.21.15.111:0/308243300 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda080b87a0 0x7fda080cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.273+0000 7fda077fe700 1 -- 172.21.15.111:0/308243300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda081601e0 msgr2=0x7fda0815ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:43.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.273+0000 7fda077fe700 1 --2- 172.21.15.111:0/308243300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda081601e0 0x7fda0815ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.273+0000 7fda077fe700 1 -- 172.21.15.111:0/308243300 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f00094a0 con 0x7fda08160b70 2023-03-29T10:57:43.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.273+0000 7fda077fe700 1 --2- 172.21.15.111:0/308243300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda08160b70 0x7fda080d2550 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fda0000caa0 tx=0x7fda0000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:43.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.274+0000 7fda04ff9700 1 -- 172.21.15.111:0/308243300 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0000a9e0 con 0x7fda08160b70 2023-03-29T10:57:43.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.274+0000 7fda04ff9700 1 -- 172.21.15.111:0/308243300 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda0000ab40 con 0x7fda08160b70 2023-03-29T10:57:43.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.274+0000 7fda0d9ff700 1 -- 172.21.15.111:0/308243300 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda080cd980 con 0x7fda08160b70 2023-03-29T10:57:43.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.274+0000 7fda04ff9700 1 -- 172.21.15.111:0/308243300 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00012400 con 0x7fda08160b70 2023-03-29T10:57:43.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.274+0000 7fda0d9ff700 1 -- 172.21.15.111:0/308243300 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda080cd6a0 con 0x7fda08160b70 2023-03-29T10:57:43.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.275+0000 7fda0d9ff700 1 -- 172.21.15.111:0/308243300 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda080d2110 con 0x7fda08160b70 2023-03-29T10:57:43.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.276+0000 7fda04ff9700 1 -- 172.21.15.111:0/308243300 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fda0001b3b0 con 0x7fda08160b70 2023-03-29T10:57:43.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.276+0000 7fda04ff9700 1 --2- 172.21.15.111:0/308243300 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd9e8069de0 0x7fd9e806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:43.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.277+0000 7fda04ff9700 1 -- 172.21.15.111:0/308243300 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fda00087fb0 con 0x7fda08160b70 2023-03-29T10:57:43.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.277+0000 7fda06ffd700 1 --2- 172.21.15.111:0/308243300 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd9e8069de0 0x7fd9e806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:43.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.277+0000 7fda06ffd700 1 --2- 172.21.15.111:0/308243300 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd9e8069de0 0x7fd9e806c290 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009be0 tx=0x7fd9f8009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:43.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.281+0000 7fda04ff9700 1 -- 172.21.15.111:0/308243300 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fda00053530 con 0x7fda08160b70 2023-03-29T10:57:43.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.476+0000 7fda0d9ff700 1 -- 172.21.15.111:0/308243300 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fda080c7130 con 0x7fda08160b70 2023-03-29T10:57:43.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.477+0000 7fda04ff9700 1 -- 172.21.15.111:0/308243300 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7fda00017020 con 0x7fda08160b70 2023-03-29T10:57:43.477 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:57:43.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.479+0000 7fda0d9ff700 1 -- 172.21.15.111:0/308243300 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd9e8069de0 msgr2=0x7fd9e806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:43.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.479+0000 7fda0d9ff700 1 --2- 172.21.15.111:0/308243300 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd9e8069de0 0x7fd9e806c290 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009be0 tx=0x7fd9f8009210 comp rx=0 tx=0).stop 2023-03-29T10:57:43.479 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.479+0000 7fda0d9ff700 1 -- 172.21.15.111:0/308243300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda08160b70 msgr2=0x7fda080d2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:43.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.479+0000 7fda0d9ff700 1 --2- 172.21.15.111:0/308243300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda08160b70 0x7fda080d2550 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fda0000caa0 tx=0x7fda0000cdb0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.479+0000 7fda0d9ff700 1 -- 172.21.15.111:0/308243300 shutdown_connections 2023-03-29T10:57:43.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.479+0000 7fda0d9ff700 1 --2- 172.21.15.111:0/308243300 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd9e8069de0 0x7fd9e806c290 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.479+0000 7fda0d9ff700 1 --2- 172.21.15.111:0/308243300 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda08160b70 0x7fda080d2550 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.479+0000 7fda0d9ff700 1 --2- 172.21.15.111:0/308243300 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda080b87a0 0x7fda080cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.479+0000 7fda0d9ff700 1 --2- 172.21.15.111:0/308243300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda081601e0 0x7fda0815ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.480+0000 7fda0d9ff700 1 -- 172.21.15.111:0/308243300 >> 172.21.15.111:0/308243300 conn(0x7fda08071b10 msgr2=0x7fda080b6860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:43.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.480+0000 7fda0d9ff700 1 -- 172.21.15.111:0/308243300 shutdown_connections 2023-03-29T10:57:43.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.480+0000 7fda0d9ff700 1 -- 172.21.15.111:0/308243300 wait complete. 2023-03-29T10:57:43.496 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-29T10:57:43.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.684+0000 7f97ba5ff700 1 Processor -- start 2023-03-29T10:57:43.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.684+0000 7f97ba5ff700 1 -- start start 2023-03-29T10:57:43.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.686+0000 7f97ba5ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f97b40cfbf0 0x7f97b41608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:43.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.686+0000 7f97ba5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f97b40d1100 0x7f97b40d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:43.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.686+0000 7f97ba5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b40d0020 con 0x7f97b40b87a0 2023-03-29T10:57:43.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.686+0000 7f97ba5ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b40d02d0 con 0x7f97b40cfbf0 2023-03-29T10:57:43.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.686+0000 7f97ba5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b40d0cb0 con 0x7f97b40d1100 2023-03-29T10:57:43.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.686+0000 7f97abfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f97b40cfbf0 0x7f97b41608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:43.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.686+0000 7f97abfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f97b40cfbf0 0x7f97b41608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58192/0 (socket says 172.21.15.111:58192) 2023-03-29T10:57:43.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.686+0000 7f97abfff700 1 -- 172.21.15.111:0/1325208149 learned_addr learned my addr 172.21.15.111:0/1325208149 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:43.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.686+0000 7f97b8b9c700 1 --2- 172.21.15.111:0/1325208149 >> v2:172.21.15.119:3300/0 conn(0x7f97b40d1100 0x7f97b40d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:43.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.686+0000 7f97abfff700 1 -- 172.21.15.111:0/1325208149 >> v2:172.21.15.119:3300/0 conn(0x7f97b40d1100 msgr2=0x7f97b40d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:43.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.686+0000 7f97abfff700 1 --2- 172.21.15.111:0/1325208149 >> v2:172.21.15.119:3300/0 conn(0x7f97b40d1100 0x7f97b40d1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.686+0000 7f97abfff700 1 -- 172.21.15.111:0/1325208149 >> v1:172.21.15.119:6789/0 conn(0x7f97b40b87a0 legacy=0x7f97b41601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:43.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.686+0000 7f97abfff700 1 -- 172.21.15.111:0/1325208149 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b40cdc60 con 0x7f97b40cfbf0 2023-03-29T10:57:43.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.687+0000 7f97abfff700 1 --2- 172.21.15.111:0/1325208149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97b40cfbf0 0x7f97b41608f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f97b40d0820 tx=0x7f979c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab870bbfb0116fe server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:43.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.687+0000 7f97b37fe700 1 -- 172.21.15.111:0/1325208149 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c00b860 con 0x7f97b40cfbf0 2023-03-29T10:57:43.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.687+0000 7f97b37fe700 1 -- 172.21.15.111:0/1325208149 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c00b9c0 con 0x7f97b40cfbf0 2023-03-29T10:57:43.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.687+0000 7f97b37fe700 1 -- 172.21.15.111:0/1325208149 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c00bd50 con 0x7f97b40cfbf0 2023-03-29T10:57:43.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.687+0000 7f97ba5ff700 1 -- 172.21.15.111:0/1325208149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97b40cfbf0 msgr2=0x7f97b41608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:43.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.687+0000 7f97ba5ff700 1 --2- 172.21.15.111:0/1325208149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97b40cfbf0 0x7f97b41608f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f97b40d0820 tx=0x7f979c009c00 comp rx=0 tx=0).stop 2023-03-29T10:57:43.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.687+0000 7f97ba5ff700 1 -- 172.21.15.111:0/1325208149 shutdown_connections 2023-03-29T10:57:43.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.687+0000 7f97ba5ff700 1 --2- 172.21.15.111:0/1325208149 >> v2:172.21.15.119:3300/0 conn(0x7f97b40d1100 0x7f97b40d1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.687+0000 7f97ba5ff700 1 --2- 172.21.15.111:0/1325208149 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97b40cfbf0 0x7f97b41608f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.687+0000 7f97ba5ff700 1 -- 172.21.15.111:0/1325208149 >> 172.21.15.111:0/1325208149 conn(0x7f97b4071b10 msgr2=0x7f97b4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:43.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.688+0000 7f97ba5ff700 1 -- 172.21.15.111:0/1325208149 shutdown_connections 2023-03-29T10:57:43.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.688+0000 7f97ba5ff700 1 -- 172.21.15.111:0/1325208149 wait complete. 2023-03-29T10:57:43.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.690+0000 7f97ba5ff700 1 Processor -- start 2023-03-29T10:57:43.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.690+0000 7f97ba5ff700 1 -- start start 2023-03-29T10:57:43.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.691+0000 7f97ba5ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97b40b87a0 0x7f97b4159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:43.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.691+0000 7f97ba5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97b40cfbf0 0x7f97b41e89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:43.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.691+0000 7f97ba5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97b40d1100 0x7f97b41596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:43.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.691+0000 7f97ba5ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b4159bf0 con 0x7f97b40b87a0 2023-03-29T10:57:43.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.691+0000 7f97ba5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b41e8ff0 con 0x7f97b40d1100 2023-03-29T10:57:43.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.691+0000 7f97ba5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b415dd90 con 0x7f97b40cfbf0 2023-03-29T10:57:43.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.691+0000 7f97b3fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97b40b87a0 0x7f97b4159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:43.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.691+0000 7f97b3fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97b40b87a0 0x7f97b4159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58196/0 (socket says 172.21.15.111:58196) 2023-03-29T10:57:43.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.691+0000 7f97b3fff700 1 -- 172.21.15.111:0/3913046121 learned_addr learned my addr 172.21.15.111:0/3913046121 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:43.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.692+0000 7f97abfff700 1 --2- 172.21.15.111:0/3913046121 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97b40cfbf0 0x7f97b41e89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:43.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.692+0000 7f97b8b9c700 1 --2- 172.21.15.111:0/3913046121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97b40d1100 0x7f97b41596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:43.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.692+0000 7f97b3fff700 1 -- 172.21.15.111:0/3913046121 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97b40cfbf0 msgr2=0x7f97b41e89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:43.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.692+0000 7f97b3fff700 1 --2- 172.21.15.111:0/3913046121 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97b40cfbf0 0x7f97b41e89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.692+0000 7f97b3fff700 1 -- 172.21.15.111:0/3913046121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97b40d1100 msgr2=0x7f97b41596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:43.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.692+0000 7f97b3fff700 1 --2- 172.21.15.111:0/3913046121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97b40d1100 0x7f97b41596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.692+0000 7f97b3fff700 1 -- 172.21.15.111:0/3913046121 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f979c009550 con 0x7f97b40b87a0 2023-03-29T10:57:43.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.692+0000 7f97b8b9c700 1 --2- 172.21.15.111:0/3913046121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97b40d1100 0x7f97b41596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:57:43.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.692+0000 7f97b3fff700 1 --2- 172.21.15.111:0/3913046121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97b40b87a0 0x7f97b4159d60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f97a400c450 tx=0x7f97a400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:43.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.692+0000 7f97b17fa700 1 -- 172.21.15.111:0/3913046121 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a40052d0 con 0x7f97b40b87a0 2023-03-29T10:57:43.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.692+0000 7f97b17fa700 1 -- 172.21.15.111:0/3913046121 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a4005430 con 0x7f97b40b87a0 2023-03-29T10:57:43.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.692+0000 7f97ba5ff700 1 -- 172.21.15.111:0/3913046121 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b40cd980 con 0x7f97b40b87a0 2023-03-29T10:57:43.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.692+0000 7f97b17fa700 1 -- 172.21.15.111:0/3913046121 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a40038c0 con 0x7f97b40b87a0 2023-03-29T10:57:43.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.693+0000 7f97ba5ff700 1 -- 172.21.15.111:0/3913046121 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b40cd6a0 con 0x7f97b40b87a0 2023-03-29T10:57:43.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.694+0000 7f97ba5ff700 1 -- 172.21.15.111:0/3913046121 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b4160e30 con 0x7f97b40b87a0 2023-03-29T10:57:43.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.694+0000 7f97b17fa700 1 -- 172.21.15.111:0/3913046121 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f97a4003a20 con 0x7f97b40b87a0 2023-03-29T10:57:43.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.695+0000 7f97b17fa700 1 --2- 172.21.15.111:0/3913046121 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9794069de0 0x7f979406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:43.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.695+0000 7f97b17fa700 1 -- 172.21.15.111:0/3913046121 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f97a40876e0 con 0x7f97b40b87a0 2023-03-29T10:57:43.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.696+0000 7f97abfff700 1 --2- 172.21.15.111:0/3913046121 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9794069de0 0x7f979406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:43.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.696+0000 7f97abfff700 1 --2- 172.21.15.111:0/3913046121 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9794069de0 0x7f979406c290 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f979c000c00 tx=0x7f979c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:43.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.699+0000 7f97b17fa700 1 -- 172.21.15.111:0/3913046121 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f97a4052c00 con 0x7f97b40b87a0 2023-03-29T10:57:43.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.898+0000 7f97ba5ff700 1 -- 172.21.15.111:0/3913046121 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97b4074430 con 0x7f97b40b87a0 2023-03-29T10:57:43.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.898+0000 7f97b17fa700 1 -- 172.21.15.111:0/3913046121 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7f97a4056220 con 0x7f97b40b87a0 2023-03-29T10:57:43.898 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-03-29T10:57:43.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.900+0000 7f97ba5ff700 1 -- 172.21.15.111:0/3913046121 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9794069de0 msgr2=0x7f979406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:43.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.900+0000 7f97ba5ff700 1 --2- 172.21.15.111:0/3913046121 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9794069de0 0x7f979406c290 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f979c000c00 tx=0x7f979c008030 comp rx=0 tx=0).stop 2023-03-29T10:57:43.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.900+0000 7f97ba5ff700 1 -- 172.21.15.111:0/3913046121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97b40b87a0 msgr2=0x7f97b4159d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:43.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.900+0000 7f97ba5ff700 1 --2- 172.21.15.111:0/3913046121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97b40b87a0 0x7f97b4159d60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f97a400c450 tx=0x7f97a400c760 comp rx=0 tx=0).stop 2023-03-29T10:57:43.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.901+0000 7f97ba5ff700 1 -- 172.21.15.111:0/3913046121 shutdown_connections 2023-03-29T10:57:43.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.901+0000 7f97ba5ff700 1 --2- 172.21.15.111:0/3913046121 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9794069de0 0x7f979406c290 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.901+0000 7f97ba5ff700 1 --2- 172.21.15.111:0/3913046121 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97b40b87a0 0x7f97b4159d60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.901+0000 7f97ba5ff700 1 --2- 172.21.15.111:0/3913046121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97b40d1100 0x7f97b41596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.901+0000 7f97ba5ff700 1 --2- 172.21.15.111:0/3913046121 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97b40cfbf0 0x7f97b41e89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:43.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.901+0000 7f97ba5ff700 1 -- 172.21.15.111:0/3913046121 >> 172.21.15.111:0/3913046121 conn(0x7f97b4071b10 msgr2=0x7f97b40cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:43.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.901+0000 7f97ba5ff700 1 -- 172.21.15.111:0/3913046121 shutdown_connections 2023-03-29T10:57:43.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:43.901+0000 7f97ba5ff700 1 -- 172.21.15.111:0/3913046121 wait complete. 2023-03-29T10:57:43.917 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-29T10:57:44.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.102+0000 7f9cbafff700 1 Processor -- start 2023-03-29T10:57:44.102 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.102+0000 7f9cbafff700 1 -- start start 2023-03-29T10:57:44.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.103+0000 7f9cbafff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9cb40b87a0 0x7f9cb40cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:44.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.103+0000 7f9cbafff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9cb41601e0 0x7f9cb4160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:44.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.103+0000 7f9cbafff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cb4160ff0 con 0x7f9cb4160b70 2023-03-29T10:57:44.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.103+0000 7f9cbafff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cb40d02d0 con 0x7f9cb40b87a0 2023-03-29T10:57:44.103 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.103+0000 7f9cbafff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb40d0cb0 con 0x7f9cb41601e0 2023-03-29T10:57:44.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.104+0000 7f9cb8d9b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9cb40b87a0 0x7f9cb40cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:44.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.104+0000 7f9cb959c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f9cb4160b70 0x7f9cb40d2de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47510/0 (socket says 172.21.15.111:47510) 2023-03-29T10:57:44.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.104+0000 7f9cb959c700 1 -- 172.21.15.111:0/1378661725 learned_addr learned my addr 172.21.15.111:0/1378661725 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:44.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.104+0000 7f9cb3fff700 1 --2- 172.21.15.111:0/1378661725 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9cb41601e0 0x7f9cb4160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:44.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.104+0000 7f9cb8d9b700 1 -- 172.21.15.111:0/1378661725 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9cb41601e0 msgr2=0x7f9cb4160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:44.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.104+0000 7f9cb8d9b700 1 --2- 172.21.15.111:0/1378661725 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9cb41601e0 0x7f9cb4160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:44.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.104+0000 7f9cb8d9b700 1 -- 172.21.15.111:0/1378661725 >> v1:172.21.15.111:6789/0 conn(0x7f9cb4160b70 legacy=0x7f9cb40d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:44.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.104+0000 7f9cb8d9b700 1 -- 172.21.15.111:0/1378661725 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb40cdc60 con 0x7f9cb40b87a0 2023-03-29T10:57:44.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.104+0000 7f9cb37fe700 1 -- 172.21.15.111:0/1378661725 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3351032972 0 0) 0x7f9cb4160ff0 con 0x7f9cb4160b70 2023-03-29T10:57:44.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.104+0000 7f9cb8d9b700 1 --2- 172.21.15.111:0/1378661725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cb40b87a0 0x7f9cb40cfbf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009840 tx=0x7f9ca8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1ceda98bfc7bc6 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:44.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.104+0000 7f9cb37fe700 1 -- 172.21.15.111:0/1378661725 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8011830 con 0x7f9cb40b87a0 2023-03-29T10:57:44.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.105+0000 7f9cb37fe700 1 -- 172.21.15.111:0/1378661725 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca8011990 con 0x7f9cb40b87a0 2023-03-29T10:57:44.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.105+0000 7f9cb37fe700 1 -- 172.21.15.111:0/1378661725 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8011d20 con 0x7f9cb40b87a0 2023-03-29T10:57:44.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.105+0000 7f9cbafff700 1 -- 172.21.15.111:0/1378661725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cb40b87a0 msgr2=0x7f9cb40cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:44.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.105+0000 7f9cbafff700 1 --2- 172.21.15.111:0/1378661725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cb40b87a0 0x7f9cb40cfbf0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009840 tx=0x7f9ca8009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:44.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.105+0000 7f9cbafff700 1 -- 172.21.15.111:0/1378661725 shutdown_connections 2023-03-29T10:57:44.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.105+0000 7f9cbafff700 1 --2- 172.21.15.111:0/1378661725 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cb40b87a0 0x7f9cb40cfbf0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:44.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.105+0000 7f9cbafff700 1 --2- 172.21.15.111:0/1378661725 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9cb41601e0 0x7f9cb4160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:44.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.105+0000 7f9cbafff700 1 -- 172.21.15.111:0/1378661725 >> 172.21.15.111:0/1378661725 conn(0x7f9cb4071b10 msgr2=0x7f9cb4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:44.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.105+0000 7f9cbafff700 1 -- 172.21.15.111:0/1378661725 shutdown_connections 2023-03-29T10:57:44.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.105+0000 7f9cbafff700 1 -- 172.21.15.111:0/1378661725 wait complete. 2023-03-29T10:57:44.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.107+0000 7f9cbafff700 1 Processor -- start 2023-03-29T10:57:44.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.107+0000 7f9cbafff700 1 -- start start 2023-03-29T10:57:44.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.108+0000 7f9cbafff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cb40b87a0 0x7f9cb4151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:44.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.108+0000 7f9cbafff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9cb41601e0 0x7f9cb40ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:44.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.108+0000 7f9cbafff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cb4160b70 0x7f9cb40b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:44.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.108+0000 7f9cbafff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb40cad70 con 0x7f9cb40b87a0 2023-03-29T10:57:44.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.108+0000 7f9cbafff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb40cb050 con 0x7f9cb4160b70 2023-03-29T10:57:44.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.108+0000 7f9cbafff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb40cb330 con 0x7f9cb41601e0 2023-03-29T10:57:44.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.108+0000 7f9cb8d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cb40b87a0 0x7f9cb4151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:44.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.108+0000 7f9cb8d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cb40b87a0 0x7f9cb4151260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58214/0 (socket says 172.21.15.111:58214) 2023-03-29T10:57:44.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.108+0000 7f9cb8d9b700 1 -- 172.21.15.111:0/486347892 learned_addr learned my addr 172.21.15.111:0/486347892 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:44.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.108+0000 7f9cb3fff700 1 --2- 172.21.15.111:0/486347892 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9cb41601e0 0x7f9cb40ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:44.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.108+0000 7f9cb959c700 1 --2- 172.21.15.111:0/486347892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cb4160b70 0x7f9cb40b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:44.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.109+0000 7f9cb8d9b700 1 -- 172.21.15.111:0/486347892 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9cb41601e0 msgr2=0x7f9cb40ca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:44.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.109+0000 7f9cb8d9b700 1 --2- 172.21.15.111:0/486347892 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9cb41601e0 0x7f9cb40ca2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:44.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.109+0000 7f9cb8d9b700 1 -- 172.21.15.111:0/486347892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cb4160b70 msgr2=0x7f9cb40b6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:44.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.109+0000 7f9cb8d9b700 1 --2- 172.21.15.111:0/486347892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cb4160b70 0x7f9cb40b6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:44.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.109+0000 7f9cb8d9b700 1 -- 172.21.15.111:0/486347892 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca80094a0 con 0x7f9cb40b87a0 2023-03-29T10:57:44.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.109+0000 7f9cb3fff700 1 --2- 172.21.15.111:0/486347892 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9cb41601e0 0x7f9cb40ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:57:44.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.109+0000 7f9cb8d9b700 1 --2- 172.21.15.111:0/486347892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cb40b87a0 0x7f9cb4151260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009fd0 tx=0x7f9ca8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:44.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.109+0000 7f9cb17fa700 1 -- 172.21.15.111:0/486347892 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca80038a0 con 0x7f9cb40b87a0 2023-03-29T10:57:44.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.109+0000 7f9cb17fa700 1 -- 172.21.15.111:0/486347892 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca801f070 con 0x7f9cb40b87a0 2023-03-29T10:57:44.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.109+0000 7f9cbafff700 1 -- 172.21.15.111:0/486347892 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb40cd980 con 0x7f9cb40b87a0 2023-03-29T10:57:44.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.109+0000 7f9cb17fa700 1 -- 172.21.15.111:0/486347892 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca8003c40 con 0x7f9cb40b87a0 2023-03-29T10:57:44.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.109+0000 7f9cbafff700 1 -- 172.21.15.111:0/486347892 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb40cd6a0 con 0x7f9cb40b87a0 2023-03-29T10:57:44.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.110+0000 7f9cbafff700 1 -- 172.21.15.111:0/486347892 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb40d2110 con 0x7f9cb40b87a0 2023-03-29T10:57:44.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.111+0000 7f9cb17fa700 1 -- 172.21.15.111:0/486347892 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f9ca8026480 con 0x7f9cb40b87a0 2023-03-29T10:57:44.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.112+0000 7f9cb17fa700 1 --2- 172.21.15.111:0/486347892 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9c98069de0 0x7f9c9806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:44.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.112+0000 7f9cb17fa700 1 -- 172.21.15.111:0/486347892 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9ca8021070 con 0x7f9cb40b87a0 2023-03-29T10:57:44.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.112+0000 7f9cb3fff700 1 --2- 172.21.15.111:0/486347892 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9c98069de0 0x7f9c9806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:44.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.113+0000 7f9cb3fff700 1 --2- 172.21.15.111:0/486347892 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9c98069de0 0x7f9c9806c290 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9cb415fd70 tx=0x7f9ca0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:44.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.116+0000 7f9cb17fa700 1 -- 172.21.15.111:0/486347892 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f9ca8055360 con 0x7f9cb40b87a0 2023-03-29T10:57:44.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.308+0000 7f9cbafff700 1 -- 172.21.15.111:0/486347892 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9cb40d3340 con 0x7f9cb40b87a0 2023-03-29T10:57:44.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.308+0000 7f9cb17fa700 1 -- 172.21.15.111:0/486347892 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9ca80150a0 con 0x7f9cb40b87a0 2023-03-29T10:57:44.308 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:57:44.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.310+0000 7f9cbafff700 1 -- 172.21.15.111:0/486347892 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9c98069de0 msgr2=0x7f9c9806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:44.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.310+0000 7f9cbafff700 1 --2- 172.21.15.111:0/486347892 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9c98069de0 0x7f9c9806c290 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9cb415fd70 tx=0x7f9ca0008040 comp rx=0 tx=0).stop 2023-03-29T10:57:44.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.310+0000 7f9cbafff700 1 -- 172.21.15.111:0/486347892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cb40b87a0 msgr2=0x7f9cb4151260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:44.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.310+0000 7f9cbafff700 1 --2- 172.21.15.111:0/486347892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cb40b87a0 0x7f9cb4151260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9ca8009fd0 tx=0x7f9ca8004750 comp rx=0 tx=0).stop 2023-03-29T10:57:44.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.311+0000 7f9cbafff700 1 -- 172.21.15.111:0/486347892 shutdown_connections 2023-03-29T10:57:44.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.311+0000 7f9cbafff700 1 --2- 172.21.15.111:0/486347892 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9c98069de0 0x7f9c9806c290 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:44.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.311+0000 7f9cbafff700 1 --2- 172.21.15.111:0/486347892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9cb40b87a0 0x7f9cb4151260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:44.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.311+0000 7f9cbafff700 1 --2- 172.21.15.111:0/486347892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9cb4160b70 0x7f9cb40b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:44.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.311+0000 7f9cbafff700 1 --2- 172.21.15.111:0/486347892 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9cb41601e0 0x7f9cb40ca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:44.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.311+0000 7f9cbafff700 1 -- 172.21.15.111:0/486347892 >> 172.21.15.111:0/486347892 conn(0x7f9cb4071b10 msgr2=0x7f9cb4071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:44.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.311+0000 7f9cbafff700 1 -- 172.21.15.111:0/486347892 shutdown_connections 2023-03-29T10:57:44.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.311+0000 7f9cbafff700 1 -- 172.21.15.111:0/486347892 wait complete. 2023-03-29T10:57:44.327 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-03-29T10:57:44.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.512+0000 7fdc475ff700 1 Processor -- start 2023-03-29T10:57:44.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.512+0000 7fdc475ff700 1 -- start start 2023-03-29T10:57:44.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.513+0000 7fdc475ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc400b87a0 0x7fdc400cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:44.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.513+0000 7fdc475ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc40160d40 0x7fdc400d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:44.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.513+0000 7fdc475ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc400cdc60 con 0x7fdc401601e0 2023-03-29T10:57:44.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.513+0000 7fdc475ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc400d02d0 con 0x7fdc400b87a0 2023-03-29T10:57:44.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.513+0000 7fdc475ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc400d0cb0 con 0x7fdc40160d40 2023-03-29T10:57:44.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.514+0000 7fdc45b9c700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc40160d40 0x7fdc400d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:44.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.514+0000 7fdc4539b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc400b87a0 0x7fdc400cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:44.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.514+0000 7fdc44b9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdc401601e0 0x7fdc40160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.111:51976/0 (socket says 172.21.15.111:51976) 2023-03-29T10:57:44.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.514+0000 7fdc45b9c700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc40160d40 0x7fdc400d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:56084/0 (socket says 172.21.15.111:56084) 2023-03-29T10:57:44.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.514+0000 7fdc44b9a700 1 -- 172.21.15.111:0/3354042683 learned_addr learned my addr 172.21.15.111:0/3354042683 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:44.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.514+0000 7fdc45b9c700 1 -- 172.21.15.111:0/3354042683 >> v2:172.21.15.119:3300/0 conn(0x7fdc400b87a0 msgr2=0x7fdc400cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:44.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.514+0000 7fdc45b9c700 1 --2- 172.21.15.111:0/3354042683 >> v2:172.21.15.119:3300/0 conn(0x7fdc400b87a0 0x7fdc400cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:44.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.514+0000 7fdc45b9c700 1 -- 172.21.15.111:0/3354042683 >> v1:172.21.15.119:6789/0 conn(0x7fdc401601e0 legacy=0x7fdc40160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:44.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.514+0000 7fdc45b9c700 1 -- 172.21.15.111:0/3354042683 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc400cd6a0 con 0x7fdc40160d40 2023-03-29T10:57:44.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.514+0000 7fdc4539b700 1 --2- 172.21.15.111:0/3354042683 >> v2:172.21.15.119:3300/0 conn(0x7fdc400b87a0 0x7fdc400cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:44.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.514+0000 7fdc45b9c700 1 --2- 172.21.15.111:0/3354042683 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc40160d40 0x7fdc400d1100 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009d90 tx=0x7fdc3800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cac3f5f163830cc9 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:44.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.515+0000 7fdc37fff700 1 -- 172.21.15.111:0/3354042683 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3800e070 con 0x7fdc40160d40 2023-03-29T10:57:44.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.515+0000 7fdc37fff700 1 -- 172.21.15.111:0/3354042683 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3800bb20 con 0x7fdc40160d40 2023-03-29T10:57:44.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.515+0000 7fdc37fff700 1 -- 172.21.15.111:0/3354042683 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc38012430 con 0x7fdc40160d40 2023-03-29T10:57:44.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.515+0000 7fdc475ff700 1 -- 172.21.15.111:0/3354042683 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc40160d40 msgr2=0x7fdc400d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:44.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.515+0000 7fdc475ff700 1 --2- 172.21.15.111:0/3354042683 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc40160d40 0x7fdc400d1100 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009d90 tx=0x7fdc3800b330 comp rx=0 tx=0).stop 2023-03-29T10:57:44.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.515+0000 7fdc475ff700 1 -- 172.21.15.111:0/3354042683 shutdown_connections 2023-03-29T10:57:44.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.515+0000 7fdc475ff700 1 --2- 172.21.15.111:0/3354042683 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc40160d40 0x7fdc400d1100 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:44.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.515+0000 7fdc475ff700 1 --2- 172.21.15.111:0/3354042683 >> v2:172.21.15.119:3300/0 conn(0x7fdc400b87a0 0x7fdc400cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:44.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.515+0000 7fdc475ff700 1 -- 172.21.15.111:0/3354042683 >> 172.21.15.111:0/3354042683 conn(0x7fdc40071b10 msgr2=0x7fdc40071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:44.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.515+0000 7fdc475ff700 1 -- 172.21.15.111:0/3354042683 shutdown_connections 2023-03-29T10:57:44.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.515+0000 7fdc475ff700 1 -- 172.21.15.111:0/3354042683 wait complete. 2023-03-29T10:57:44.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.517+0000 7fdc475ff700 1 Processor -- start 2023-03-29T10:57:44.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.517+0000 7fdc475ff700 1 -- start start 2023-03-29T10:57:44.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.518+0000 7fdc475ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc400b87a0 0x7fdc40155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:44.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.518+0000 7fdc475ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc401601e0 0x7fdc40159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:44.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.518+0000 7fdc475ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc40160d40 0x7fdc40150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:44.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.518+0000 7fdc475ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc40158040 con 0x7fdc400b87a0 2023-03-29T10:57:44.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.518+0000 7fdc475ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc40157430 con 0x7fdc401601e0 2023-03-29T10:57:44.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.518+0000 7fdc475ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc40157710 con 0x7fdc40160d40 2023-03-29T10:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.518+0000 7fdc4539b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc400b87a0 0x7fdc40155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.518+0000 7fdc4539b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc400b87a0 0x7fdc40155c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58218/0 (socket says 172.21.15.111:58218) 2023-03-29T10:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.518+0000 7fdc4539b700 1 -- 172.21.15.111:0/3554169989 learned_addr learned my addr 172.21.15.111:0/3554169989 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.518+0000 7fdc45b9c700 1 --2- 172.21.15.111:0/3554169989 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc40160d40 0x7fdc40150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.518+0000 7fdc44b9a700 1 --2- 172.21.15.111:0/3554169989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc401601e0 0x7fdc40159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.519+0000 7fdc4539b700 1 -- 172.21.15.111:0/3554169989 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc40160d40 msgr2=0x7fdc40150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.519+0000 7fdc4539b700 1 --2- 172.21.15.111:0/3554169989 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc40160d40 0x7fdc40150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.519+0000 7fdc4539b700 1 -- 172.21.15.111:0/3554169989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc401601e0 msgr2=0x7fdc40159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.519+0000 7fdc4539b700 1 --2- 172.21.15.111:0/3554169989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc401601e0 0x7fdc40159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:44.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.519+0000 7fdc4539b700 1 -- 172.21.15.111:0/3554169989 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c0094a0 con 0x7fdc400b87a0 2023-03-29T10:57:44.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.519+0000 7fdc4539b700 1 --2- 172.21.15.111:0/3554169989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc400b87a0 0x7fdc40155c70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00ebe0 tx=0x7fdc3c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:44.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.519+0000 7fdc35ffb700 1 -- 172.21.15.111:0/3554169989 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c00c850 con 0x7fdc400b87a0 2023-03-29T10:57:44.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.519+0000 7fdc35ffb700 1 -- 172.21.15.111:0/3554169989 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3c00c9b0 con 0x7fdc400b87a0 2023-03-29T10:57:44.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.519+0000 7fdc475ff700 1 -- 172.21.15.111:0/3554169989 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc38009a70 con 0x7fdc400b87a0 2023-03-29T10:57:44.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.519+0000 7fdc35ffb700 1 -- 172.21.15.111:0/3554169989 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c00cd40 con 0x7fdc400b87a0 2023-03-29T10:57:44.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.519+0000 7fdc475ff700 1 -- 172.21.15.111:0/3554169989 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc400d1640 con 0x7fdc400b87a0 2023-03-29T10:57:44.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.520+0000 7fdc475ff700 1 -- 172.21.15.111:0/3554169989 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc400cd980 con 0x7fdc400b87a0 2023-03-29T10:57:44.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.521+0000 7fdc35ffb700 1 -- 172.21.15.111:0/3554169989 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fdc3c013470 con 0x7fdc400b87a0 2023-03-29T10:57:44.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.522+0000 7fdc35ffb700 1 --2- 172.21.15.111:0/3554169989 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdc24069de0 0x7fdc2406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:44.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.522+0000 7fdc35ffb700 1 -- 172.21.15.111:0/3554169989 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdc3c0882c0 con 0x7fdc400b87a0 2023-03-29T10:57:44.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.522+0000 7fdc44b9a700 1 --2- 172.21.15.111:0/3554169989 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdc24069de0 0x7fdc2406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:44.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.523+0000 7fdc44b9a700 1 --2- 172.21.15.111:0/3554169989 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdc24069de0 0x7fdc2406c290 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdc400b6150 tx=0x7fdc30009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:44.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.526+0000 7fdc35ffb700 1 -- 172.21.15.111:0/3554169989 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fdc3c057340 con 0x7fdc400b87a0 2023-03-29T10:57:44.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.735+0000 7fdc475ff700 1 -- 172.21.15.111:0/3554169989 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdc4015cce0 con 0x7fdc400b87a0 2023-03-29T10:57:44.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.736+0000 7fdc35ffb700 1 -- 172.21.15.111:0/3554169989 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v28) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdc3c00cea0 con 0x7fdc400b87a0 2023-03-29T10:57:44.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.738+0000 7fdc475ff700 1 -- 172.21.15.111:0/3554169989 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdc24069de0 msgr2=0x7fdc2406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:44.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.738+0000 7fdc475ff700 1 --2- 172.21.15.111:0/3554169989 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdc24069de0 0x7fdc2406c290 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdc400b6150 tx=0x7fdc30009040 comp rx=0 tx=0).stop 2023-03-29T10:57:44.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.738+0000 7fdc475ff700 1 -- 172.21.15.111:0/3554169989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc400b87a0 msgr2=0x7fdc40155c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:44.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.738+0000 7fdc475ff700 1 --2- 172.21.15.111:0/3554169989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc400b87a0 0x7fdc40155c70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00ebe0 tx=0x7fdc3c00eef0 comp rx=0 tx=0).stop 2023-03-29T10:57:44.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.738+0000 7fdc475ff700 1 -- 172.21.15.111:0/3554169989 shutdown_connections 2023-03-29T10:57:44.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.738+0000 7fdc475ff700 1 --2- 172.21.15.111:0/3554169989 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdc24069de0 0x7fdc2406c290 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:44.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.738+0000 7fdc475ff700 1 --2- 172.21.15.111:0/3554169989 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc400b87a0 0x7fdc40155c70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:44.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.738+0000 7fdc475ff700 1 --2- 172.21.15.111:0/3554169989 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc40160d40 0x7fdc40150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:44.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.738+0000 7fdc475ff700 1 --2- 172.21.15.111:0/3554169989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc401601e0 0x7fdc40159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:44.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.738+0000 7fdc475ff700 1 -- 172.21.15.111:0/3554169989 >> 172.21.15.111:0/3554169989 conn(0x7fdc40071b10 msgr2=0x7fdc400cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:44.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.738+0000 7fdc475ff700 1 -- 172.21.15.111:0/3554169989 shutdown_connections 2023-03-29T10:57:44.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:44.738+0000 7fdc475ff700 1 -- 172.21.15.111:0/3554169989 wait complete. 2023-03-29T10:57:44.739 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-03-29T10:57:44.755 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi111.front.sepia.ceph.com... 2023-03-29T10:57:44.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:57:44.755 DEBUG:teuthology.orchestra.run.smithi111:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-03-29T10:57:44.815 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi111.stderr:ceph-fuse[118142]: fuse finished with error 0 and tester_r 0 2023-03-29T10:57:44.817 INFO:teuthology.orchestra.run:waiting for 300 2023-03-29T10:57:50.919 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-03-29T10:57:50.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:57:50.919 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-03-29T10:57:50.971 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:57:51.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.159+0000 7f690afff700 1 Processor -- start 2023-03-29T10:57:51.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.159+0000 7f690afff700 1 -- start start 2023-03-29T10:57:51.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.160+0000 7f690afff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f69040cf970 0x7f69040cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:51.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.160+0000 7f690afff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69040b70f0 0x7f69040b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:51.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.160+0000 7f690afff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69040cdc10 con 0x7f69040cfd40 2023-03-29T10:57:51.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.160+0000 7f690afff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69040d08d0 con 0x7f69040cf970 2023-03-29T10:57:51.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.160+0000 7f690afff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69040b6c60 con 0x7f69040b70f0 2023-03-29T10:57:51.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.160+0000 7f6903fff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f69040cfd40 0x7f69040d01c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41584/0 (socket says 172.21.15.111:41584) 2023-03-29T10:57:51.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.160+0000 7f6903fff700 1 -- 172.21.15.111:0/868822632 learned_addr learned my addr 172.21.15.111:0/868822632 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:51.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.161+0000 7f690959c700 1 --2- 172.21.15.111:0/868822632 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69040b70f0 0x7f69040b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:51.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.161+0000 7f6908d9b700 1 --2- 172.21.15.111:0/868822632 >> v2:172.21.15.119:3300/0 conn(0x7f69040cf970 0x7f69040cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:51.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.161+0000 7f69037fe700 1 -- 172.21.15.111:0/868822632 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625787296 0 0) 0x7f69040cdc10 con 0x7f69040cfd40 2023-03-29T10:57:51.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.161+0000 7f69037fe700 1 -- 172.21.15.111:0/868822632 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68f40035b0 con 0x7f69040cfd40 2023-03-29T10:57:51.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.161+0000 7f69037fe700 1 -- 172.21.15.111:0/868822632 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 680039374 0 0) 0x7f68f40035b0 con 0x7f69040cfd40 2023-03-29T10:57:51.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.161+0000 7f69037fe700 1 -- 172.21.15.111:0/868822632 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69040b70f0 msgr2=0x7f69040b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:51.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.161+0000 7f69037fe700 1 --2- 172.21.15.111:0/868822632 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69040b70f0 0x7f69040b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:51.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.161+0000 7f69037fe700 1 -- 172.21.15.111:0/868822632 >> v2:172.21.15.119:3300/0 conn(0x7f69040cf970 msgr2=0x7f69040cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:51.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.161+0000 7f69037fe700 1 --2- 172.21.15.111:0/868822632 >> v2:172.21.15.119:3300/0 conn(0x7f69040cf970 0x7f69040cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:51.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.161+0000 7f69037fe700 1 -- 172.21.15.111:0/868822632 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69040cd650 con 0x7f69040cfd40 2023-03-29T10:57:51.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.161+0000 7f69037fe700 1 -- 172.21.15.111:0/868822632 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f68f0001670 con 0x7f69040cfd40 2023-03-29T10:57:51.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.161+0000 7f6908d9b700 1 --2- 172.21.15.111:0/868822632 >> v2:172.21.15.119:3300/0 conn(0x7f69040cf970 0x7f69040cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:51.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.161+0000 7f69037fe700 1 -- 172.21.15.111:0/868822632 >> v1:172.21.15.111:6789/0 conn(0x7f69040cfd40 legacy=0x7f69040d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:51.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.161+0000 7f69037fe700 1 --2- 172.21.15.111:0/868822632 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68f4003a70 0x7f68f4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:51.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.161+0000 7f69037fe700 1 --2- 172.21.15.111:0/868822632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f4004850 0x7f68f4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:51.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.161+0000 7f69037fe700 1 --2- 172.21.15.111:0/868822632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68f4005200 0x7f68f400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:51.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.161+0000 7f69037fe700 1 -- 172.21.15.111:0/868822632 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69040d08d0 con 0x7f68f4004850 2023-03-29T10:57:51.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.161+0000 7f69037fe700 1 -- 172.21.15.111:0/868822632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69040b6c60 con 0x7f68f4005200 2023-03-29T10:57:51.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.161+0000 7f69037fe700 1 -- 172.21.15.111:0/868822632 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f400bfd0 con 0x7f68f4003a70 2023-03-29T10:57:51.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.162+0000 7f690959c700 1 --2- 172.21.15.111:0/868822632 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f69040b70f0 0x7f69040b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:51.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.162+0000 7f6903fff700 1 --2- 172.21.15.111:0/868822632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f4004850 0x7f68f4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:51.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.162+0000 7f6908d9b700 1 --2- 172.21.15.111:0/868822632 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68f4003a70 0x7f68f4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:51.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.162+0000 7f690959c700 1 --2- 172.21.15.111:0/868822632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68f4005200 0x7f68f400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:51.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.162+0000 7f6903fff700 1 -- 172.21.15.111:0/868822632 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68f4003a70 msgr2=0x7f68f4003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:51.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.162+0000 7f6903fff700 1 --2- 172.21.15.111:0/868822632 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f68f4003a70 0x7f68f4003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:51.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.162+0000 7f6903fff700 1 -- 172.21.15.111:0/868822632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68f4005200 msgr2=0x7f68f400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:51.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.162+0000 7f6903fff700 1 --2- 172.21.15.111:0/868822632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68f4005200 0x7f68f400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:51.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.162+0000 7f6903fff700 1 -- 172.21.15.111:0/868822632 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f40035b0 con 0x7f68f4004850 2023-03-29T10:57:51.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.162+0000 7f6903fff700 1 --2- 172.21.15.111:0/868822632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f4004850 0x7f68f4004cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f68f000c320 tx=0x7f68f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad444a2e07daa7a server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:51.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.163+0000 7f69037fe700 1 -- 172.21.15.111:0/868822632 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f000ce60 con 0x7f68f4004850 2023-03-29T10:57:51.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.163+0000 7f69037fe700 1 -- 172.21.15.111:0/868822632 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f0005960 con 0x7f68f4004850 2023-03-29T10:57:51.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.163+0000 7f69037fe700 1 -- 172.21.15.111:0/868822632 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f0005cf0 con 0x7f68f4004850 2023-03-29T10:57:51.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.163+0000 7f690afff700 1 -- 172.21.15.111:0/868822632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f4004850 msgr2=0x7f68f4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:51.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.163+0000 7f690afff700 1 --2- 172.21.15.111:0/868822632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f4004850 0x7f68f4004cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f68f000c320 tx=0x7f68f4005880 comp rx=0 tx=0).stop 2023-03-29T10:57:51.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.163+0000 7f690959c700 1 -- 172.21.15.111:0/868822632 reap_dead start 2023-03-29T10:57:51.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.163+0000 7f690afff700 1 -- 172.21.15.111:0/868822632 shutdown_connections 2023-03-29T10:57:51.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.163+0000 7f690afff700 1 -- 172.21.15.111:0/868822632 >> 172.21.15.111:0/868822632 conn(0x7f6904071b10 msgr2=0x7f6904071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:51.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.163+0000 7f690afff700 1 -- 172.21.15.111:0/868822632 shutdown_connections 2023-03-29T10:57:51.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.163+0000 7f690afff700 1 -- 172.21.15.111:0/868822632 wait complete. 2023-03-29T10:57:51.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.165+0000 7f690afff700 1 Processor -- start 2023-03-29T10:57:51.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.165+0000 7f690afff700 1 -- start start 2023-03-29T10:57:51.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.166+0000 7f690afff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f4004850 0x7f690414d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:51.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.166+0000 7f690afff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69040cc040 0x7f69040cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:51.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.166+0000 7f690afff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f690415da80 0x7f690415df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:51.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.166+0000 7f690afff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69040c87e0 con 0x7f68f4004850 2023-03-29T10:57:51.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.166+0000 7f690afff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69040cc9f0 con 0x7f69040cc040 2023-03-29T10:57:51.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.166+0000 7f690afff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69040c7ba0 con 0x7f690415da80 2023-03-29T10:57:51.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.166+0000 7f6908d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f4004850 0x7f690414d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:51.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.166+0000 7f6908d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f4004850 0x7f690414d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38142/0 (socket says 172.21.15.111:38142) 2023-03-29T10:57:51.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.166+0000 7f6908d9b700 1 -- 172.21.15.111:0/1692160302 learned_addr learned my addr 172.21.15.111:0/1692160302 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:51.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.167+0000 7f6908d9b700 1 -- 172.21.15.111:0/1692160302 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f690415da80 msgr2=0x7f690415df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:51.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.167+0000 7f6908d9b700 1 --2- 172.21.15.111:0/1692160302 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f690415da80 0x7f690415df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:51.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.167+0000 7f6908d9b700 1 -- 172.21.15.111:0/1692160302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69040cc040 msgr2=0x7f69040cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:51.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.167+0000 7f6908d9b700 1 --2- 172.21.15.111:0/1692160302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69040cc040 0x7f69040cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:51.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.167+0000 7f6908d9b700 1 -- 172.21.15.111:0/1692160302 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69040b68f0 con 0x7f68f4004850 2023-03-29T10:57:51.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.167+0000 7f6908d9b700 1 --2- 172.21.15.111:0/1692160302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f4004850 0x7f690414d410 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f68f8011cd0 tx=0x7f68f800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:51.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.167+0000 7f69017fa700 1 -- 172.21.15.111:0/1692160302 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f800fab0 con 0x7f68f4004850 2023-03-29T10:57:51.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.167+0000 7f69017fa700 1 -- 172.21.15.111:0/1692160302 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f800fc10 con 0x7f68f4004850 2023-03-29T10:57:51.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.167+0000 7f690afff700 1 -- 172.21.15.111:0/1692160302 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69040cd930 con 0x7f68f4004850 2023-03-29T10:57:51.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.167+0000 7f69017fa700 1 -- 172.21.15.111:0/1692160302 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8004020 con 0x7f68f4004850 2023-03-29T10:57:51.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.167+0000 7f690afff700 1 -- 172.21.15.111:0/1692160302 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69040cf650 con 0x7f68f4004850 2023-03-29T10:57:51.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.168+0000 7f690afff700 1 -- 172.21.15.111:0/1692160302 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6904074430 con 0x7f68f4004850 2023-03-29T10:57:51.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.169+0000 7f69017fa700 1 -- 172.21.15.111:0/1692160302 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f68f8004180 con 0x7f68f4004850 2023-03-29T10:57:51.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.171+0000 7f69017fa700 1 --2- 172.21.15.111:0/1692160302 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f68e8069de0 0x7f68e806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:51.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.172+0000 7f69017fa700 1 -- 172.21.15.111:0/1692160302 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f68f8088780 con 0x7f68f4004850 2023-03-29T10:57:51.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.172+0000 7f6903fff700 1 --2- 172.21.15.111:0/1692160302 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f68e8069de0 0x7f68e806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:51.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.172+0000 7f6903fff700 1 --2- 172.21.15.111:0/1692160302 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f68e8069de0 0x7f68e806c290 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f690415ba20 tx=0x7f68f000f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:51.175 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.176+0000 7f69017fa700 1 -- 172.21.15.111:0/1692160302 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f68f8057760 con 0x7f68f4004850 2023-03-29T10:57:51.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.376+0000 7f690afff700 1 -- 172.21.15.111:0/1692160302 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69040b52f0 con 0x7f68f4004850 2023-03-29T10:57:51.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.378+0000 7f69017fa700 1 -- 172.21.15.111:0/1692160302 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19796 (secure 0 0 0) 0x7f68f8057350 con 0x7f68f4004850 2023-03-29T10:57:51.378 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:51.378 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":28,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:57:32.331880+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:57:51.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.380+0000 7f690afff700 1 -- 172.21.15.111:0/1692160302 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f68e8069de0 msgr2=0x7f68e806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:51.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.380+0000 7f690afff700 1 --2- 172.21.15.111:0/1692160302 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f68e8069de0 0x7f68e806c290 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f690415ba20 tx=0x7f68f000f000 comp rx=0 tx=0).stop 2023-03-29T10:57:51.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.380+0000 7f690afff700 1 -- 172.21.15.111:0/1692160302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f4004850 msgr2=0x7f690414d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:51.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.380+0000 7f690afff700 1 --2- 172.21.15.111:0/1692160302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f4004850 0x7f690414d410 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f68f8011cd0 tx=0x7f68f800f520 comp rx=0 tx=0).stop 2023-03-29T10:57:51.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.380+0000 7f690afff700 1 -- 172.21.15.111:0/1692160302 shutdown_connections 2023-03-29T10:57:51.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.380+0000 7f690afff700 1 --2- 172.21.15.111:0/1692160302 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f68e8069de0 0x7f68e806c290 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:51.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.380+0000 7f690afff700 1 --2- 172.21.15.111:0/1692160302 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68f4004850 0x7f690414d410 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:51.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.380+0000 7f690afff700 1 --2- 172.21.15.111:0/1692160302 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f690415da80 0x7f690415df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:51.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.380+0000 7f690afff700 1 --2- 172.21.15.111:0/1692160302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69040cc040 0x7f69040cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:51.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.380+0000 7f690afff700 1 -- 172.21.15.111:0/1692160302 >> 172.21.15.111:0/1692160302 conn(0x7f6904071b10 msgr2=0x7f690415ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:51.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.380+0000 7f690afff700 1 -- 172.21.15.111:0/1692160302 shutdown_connections 2023-03-29T10:57:51.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.381+0000 7f690afff700 1 -- 172.21.15.111:0/1692160302 wait complete. 2023-03-29T10:57:51.397 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:57:51.484 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T10:57:51.488 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T10:57:51.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.585+0000 7fdbda9ff700 1 Processor -- start 2023-03-29T10:57:51.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.586+0000 7fdbda9ff700 1 -- start start 2023-03-29T10:57:51.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.587+0000 7fdbda9ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbd40cf970 0x7fdbd40cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:51.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.587+0000 7fdbda9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdbd40b70f0 0x7fdbd40b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:51.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.587+0000 7fdbda9ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd40cdc10 con 0x7fdbd40cfd40 2023-03-29T10:57:51.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.587+0000 7fdbda9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd40d08c0 con 0x7fdbd40b70f0 2023-03-29T10:57:51.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.587+0000 7fdbda9ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd40b6c60 con 0x7fdbd40cf970 2023-03-29T10:57:51.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.587+0000 7fdbd3fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbd40cf970 0x7fdbd40cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:51.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.587+0000 7fdbd3fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbd40cf970 0x7fdbd40cf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:50544/0 (socket says 172.21.15.111:50544) 2023-03-29T10:57:51.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.587+0000 7fdbd3fff700 1 -- 172.21.15.111:0/53907621 learned_addr learned my addr 172.21.15.111:0/53907621 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:51.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.587+0000 7fdbd8f9c700 1 --2- 172.21.15.111:0/53907621 >> v2:172.21.15.119:3300/0 conn(0x7fdbd40b70f0 0x7fdbd40b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:51.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.587+0000 7fdbcbfff700 1 --1- 172.21.15.111:0/53907621 >> v1:172.21.15.119:6789/0 conn(0x7fdbd40cfd40 0x7fdbd40d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.111:35084/0 (socket says 172.21.15.111:35084) 2023-03-29T10:57:51.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.588+0000 7fdbd3fff700 1 -- 172.21.15.111:0/53907621 >> v2:172.21.15.119:3300/0 conn(0x7fdbd40b70f0 msgr2=0x7fdbd40b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:51.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.588+0000 7fdbd3fff700 1 --2- 172.21.15.111:0/53907621 >> v2:172.21.15.119:3300/0 conn(0x7fdbd40b70f0 0x7fdbd40b7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:51.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.588+0000 7fdbd3fff700 1 -- 172.21.15.111:0/53907621 >> v1:172.21.15.119:6789/0 conn(0x7fdbd40cfd40 legacy=0x7fdbd40d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:51.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.588+0000 7fdbd3fff700 1 -- 172.21.15.111:0/53907621 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd40cd650 con 0x7fdbd40cf970 2023-03-29T10:57:51.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.588+0000 7fdbd3fff700 1 --2- 172.21.15.111:0/53907621 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbd40cf970 0x7fdbd40cf110 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40098f0 tx=0x7fdbc4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58704ad506c201e2 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:51.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.588+0000 7fdbd37fe700 1 -- 172.21.15.111:0/53907621 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4011830 con 0x7fdbd40cf970 2023-03-29T10:57:51.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.588+0000 7fdbd37fe700 1 -- 172.21.15.111:0/53907621 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc4011990 con 0x7fdbd40cf970 2023-03-29T10:57:51.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.588+0000 7fdbd37fe700 1 -- 172.21.15.111:0/53907621 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4011d20 con 0x7fdbd40cf970 2023-03-29T10:57:51.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.589+0000 7fdbda9ff700 1 -- 172.21.15.111:0/53907621 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbd40cf970 msgr2=0x7fdbd40cf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:51.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.589+0000 7fdbda9ff700 1 --2- 172.21.15.111:0/53907621 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbd40cf970 0x7fdbd40cf110 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdbc40098f0 tx=0x7fdbc4009c00 comp rx=0 tx=0).stop 2023-03-29T10:57:51.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.589+0000 7fdbda9ff700 1 -- 172.21.15.111:0/53907621 shutdown_connections 2023-03-29T10:57:51.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.589+0000 7fdbda9ff700 1 --2- 172.21.15.111:0/53907621 >> v2:172.21.15.119:3300/0 conn(0x7fdbd40b70f0 0x7fdbd40b7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:51.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.589+0000 7fdbda9ff700 1 --2- 172.21.15.111:0/53907621 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbd40cf970 0x7fdbd40cf110 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:51.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.589+0000 7fdbda9ff700 1 -- 172.21.15.111:0/53907621 >> 172.21.15.111:0/53907621 conn(0x7fdbd4071b10 msgr2=0x7fdbd4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:51.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.589+0000 7fdbda9ff700 1 -- 172.21.15.111:0/53907621 shutdown_connections 2023-03-29T10:57:51.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.589+0000 7fdbda9ff700 1 -- 172.21.15.111:0/53907621 wait complete. 2023-03-29T10:57:51.590 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.590+0000 7fdbda9ff700 1 Processor -- start 2023-03-29T10:57:51.590 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.591+0000 7fdbda9ff700 1 -- start start 2023-03-29T10:57:51.591 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.591+0000 7fdbda9ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbd40b70f0 0x7fdbd41e87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:51.591 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.591+0000 7fdbda9ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd40cf970 0x7fdbd4159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:51.591 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.591+0000 7fdbda9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbd40cfd40 0x7fdbd415eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:51.591 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.591+0000 7fdbda9ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd41e8330 con 0x7fdbd40cf970 2023-03-29T10:57:51.591 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.591+0000 7fdbda9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd41e7720 con 0x7fdbd40cfd40 2023-03-29T10:57:51.592 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.591+0000 7fdbda9ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd41e7a00 con 0x7fdbd40b70f0 2023-03-29T10:57:51.592 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.592+0000 7fdbcbfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd40cf970 0x7fdbd4159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:51.592 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.592+0000 7fdbcbfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd40cf970 0x7fdbd4159eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38144/0 (socket says 172.21.15.111:38144) 2023-03-29T10:57:51.592 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.592+0000 7fdbcbfff700 1 -- 172.21.15.111:0/3246143912 learned_addr learned my addr 172.21.15.111:0/3246143912 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:51.592 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.592+0000 7fdbd8f9c700 1 --2- 172.21.15.111:0/3246143912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbd40cfd40 0x7fdbd415eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:51.592 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.592+0000 7fdbd3fff700 1 --2- 172.21.15.111:0/3246143912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbd40b70f0 0x7fdbd41e87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:51.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.592+0000 7fdbcbfff700 1 -- 172.21.15.111:0/3246143912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbd40b70f0 msgr2=0x7fdbd41e87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:51.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.592+0000 7fdbcbfff700 1 --2- 172.21.15.111:0/3246143912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbd40b70f0 0x7fdbd41e87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:51.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.592+0000 7fdbcbfff700 1 -- 172.21.15.111:0/3246143912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbd40cfd40 msgr2=0x7fdbd415eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:51.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.592+0000 7fdbcbfff700 1 --2- 172.21.15.111:0/3246143912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbd40cfd40 0x7fdbd415eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:51.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.592+0000 7fdbcbfff700 1 -- 172.21.15.111:0/3246143912 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc4009550 con 0x7fdbd40cf970 2023-03-29T10:57:51.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.592+0000 7fdbcbfff700 1 --2- 172.21.15.111:0/3246143912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd40cf970 0x7fdbd4159eb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00c500 tx=0x7fdbbc00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:51.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.593+0000 7fdbd17fa700 1 -- 172.21.15.111:0/3246143912 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc005cb0 con 0x7fdbd40cf970 2023-03-29T10:57:51.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.593+0000 7fdbd17fa700 1 -- 172.21.15.111:0/3246143912 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbbc005e10 con 0x7fdbd40cf970 2023-03-29T10:57:51.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.593+0000 7fdbda9ff700 1 -- 172.21.15.111:0/3246143912 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd40b68f0 con 0x7fdbd40cf970 2023-03-29T10:57:51.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.593+0000 7fdbd17fa700 1 -- 172.21.15.111:0/3246143912 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc0072d0 con 0x7fdbd40cf970 2023-03-29T10:57:51.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.593+0000 7fdbda9ff700 1 -- 172.21.15.111:0/3246143912 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd40cd930 con 0x7fdbd40cf970 2023-03-29T10:57:51.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.594+0000 7fdbda9ff700 1 -- 172.21.15.111:0/3246143912 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd40cf650 con 0x7fdbd40cf970 2023-03-29T10:57:51.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.595+0000 7fdbd17fa700 1 -- 172.21.15.111:0/3246143912 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fdbbc007430 con 0x7fdbd40cf970 2023-03-29T10:57:51.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.595+0000 7fdbd17fa700 1 --2- 172.21.15.111:0/3246143912 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdbb4069e10 0x7fdbb406c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:51.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.596+0000 7fdbd17fa700 1 -- 172.21.15.111:0/3246143912 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdbbc087530 con 0x7fdbd40cf970 2023-03-29T10:57:51.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.596+0000 7fdbd3fff700 1 --2- 172.21.15.111:0/3246143912 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdbb4069e10 0x7fdbb406c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:51.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.596+0000 7fdbd3fff700 1 --2- 172.21.15.111:0/3246143912 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdbb4069e10 0x7fdbb406c2c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4014040 tx=0x7fdbc4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:51.599 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.600+0000 7fdbd17fa700 1 -- 172.21.15.111:0/3246143912 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fdbbc04c160 con 0x7fdbd40cf970 2023-03-29T10:57:51.793 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.793+0000 7fdbda9ff700 1 -- 172.21.15.111:0/3246143912 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbd4161150 con 0x7fdbd40cf970 2023-03-29T10:57:51.794 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.794+0000 7fdbd17fa700 1 -- 172.21.15.111:0/3246143912 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19796 (secure 0 0 0) 0x7fdbbc017090 con 0x7fdbd40cf970 2023-03-29T10:57:51.794 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:51.795 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":28,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:57:32.331880+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:57:51.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.797+0000 7fdbda9ff700 1 -- 172.21.15.111:0/3246143912 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdbb4069e10 msgr2=0x7fdbb406c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:51.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.797+0000 7fdbda9ff700 1 --2- 172.21.15.111:0/3246143912 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdbb4069e10 0x7fdbb406c2c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fdbc4014040 tx=0x7fdbc4008030 comp rx=0 tx=0).stop 2023-03-29T10:57:51.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.797+0000 7fdbda9ff700 1 -- 172.21.15.111:0/3246143912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd40cf970 msgr2=0x7fdbd4159eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:51.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.797+0000 7fdbda9ff700 1 --2- 172.21.15.111:0/3246143912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd40cf970 0x7fdbd4159eb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc00c500 tx=0x7fdbbc00c810 comp rx=0 tx=0).stop 2023-03-29T10:57:51.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.797+0000 7fdbda9ff700 1 -- 172.21.15.111:0/3246143912 shutdown_connections 2023-03-29T10:57:51.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.797+0000 7fdbda9ff700 1 --2- 172.21.15.111:0/3246143912 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdbb4069e10 0x7fdbb406c2c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:51.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.797+0000 7fdbda9ff700 1 --2- 172.21.15.111:0/3246143912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbd40cfd40 0x7fdbd415eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:51.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.797+0000 7fdbda9ff700 1 --2- 172.21.15.111:0/3246143912 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdbd40b70f0 0x7fdbd41e87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:51.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.797+0000 7fdbda9ff700 1 --2- 172.21.15.111:0/3246143912 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdbd40cf970 0x7fdbd4159eb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:51.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.797+0000 7fdbda9ff700 1 -- 172.21.15.111:0/3246143912 >> 172.21.15.111:0/3246143912 conn(0x7fdbd4071b10 msgr2=0x7fdbd4072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:51.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.797+0000 7fdbda9ff700 1 -- 172.21.15.111:0/3246143912 shutdown_connections 2023-03-29T10:57:51.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:51.797+0000 7fdbda9ff700 1 -- 172.21.15.111:0/3246143912 wait complete. 2023-03-29T10:57:51.814 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-03-29T10:57:52.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.038+0000 7fa6111ff700 1 Processor -- start 2023-03-29T10:57:52.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.038+0000 7fa6111ff700 1 -- start start 2023-03-29T10:57:52.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.039+0000 7fa6111ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa60c0cd640 0x7fa60c0cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:52.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.039+0000 7fa6111ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa60c15d480 con 0x7fa60c15a280 2023-03-29T10:57:52.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.039+0000 7fa6111ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa60c15d6e0 con 0x7fa60c0d00b0 2023-03-29T10:57:52.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.039+0000 7fa6111ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c15d9c0 con 0x7fa60c0cd640 2023-03-29T10:57:52.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.040+0000 7fa60ad9d700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fa60c15a280 0x7fa60c15cd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41590/0 (socket says 172.21.15.111:41590) 2023-03-29T10:57:52.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.040+0000 7fa60ad9d700 1 -- 172.21.15.111:0/2671851896 learned_addr learned my addr 172.21.15.111:0/2671851896 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:52.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.040+0000 7fa60a59c700 1 --2- 172.21.15.111:0/2671851896 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa60c0cd640 0x7fa60c0cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:52.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.040+0000 7fa609d9b700 1 -- 172.21.15.111:0/2671851896 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184621397 0 0) 0x7fa60c15d480 con 0x7fa60c15a280 2023-03-29T10:57:52.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.040+0000 7fa609d9b700 1 -- 172.21.15.111:0/2671851896 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5f40035b0 con 0x7fa60c15a280 2023-03-29T10:57:52.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.040+0000 7fa609d9b700 1 -- 172.21.15.111:0/2671851896 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3491516506 0 0) 0x7fa60c15d6e0 con 0x7fa60c0d00b0 2023-03-29T10:57:52.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.040+0000 7fa609d9b700 1 -- 172.21.15.111:0/2671851896 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa60c15d480 con 0x7fa60c0d00b0 2023-03-29T10:57:52.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.040+0000 7fa60a59c700 1 -- 172.21.15.111:0/2671851896 >> v1:172.21.15.119:6789/0 conn(0x7fa60c0d00b0 legacy=0x7fa60c0d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:52.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.040+0000 7fa60a59c700 1 -- 172.21.15.111:0/2671851896 >> v1:172.21.15.111:6789/0 conn(0x7fa60c15a280 legacy=0x7fa60c15cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:52.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.040+0000 7fa60a59c700 1 -- 172.21.15.111:0/2671851896 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60c15dca0 con 0x7fa60c0cd640 2023-03-29T10:57:52.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.041+0000 7fa60a59c700 1 --2- 172.21.15.111:0/2671851896 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa60c0cd640 0x7fa60c0cfb70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa600009870 tx=0x7fa600009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=751f28d64ce951e3 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:52.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.041+0000 7fa609d9b700 1 -- 172.21.15.111:0/2671851896 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa600011830 con 0x7fa60c0cd640 2023-03-29T10:57:52.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.041+0000 7fa609d9b700 1 -- 172.21.15.111:0/2671851896 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa600011990 con 0x7fa60c0cd640 2023-03-29T10:57:52.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.041+0000 7fa609d9b700 1 -- 172.21.15.111:0/2671851896 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa600011d20 con 0x7fa60c0cd640 2023-03-29T10:57:52.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.041+0000 7fa6111ff700 1 -- 172.21.15.111:0/2671851896 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa60c0cd640 msgr2=0x7fa60c0cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:52.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.041+0000 7fa6111ff700 1 --2- 172.21.15.111:0/2671851896 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa60c0cd640 0x7fa60c0cfb70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa600009870 tx=0x7fa600009b80 comp rx=0 tx=0).stop 2023-03-29T10:57:52.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.041+0000 7fa6111ff700 1 -- 172.21.15.111:0/2671851896 shutdown_connections 2023-03-29T10:57:52.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.041+0000 7fa6111ff700 1 --2- 172.21.15.111:0/2671851896 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa60c0cd640 0x7fa60c0cfb70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:52.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.041+0000 7fa6111ff700 1 -- 172.21.15.111:0/2671851896 >> 172.21.15.111:0/2671851896 conn(0x7fa60c066260 msgr2=0x7fa60c071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:52.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.041+0000 7fa6111ff700 1 -- 172.21.15.111:0/2671851896 shutdown_connections 2023-03-29T10:57:52.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.042+0000 7fa6111ff700 1 -- 172.21.15.111:0/2671851896 wait complete. 2023-03-29T10:57:52.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.044+0000 7fa6111ff700 1 Processor -- start 2023-03-29T10:57:52.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.044+0000 7fa6111ff700 1 -- start start 2023-03-29T10:57:52.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.045+0000 7fa6111ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60c0cd640 0x7fa60c0cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:52.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.045+0000 7fa6111ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa60c0d00b0 0x7fa60c14c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:52.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.045+0000 7fa6111ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa60c15a280 0x7fa60c158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:52.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.045+0000 7fa6111ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c0cbce0 con 0x7fa60c0cd640 2023-03-29T10:57:52.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.045+0000 7fa6111ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c14c9f0 con 0x7fa60c15a280 2023-03-29T10:57:52.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.045+0000 7fa6111ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60c0b6220 con 0x7fa60c0d00b0 2023-03-29T10:57:52.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.045+0000 7fa60ad9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60c0cd640 0x7fa60c0cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:52.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.045+0000 7fa60ad9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60c0cd640 0x7fa60c0cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38150/0 (socket says 172.21.15.111:38150) 2023-03-29T10:57:52.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.045+0000 7fa60ad9d700 1 -- 172.21.15.111:0/1172411951 learned_addr learned my addr 172.21.15.111:0/1172411951 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:52.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.046+0000 7fa60a59c700 1 --2- 172.21.15.111:0/1172411951 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa60c0d00b0 0x7fa60c14c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:52.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.046+0000 7fa60ad9d700 1 -- 172.21.15.111:0/1172411951 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa60c0d00b0 msgr2=0x7fa60c14c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:52.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.046+0000 7fa60b59e700 1 --2- 172.21.15.111:0/1172411951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa60c15a280 0x7fa60c158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:52.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.046+0000 7fa60ad9d700 1 --2- 172.21.15.111:0/1172411951 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa60c0d00b0 0x7fa60c14c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:52.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.046+0000 7fa60ad9d700 1 -- 172.21.15.111:0/1172411951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa60c15a280 msgr2=0x7fa60c158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:52.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.046+0000 7fa60ad9d700 1 --2- 172.21.15.111:0/1172411951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa60c15a280 0x7fa60c158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:52.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.046+0000 7fa60ad9d700 1 -- 172.21.15.111:0/1172411951 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6000094a0 con 0x7fa60c0cd640 2023-03-29T10:57:52.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.046+0000 7fa60a59c700 1 --2- 172.21.15.111:0/1172411951 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa60c0d00b0 0x7fa60c14c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:52.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.046+0000 7fa60b59e700 1 --2- 172.21.15.111:0/1172411951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa60c15a280 0x7fa60c158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:52.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.046+0000 7fa60ad9d700 1 --2- 172.21.15.111:0/1172411951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60c0cd640 0x7fa60c0cb7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00c700 tx=0x7fa5fc00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:52.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.046+0000 7fa5f37fe700 1 -- 172.21.15.111:0/1172411951 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc005d70 con 0x7fa60c0cd640 2023-03-29T10:57:52.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.046+0000 7fa5f37fe700 1 -- 172.21.15.111:0/1172411951 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc0070d0 con 0x7fa60c0cd640 2023-03-29T10:57:52.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.046+0000 7fa6111ff700 1 -- 172.21.15.111:0/1172411951 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa60c0cf650 con 0x7fa60c0cd640 2023-03-29T10:57:52.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.046+0000 7fa5f37fe700 1 -- 172.21.15.111:0/1172411951 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc007480 con 0x7fa60c0cd640 2023-03-29T10:57:52.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.046+0000 7fa6111ff700 1 -- 172.21.15.111:0/1172411951 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa60c0cf370 con 0x7fa60c0cd640 2023-03-29T10:57:52.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.047+0000 7fa6111ff700 1 -- 172.21.15.111:0/1172411951 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60c15a650 con 0x7fa60c0cd640 2023-03-29T10:57:52.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.048+0000 7fa5f37fe700 1 -- 172.21.15.111:0/1172411951 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fa5fc0075e0 con 0x7fa60c0cd640 2023-03-29T10:57:52.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.049+0000 7fa5f37fe700 1 --2- 172.21.15.111:0/1172411951 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa5f8069de0 0x7fa5f806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:52.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.049+0000 7fa5f37fe700 1 -- 172.21.15.111:0/1172411951 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa5fc0879b0 con 0x7fa60c0cd640 2023-03-29T10:57:52.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.049+0000 7fa60a59c700 1 --2- 172.21.15.111:0/1172411951 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa5f8069de0 0x7fa5f806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:52.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.050+0000 7fa60a59c700 1 --2- 172.21.15.111:0/1172411951 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa5f8069de0 0x7fa5f806c290 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0b8060 tx=0x7fa60001c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:52.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.053+0000 7fa5f37fe700 1 -- 172.21.15.111:0/1172411951 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fa5fc052ee0 con 0x7fa60c0cd640 2023-03-29T10:57:52.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.289+0000 7fa6111ff700 1 -- 172.21.15.111:0/1172411951 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa60c15cf20 con 0x7fa60c0cd640 2023-03-29T10:57:52.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.368+0000 7fa5f37fe700 1 -- 172.21.15.111:0/1172411951 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa5fc056500 con 0x7fa60c0cd640 2023-03-29T10:57:52.370 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.370+0000 7fa6111ff700 1 -- 172.21.15.111:0/1172411951 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa5f8069de0 msgr2=0x7fa5f806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:52.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.370+0000 7fa6111ff700 1 --2- 172.21.15.111:0/1172411951 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa5f8069de0 0x7fa5f806c290 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0b8060 tx=0x7fa60001c040 comp rx=0 tx=0).stop 2023-03-29T10:57:52.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.370+0000 7fa6111ff700 1 -- 172.21.15.111:0/1172411951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60c0cd640 msgr2=0x7fa60c0cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:52.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.370+0000 7fa6111ff700 1 --2- 172.21.15.111:0/1172411951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60c0cd640 0x7fa60c0cb7a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc00c700 tx=0x7fa5fc00cac0 comp rx=0 tx=0).stop 2023-03-29T10:57:52.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.370+0000 7fa6111ff700 1 -- 172.21.15.111:0/1172411951 shutdown_connections 2023-03-29T10:57:52.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.370+0000 7fa6111ff700 1 --2- 172.21.15.111:0/1172411951 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa5f8069de0 0x7fa5f806c290 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:52.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.370+0000 7fa6111ff700 1 --2- 172.21.15.111:0/1172411951 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa60c0cd640 0x7fa60c0cb7a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:52.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.370+0000 7fa6111ff700 1 --2- 172.21.15.111:0/1172411951 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa60c15a280 0x7fa60c158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:52.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.370+0000 7fa6111ff700 1 --2- 172.21.15.111:0/1172411951 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa60c0d00b0 0x7fa60c14c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:52.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.370+0000 7fa6111ff700 1 -- 172.21.15.111:0/1172411951 >> 172.21.15.111:0/1172411951 conn(0x7fa60c066260 msgr2=0x7fa60c0d2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:52.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.370+0000 7fa6111ff700 1 -- 172.21.15.111:0/1172411951 shutdown_connections 2023-03-29T10:57:52.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.370+0000 7fa6111ff700 1 -- 172.21.15.111:0/1172411951 wait complete. 2023-03-29T10:57:52.388 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-03-29T10:57:52.388 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-03-29T10:57:52.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.576+0000 7fbc9e5ff700 1 Processor -- start 2023-03-29T10:57:52.576 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.576+0000 7fbc9e5ff700 1 -- start start 2023-03-29T10:57:52.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.577+0000 7fbc9e5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc980b87a0 0x7fbc980cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:52.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.577+0000 7fbc9e5ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbc98160d40 0x7fbc980d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:52.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.577+0000 7fbc9e5ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc980cdc60 con 0x7fbc981601e0 2023-03-29T10:57:52.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.577+0000 7fbc9e5ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc980d02d0 con 0x7fbc98160d40 2023-03-29T10:57:52.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.577+0000 7fbc9e5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc980d0cb0 con 0x7fbc980b87a0 2023-03-29T10:57:52.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.578+0000 7fbc8ffff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fbc981601e0 0x7fbc98160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41596/0 (socket says 172.21.15.111:41596) 2023-03-29T10:57:52.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.578+0000 7fbc8ffff700 1 -- 172.21.15.111:0/1173614936 learned_addr learned my addr 172.21.15.111:0/1173614936 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:52.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.578+0000 7fbc9cb9c700 1 --2- 172.21.15.111:0/1173614936 >> v2:172.21.15.111:3300/0 conn(0x7fbc98160d40 0x7fbc980d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.578+0000 7fbc97fff700 1 --2- 172.21.15.111:0/1173614936 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc980b87a0 0x7fbc980cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.578+0000 7fbc977fe700 1 -- 172.21.15.111:0/1173614936 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914043464 0 0) 0x7fbc980cdc60 con 0x7fbc981601e0 2023-03-29T10:57:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.578+0000 7fbc977fe700 1 -- 172.21.15.111:0/1173614936 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc840035b0 con 0x7fbc981601e0 2023-03-29T10:57:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.578+0000 7fbc9cb9c700 1 -- 172.21.15.111:0/1173614936 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc980b87a0 msgr2=0x7fbc980cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.578+0000 7fbc9cb9c700 1 --2- 172.21.15.111:0/1173614936 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc980b87a0 0x7fbc980cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.578+0000 7fbc9cb9c700 1 -- 172.21.15.111:0/1173614936 >> v1:172.21.15.111:6789/0 conn(0x7fbc981601e0 legacy=0x7fbc98160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.578+0000 7fbc9cb9c700 1 -- 172.21.15.111:0/1173614936 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc980cd6a0 con 0x7fbc98160d40 2023-03-29T10:57:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.578+0000 7fbc9cb9c700 1 --2- 172.21.15.111:0/1173614936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc98160d40 0x7fbc980d1100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009d90 tx=0x7fbc9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2c5367df7f029f server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.579+0000 7fbc977fe700 1 -- 172.21.15.111:0/1173614936 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9000e070 con 0x7fbc98160d40 2023-03-29T10:57:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.579+0000 7fbc977fe700 1 -- 172.21.15.111:0/1173614936 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9000bb20 con 0x7fbc98160d40 2023-03-29T10:57:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.579+0000 7fbc977fe700 1 -- 172.21.15.111:0/1173614936 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc90012430 con 0x7fbc98160d40 2023-03-29T10:57:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.579+0000 7fbc9e5ff700 1 -- 172.21.15.111:0/1173614936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc98160d40 msgr2=0x7fbc980d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:52.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.579+0000 7fbc9e5ff700 1 --2- 172.21.15.111:0/1173614936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc98160d40 0x7fbc980d1100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009d90 tx=0x7fbc9000b330 comp rx=0 tx=0).stop 2023-03-29T10:57:52.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.579+0000 7fbc9e5ff700 1 -- 172.21.15.111:0/1173614936 shutdown_connections 2023-03-29T10:57:52.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.579+0000 7fbc9e5ff700 1 --2- 172.21.15.111:0/1173614936 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc98160d40 0x7fbc980d1100 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:52.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.579+0000 7fbc9e5ff700 1 --2- 172.21.15.111:0/1173614936 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc980b87a0 0x7fbc980cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:52.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.579+0000 7fbc9e5ff700 1 -- 172.21.15.111:0/1173614936 >> 172.21.15.111:0/1173614936 conn(0x7fbc98071b10 msgr2=0x7fbc98071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:52.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.579+0000 7fbc9e5ff700 1 -- 172.21.15.111:0/1173614936 shutdown_connections 2023-03-29T10:57:52.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.579+0000 7fbc9e5ff700 1 -- 172.21.15.111:0/1173614936 wait complete. 2023-03-29T10:57:52.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.581+0000 7fbc9e5ff700 1 Processor -- start 2023-03-29T10:57:52.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.581+0000 7fbc9e5ff700 1 -- start start 2023-03-29T10:57:52.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.582+0000 7fbc9e5ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc980b87a0 0x7fbc9815d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:52.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.582+0000 7fbc9e5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc981601e0 0x7fbc9815f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:52.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.582+0000 7fbc9e5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc98160d40 0x7fbc98158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:52.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.582+0000 7fbc9e5ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9815fbd0 con 0x7fbc980b87a0 2023-03-29T10:57:52.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.582+0000 7fbc9e5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9815df10 con 0x7fbc981601e0 2023-03-29T10:57:52.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.582+0000 7fbc9e5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc980cadf0 con 0x7fbc98160d40 2023-03-29T10:57:52.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.582+0000 7fbc97fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc980b87a0 0x7fbc9815d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:52.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.582+0000 7fbc97fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc980b87a0 0x7fbc9815d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38174/0 (socket says 172.21.15.111:38174) 2023-03-29T10:57:52.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.582+0000 7fbc97fff700 1 -- 172.21.15.111:0/371182487 learned_addr learned my addr 172.21.15.111:0/371182487 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:52.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.582+0000 7fbc8ffff700 1 --2- 172.21.15.111:0/371182487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc981601e0 0x7fbc9815f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:52.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.582+0000 7fbc9cb9c700 1 --2- 172.21.15.111:0/371182487 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc98160d40 0x7fbc98158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:52.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.583+0000 7fbc97fff700 1 -- 172.21.15.111:0/371182487 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc98160d40 msgr2=0x7fbc98158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:52.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.583+0000 7fbc97fff700 1 --2- 172.21.15.111:0/371182487 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc98160d40 0x7fbc98158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:52.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.583+0000 7fbc97fff700 1 -- 172.21.15.111:0/371182487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc981601e0 msgr2=0x7fbc9815f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:52.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.583+0000 7fbc97fff700 1 --2- 172.21.15.111:0/371182487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc981601e0 0x7fbc9815f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:52.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.583+0000 7fbc97fff700 1 -- 172.21.15.111:0/371182487 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc90009a70 con 0x7fbc980b87a0 2023-03-29T10:57:52.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.583+0000 7fbc97fff700 1 --2- 172.21.15.111:0/371182487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc980b87a0 0x7fbc9815d8c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800e9b0 tx=0x7fbc8800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:52.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.583+0000 7fbc957fa700 1 -- 172.21.15.111:0/371182487 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc88009710 con 0x7fbc980b87a0 2023-03-29T10:57:52.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.583+0000 7fbc957fa700 1 -- 172.21.15.111:0/371182487 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc8800ee50 con 0x7fbc980b87a0 2023-03-29T10:57:52.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.583+0000 7fbc9e5ff700 1 -- 172.21.15.111:0/371182487 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc980d1640 con 0x7fbc980b87a0 2023-03-29T10:57:52.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.583+0000 7fbc957fa700 1 -- 172.21.15.111:0/371182487 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc8800ca50 con 0x7fbc980b87a0 2023-03-29T10:57:52.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.583+0000 7fbc9e5ff700 1 -- 172.21.15.111:0/371182487 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc980cd980 con 0x7fbc980b87a0 2023-03-29T10:57:52.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.584+0000 7fbc9e5ff700 1 -- 172.21.15.111:0/371182487 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc980d17d0 con 0x7fbc980b87a0 2023-03-29T10:57:52.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.585+0000 7fbc957fa700 1 -- 172.21.15.111:0/371182487 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fbc8800cbb0 con 0x7fbc980b87a0 2023-03-29T10:57:52.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.586+0000 7fbc957fa700 1 --2- 172.21.15.111:0/371182487 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbc78069de0 0x7fbc7806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:52.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.586+0000 7fbc957fa700 1 -- 172.21.15.111:0/371182487 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbc88088300 con 0x7fbc980b87a0 2023-03-29T10:57:52.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.586+0000 7fbc8ffff700 1 --2- 172.21.15.111:0/371182487 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbc78069de0 0x7fbc7806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:52.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.587+0000 7fbc8ffff700 1 --2- 172.21.15.111:0/371182487 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbc78069de0 0x7fbc7806c290 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbc9815a8a0 tx=0x7fbc80007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:52.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.590+0000 7fbc957fa700 1 -- 172.21.15.111:0/371182487 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fbc880572e0 con 0x7fbc980b87a0 2023-03-29T10:57:52.821 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:52.821+0000 7fbc9e5ff700 1 -- 172.21.15.111:0/371182487 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbc9815e050 con 0x7fbc980b87a0 2023-03-29T10:57:53.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.376+0000 7fbc957fa700 1 -- 172.21.15.111:0/371182487 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbc8800ce60 con 0x7fbc980b87a0 2023-03-29T10:57:53.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.378+0000 7fbc9e5ff700 1 -- 172.21.15.111:0/371182487 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbc78069de0 msgr2=0x7fbc7806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:53.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.378+0000 7fbc9e5ff700 1 --2- 172.21.15.111:0/371182487 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbc78069de0 0x7fbc7806c290 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbc9815a8a0 tx=0x7fbc80007dc0 comp rx=0 tx=0).stop 2023-03-29T10:57:53.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.378+0000 7fbc9e5ff700 1 -- 172.21.15.111:0/371182487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc980b87a0 msgr2=0x7fbc9815d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:53.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.378+0000 7fbc9e5ff700 1 --2- 172.21.15.111:0/371182487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc980b87a0 0x7fbc9815d8c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800e9b0 tx=0x7fbc8800ecc0 comp rx=0 tx=0).stop 2023-03-29T10:57:53.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.378+0000 7fbc9e5ff700 1 -- 172.21.15.111:0/371182487 shutdown_connections 2023-03-29T10:57:53.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.378+0000 7fbc9e5ff700 1 --2- 172.21.15.111:0/371182487 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbc78069de0 0x7fbc7806c290 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:53.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.378+0000 7fbc9e5ff700 1 --2- 172.21.15.111:0/371182487 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbc980b87a0 0x7fbc9815d8c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:53.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.378+0000 7fbc9e5ff700 1 --2- 172.21.15.111:0/371182487 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbc98160d40 0x7fbc98158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:53.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.378+0000 7fbc9e5ff700 1 --2- 172.21.15.111:0/371182487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc981601e0 0x7fbc9815f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:53.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.379+0000 7fbc9e5ff700 1 -- 172.21.15.111:0/371182487 >> 172.21.15.111:0/371182487 conn(0x7fbc98071b10 msgr2=0x7fbc980b2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:53.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.379+0000 7fbc9e5ff700 1 -- 172.21.15.111:0/371182487 shutdown_connections 2023-03-29T10:57:53.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.379+0000 7fbc9e5ff700 1 -- 172.21.15.111:0/371182487 wait complete. 2023-03-29T10:57:53.397 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-03-29T10:57:53.397 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-29T10:57:53.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.580+0000 7fc0187ff700 1 Processor -- start 2023-03-29T10:57:53.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.580+0000 7fc0187ff700 1 -- start start 2023-03-29T10:57:53.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.582+0000 7fc0187ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc0100cf970 0x7fc0100cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:53.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.582+0000 7fc0187ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc0100cfd40 0x7fc0100d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:53.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.582+0000 7fc0187ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc010160290 0x7fc0101606d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:53.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.582+0000 7fc0187ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0100cdc10 con 0x7fc010160290 2023-03-29T10:57:53.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.582+0000 7fc0187ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0100b6c60 con 0x7fc0100cf970 2023-03-29T10:57:53.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.582+0000 7fc0187ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0100cd650 con 0x7fc0100cfd40 2023-03-29T10:57:53.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.582+0000 7fc016d9c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc010160290 0x7fc0101606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:53.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.582+0000 7fc016d9c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc010160290 0x7fc0101606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38180/0 (socket says 172.21.15.111:38180) 2023-03-29T10:57:53.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.582+0000 7fc016d9c700 1 -- 172.21.15.111:0/42188445 learned_addr learned my addr 172.21.15.111:0/42188445 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:53.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.582+0000 7fc01659b700 1 --2- 172.21.15.111:0/42188445 >> v2:172.21.15.119:3300/0 conn(0x7fc0100cf970 0x7fc0100cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:53.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.582+0000 7fc015d9a700 1 --2- 172.21.15.111:0/42188445 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc0100cfd40 0x7fc0100d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:53.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.582+0000 7fc016d9c700 1 -- 172.21.15.111:0/42188445 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc0100cfd40 msgr2=0x7fc0100d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:53.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.582+0000 7fc016d9c700 1 --2- 172.21.15.111:0/42188445 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc0100cfd40 0x7fc0100d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:53.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.582+0000 7fc016d9c700 1 -- 172.21.15.111:0/42188445 >> v2:172.21.15.119:3300/0 conn(0x7fc0100cf970 msgr2=0x7fc0100cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:53.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.582+0000 7fc016d9c700 1 --2- 172.21.15.111:0/42188445 >> v2:172.21.15.119:3300/0 conn(0x7fc0100cf970 0x7fc0100cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:53.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.582+0000 7fc016d9c700 1 -- 172.21.15.111:0/42188445 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0100cd930 con 0x7fc010160290 2023-03-29T10:57:53.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.583+0000 7fc016d9c700 1 --2- 172.21.15.111:0/42188445 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc010160290 0x7fc0101606d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc0100d0a10 tx=0x7fc00800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73bf6ecfeb0c3719 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:53.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.583+0000 7fc015599700 1 -- 172.21.15.111:0/42188445 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00800e070 con 0x7fc010160290 2023-03-29T10:57:53.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.583+0000 7fc015599700 1 -- 172.21.15.111:0/42188445 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00800b9c0 con 0x7fc010160290 2023-03-29T10:57:53.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.583+0000 7fc015599700 1 -- 172.21.15.111:0/42188445 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00800bd70 con 0x7fc010160290 2023-03-29T10:57:53.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.583+0000 7fc0187ff700 1 -- 172.21.15.111:0/42188445 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc010160290 msgr2=0x7fc0101606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:53.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.583+0000 7fc0187ff700 1 --2- 172.21.15.111:0/42188445 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc010160290 0x7fc0101606d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc0100d0a10 tx=0x7fc00800b330 comp rx=0 tx=0).stop 2023-03-29T10:57:53.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.583+0000 7fc0187ff700 1 -- 172.21.15.111:0/42188445 shutdown_connections 2023-03-29T10:57:53.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.583+0000 7fc0187ff700 1 --2- 172.21.15.111:0/42188445 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc010160290 0x7fc0101606d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:53.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.583+0000 7fc0187ff700 1 --2- 172.21.15.111:0/42188445 >> v2:172.21.15.119:3300/0 conn(0x7fc0100cf970 0x7fc0100cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:53.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.583+0000 7fc0187ff700 1 --2- 172.21.15.111:0/42188445 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc0100cfd40 0x7fc0100d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:53.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.583+0000 7fc0187ff700 1 -- 172.21.15.111:0/42188445 >> 172.21.15.111:0/42188445 conn(0x7fc010071b10 msgr2=0x7fc010071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:53.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.584+0000 7fc0187ff700 1 -- 172.21.15.111:0/42188445 shutdown_connections 2023-03-29T10:57:53.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.584+0000 7fc0187ff700 1 -- 172.21.15.111:0/42188445 wait complete. 2023-03-29T10:57:53.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.585+0000 7fc0187ff700 1 Processor -- start 2023-03-29T10:57:53.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.585+0000 7fc0187ff700 1 -- start start 2023-03-29T10:57:53.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.586+0000 7fc0187ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc0100cf970 0x7fc0101e8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:53.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.586+0000 7fc0187ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0100cfd40 0x7fc0101e64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:53.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.586+0000 7fc0187ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc010160290 0x7fc0101e6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:53.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.586+0000 7fc0187ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01015f790 con 0x7fc010160290 2023-03-29T10:57:53.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.586+0000 7fc0187ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01015eb80 con 0x7fc0100cfd40 2023-03-29T10:57:53.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.586+0000 7fc0187ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01015ee60 con 0x7fc0100cf970 2023-03-29T10:57:53.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.586+0000 7fc016d9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc010160290 0x7fc0101e6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:53.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.586+0000 7fc016d9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc010160290 0x7fc0101e6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38186/0 (socket says 172.21.15.111:38186) 2023-03-29T10:57:53.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.586+0000 7fc016d9c700 1 -- 172.21.15.111:0/4292085218 learned_addr learned my addr 172.21.15.111:0/4292085218 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:53.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.587+0000 7fc01659b700 1 --2- 172.21.15.111:0/4292085218 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc0100cf970 0x7fc0101e8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:53.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.587+0000 7fc015d9a700 1 --2- 172.21.15.111:0/4292085218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0100cfd40 0x7fc0101e64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:53.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.587+0000 7fc016d9c700 1 -- 172.21.15.111:0/4292085218 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc0100cf970 msgr2=0x7fc0101e8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:53.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.587+0000 7fc016d9c700 1 --2- 172.21.15.111:0/4292085218 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc0100cf970 0x7fc0101e8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:53.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.587+0000 7fc016d9c700 1 -- 172.21.15.111:0/4292085218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0100cfd40 msgr2=0x7fc0101e64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:53.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.587+0000 7fc016d9c700 1 --2- 172.21.15.111:0/4292085218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0100cfd40 0x7fc0101e64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:53.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.587+0000 7fc016d9c700 1 -- 172.21.15.111:0/4292085218 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc008009b30 con 0x7fc010160290 2023-03-29T10:57:53.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.587+0000 7fc016d9c700 1 --2- 172.21.15.111:0/4292085218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc010160290 0x7fc0101e6a00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc010072670 tx=0x7fc008004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:53.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.587+0000 7fc006ffd700 1 -- 172.21.15.111:0/4292085218 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc008004250 con 0x7fc010160290 2023-03-29T10:57:53.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.587+0000 7fc006ffd700 1 -- 172.21.15.111:0/4292085218 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0080043d0 con 0x7fc010160290 2023-03-29T10:57:53.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.587+0000 7fc0187ff700 1 -- 172.21.15.111:0/4292085218 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0100cf650 con 0x7fc010160290 2023-03-29T10:57:53.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.587+0000 7fc006ffd700 1 -- 172.21.15.111:0/4292085218 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc008004780 con 0x7fc010160290 2023-03-29T10:57:53.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.587+0000 7fc0187ff700 1 -- 172.21.15.111:0/4292085218 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0100b68f0 con 0x7fc010160290 2023-03-29T10:57:53.590 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.588+0000 7fc0187ff700 1 -- 172.21.15.111:0/4292085218 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0100d06d0 con 0x7fc010160290 2023-03-29T10:57:53.590 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.589+0000 7fc006ffd700 1 -- 172.21.15.111:0/4292085218 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fc00801c440 con 0x7fc010160290 2023-03-29T10:57:53.590 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.590+0000 7fc006ffd700 1 --2- 172.21.15.111:0/4292085218 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbff4069de0 0x7fbff406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:53.590 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.590+0000 7fc006ffd700 1 -- 172.21.15.111:0/4292085218 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc008089860 con 0x7fc010160290 2023-03-29T10:57:53.590 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.590+0000 7fc01659b700 1 --2- 172.21.15.111:0/4292085218 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbff4069de0 0x7fbff406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:53.590 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.591+0000 7fc01659b700 1 --2- 172.21.15.111:0/4292085218 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbff4069de0 0x7fbff406c290 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00ba10 tx=0x7fc00c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:53.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.594+0000 7fc006ffd700 1 -- 172.21.15.111:0/4292085218 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fc008054db0 con 0x7fc010160290 2023-03-29T10:57:53.828 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.828+0000 7fc0187ff700 1 -- 172.21.15.111:0/4292085218 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc010146d80 con 0x7fc010160290 2023-03-29T10:57:53.829 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.829+0000 7fc006ffd700 1 -- 172.21.15.111:0/4292085218 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fc008059e60 con 0x7fc010160290 2023-03-29T10:57:53.829 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:53.829 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/556826332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":556826332},{"type":"v1","addr":"172.21.15.111:6837","nonce":556826332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/814623168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":814623168},{"type":"v1","addr":"172.21.15.111:6835","nonce":814623168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/514422035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":514422035},{"type":"v1","addr":"172.21.15.119:6834","nonce":514422035}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-29T10:57:27.824135+0000","modified":"2023-03-29T10:57:53.367898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/3160800581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":3160800581},{"type":"v1","addr":"172.21.15.119:6835","nonce":3160800581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-03-29T10:57:53.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.831+0000 7fc0187ff700 1 -- 172.21.15.111:0/4292085218 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbff4069de0 msgr2=0x7fbff406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:53.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.831+0000 7fc0187ff700 1 --2- 172.21.15.111:0/4292085218 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbff4069de0 0x7fbff406c290 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc00c00ba10 tx=0x7fc00c008040 comp rx=0 tx=0).stop 2023-03-29T10:57:53.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.831+0000 7fc0187ff700 1 -- 172.21.15.111:0/4292085218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc010160290 msgr2=0x7fc0101e6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:53.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.831+0000 7fc0187ff700 1 --2- 172.21.15.111:0/4292085218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc010160290 0x7fc0101e6a00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc010072670 tx=0x7fc008004ad0 comp rx=0 tx=0).stop 2023-03-29T10:57:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.832+0000 7fc0187ff700 1 -- 172.21.15.111:0/4292085218 shutdown_connections 2023-03-29T10:57:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.832+0000 7fc0187ff700 1 --2- 172.21.15.111:0/4292085218 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbff4069de0 0x7fbff406c290 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.832+0000 7fc0187ff700 1 --2- 172.21.15.111:0/4292085218 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc010160290 0x7fc0101e6a00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.832+0000 7fc0187ff700 1 --2- 172.21.15.111:0/4292085218 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc0100cf970 0x7fc0101e8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.832+0000 7fc0187ff700 1 --2- 172.21.15.111:0/4292085218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0100cfd40 0x7fc0101e64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.832+0000 7fc0187ff700 1 -- 172.21.15.111:0/4292085218 >> 172.21.15.111:0/4292085218 conn(0x7fc010071b10 msgr2=0x7fc010071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.832+0000 7fc0187ff700 1 -- 172.21.15.111:0/4292085218 shutdown_connections 2023-03-29T10:57:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:53.832+0000 7fc0187ff700 1 -- 172.21.15.111:0/4292085218 wait complete. 2023-03-29T10:57:53.832 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2023-03-29T10:57:53.849 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:57:54.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.033+0000 7f9d5d5ff700 1 Processor -- start 2023-03-29T10:57:54.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.034+0000 7f9d5d5ff700 1 -- start start 2023-03-29T10:57:54.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.035+0000 7f9d5d5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d580cf970 0x7f9d580cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:54.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.035+0000 7f9d5d5ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9d580cfd40 0x7f9d580d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:54.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.035+0000 7f9d5d5ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d580cdc10 con 0x7f9d58160290 2023-03-29T10:57:54.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.035+0000 7f9d5d5ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d580b6c60 con 0x7f9d580cfd40 2023-03-29T10:57:54.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.035+0000 7f9d5d5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d580cd650 con 0x7f9d580cf970 2023-03-29T10:57:54.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.035+0000 7f9d4ffff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9d580cfd40 0x7f9d580d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:54.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.035+0000 7f9d577fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f9d58160290 0x7f9d581606d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41612/0 (socket says 172.21.15.111:41612) 2023-03-29T10:57:54.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.035+0000 7f9d577fe700 1 -- 172.21.15.111:0/142960184 learned_addr learned my addr 172.21.15.111:0/142960184 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:54.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.035+0000 7f9d56ffd700 1 --2- 172.21.15.111:0/142960184 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d580cf970 0x7f9d580cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:54.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.035+0000 7f9d567fc700 1 -- 172.21.15.111:0/142960184 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226043063 0 0) 0x7f9d580cdc10 con 0x7f9d58160290 2023-03-29T10:57:54.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.035+0000 7f9d567fc700 1 -- 172.21.15.111:0/142960184 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d440035b0 con 0x7f9d58160290 2023-03-29T10:57:54.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.035+0000 7f9d4ffff700 1 -- 172.21.15.111:0/142960184 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d580cf970 msgr2=0x7f9d580cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:54.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.035+0000 7f9d4ffff700 1 --2- 172.21.15.111:0/142960184 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d580cf970 0x7f9d580cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.035+0000 7f9d4ffff700 1 -- 172.21.15.111:0/142960184 >> v1:172.21.15.111:6789/0 conn(0x7f9d58160290 legacy=0x7f9d581606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:54.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.036+0000 7f9d4ffff700 1 -- 172.21.15.111:0/142960184 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d580cd930 con 0x7f9d580cfd40 2023-03-29T10:57:54.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.036+0000 7f9d4ffff700 1 --2- 172.21.15.111:0/142960184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d580cfd40 0x7f9d580d01b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009840 tx=0x7f9d40009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9df68fe6d525719 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:54.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.036+0000 7f9d567fc700 1 -- 172.21.15.111:0/142960184 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d40011830 con 0x7f9d580cfd40 2023-03-29T10:57:54.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.036+0000 7f9d567fc700 1 -- 172.21.15.111:0/142960184 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d40011990 con 0x7f9d580cfd40 2023-03-29T10:57:54.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.036+0000 7f9d567fc700 1 -- 172.21.15.111:0/142960184 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d40011d20 con 0x7f9d580cfd40 2023-03-29T10:57:54.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.036+0000 7f9d5d5ff700 1 -- 172.21.15.111:0/142960184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d580cfd40 msgr2=0x7f9d580d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:54.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.036+0000 7f9d5d5ff700 1 --2- 172.21.15.111:0/142960184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d580cfd40 0x7f9d580d01b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9d40009840 tx=0x7f9d40009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:54.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.037+0000 7f9d5d5ff700 1 -- 172.21.15.111:0/142960184 shutdown_connections 2023-03-29T10:57:54.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.037+0000 7f9d5d5ff700 1 --2- 172.21.15.111:0/142960184 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d580cf970 0x7f9d580cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.037+0000 7f9d5d5ff700 1 --2- 172.21.15.111:0/142960184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d580cfd40 0x7f9d580d01b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.037+0000 7f9d5d5ff700 1 -- 172.21.15.111:0/142960184 >> 172.21.15.111:0/142960184 conn(0x7f9d58071b10 msgr2=0x7f9d58071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:54.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.037+0000 7f9d5d5ff700 1 -- 172.21.15.111:0/142960184 shutdown_connections 2023-03-29T10:57:54.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.037+0000 7f9d5d5ff700 1 -- 172.21.15.111:0/142960184 wait complete. 2023-03-29T10:57:54.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.038+0000 7f9d5d5ff700 1 Processor -- start 2023-03-29T10:57:54.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.038+0000 7f9d5d5ff700 1 -- start start 2023-03-29T10:57:54.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.039+0000 7f9d5d5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d580cf970 0x7f9d58153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:54.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.039+0000 7f9d5d5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d580cfd40 0x7f9d58151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:54.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.039+0000 7f9d5d5ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d58160290 0x7f9d58151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:54.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.039+0000 7f9d5d5ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d581527f0 con 0x7f9d58160290 2023-03-29T10:57:54.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.039+0000 7f9d5d5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d58152ad0 con 0x7f9d580cfd40 2023-03-29T10:57:54.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.039+0000 7f9d5d5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d58152db0 con 0x7f9d580cf970 2023-03-29T10:57:54.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.040+0000 7f9d577fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d58160290 0x7f9d58151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:54.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.040+0000 7f9d577fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d58160290 0x7f9d58151ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38198/0 (socket says 172.21.15.111:38198) 2023-03-29T10:57:54.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.040+0000 7f9d577fe700 1 -- 172.21.15.111:0/2821593213 learned_addr learned my addr 172.21.15.111:0/2821593213 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:54.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.040+0000 7f9d56ffd700 1 --2- 172.21.15.111:0/2821593213 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d580cf970 0x7f9d58153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:54.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.040+0000 7f9d4ffff700 1 --2- 172.21.15.111:0/2821593213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d580cfd40 0x7f9d58151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:54.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.040+0000 7f9d577fe700 1 -- 172.21.15.111:0/2821593213 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d580cf970 msgr2=0x7f9d58153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:54.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.040+0000 7f9d577fe700 1 --2- 172.21.15.111:0/2821593213 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d580cf970 0x7f9d58153890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.040+0000 7f9d577fe700 1 -- 172.21.15.111:0/2821593213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d580cfd40 msgr2=0x7f9d58151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:54.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.040+0000 7f9d577fe700 1 --2- 172.21.15.111:0/2821593213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d580cfd40 0x7f9d58151600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.040+0000 7f9d577fe700 1 -- 172.21.15.111:0/2821593213 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d400094a0 con 0x7f9d58160290 2023-03-29T10:57:54.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.040+0000 7f9d577fe700 1 --2- 172.21.15.111:0/2821593213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d58160290 0x7f9d58151ea0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000caa0 tx=0x7f9d5000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:54.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.040+0000 7f9d4f7fe700 1 -- 172.21.15.111:0/2821593213 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5000a9e0 con 0x7f9d58160290 2023-03-29T10:57:54.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.041+0000 7f9d4f7fe700 1 -- 172.21.15.111:0/2821593213 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5000ab40 con 0x7f9d58160290 2023-03-29T10:57:54.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.041+0000 7f9d5d5ff700 1 -- 172.21.15.111:0/2821593213 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d580cf650 con 0x7f9d58160290 2023-03-29T10:57:54.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.041+0000 7f9d4f7fe700 1 -- 172.21.15.111:0/2821593213 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50012400 con 0x7f9d58160290 2023-03-29T10:57:54.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.041+0000 7f9d5d5ff700 1 -- 172.21.15.111:0/2821593213 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d580b68f0 con 0x7f9d58160290 2023-03-29T10:57:54.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.042+0000 7f9d5d5ff700 1 -- 172.21.15.111:0/2821593213 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d580d06f0 con 0x7f9d58160290 2023-03-29T10:57:54.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.043+0000 7f9d4f7fe700 1 -- 172.21.15.111:0/2821593213 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f9d5001b3b0 con 0x7f9d58160290 2023-03-29T10:57:54.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.043+0000 7f9d4f7fe700 1 --2- 172.21.15.111:0/2821593213 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9d38069de0 0x7f9d3806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:54.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.044+0000 7f9d4f7fe700 1 -- 172.21.15.111:0/2821593213 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9d50087e80 con 0x7f9d58160290 2023-03-29T10:57:54.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.044+0000 7f9d56ffd700 1 --2- 172.21.15.111:0/2821593213 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9d38069de0 0x7f9d3806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:54.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.044+0000 7f9d56ffd700 1 --2- 172.21.15.111:0/2821593213 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9d38069de0 0x7f9d3806c290 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009b40 tx=0x7f9d48009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:54.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.047+0000 7f9d4f7fe700 1 -- 172.21.15.111:0/2821593213 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f9d50053530 con 0x7f9d58160290 2023-03-29T10:57:54.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.246+0000 7f9d5d5ff700 1 -- 172.21.15.111:0/2821593213 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d580b50e0 con 0x7f9d58160290 2023-03-29T10:57:54.247 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.247+0000 7f9d4f7fe700 1 -- 172.21.15.111:0/2821593213 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19796 (secure 0 0 0) 0x7f9d50056b50 con 0x7f9d58160290 2023-03-29T10:57:54.247 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:54.247 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":28,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:57:32.331880+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:57:54.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.250+0000 7f9d5d5ff700 1 -- 172.21.15.111:0/2821593213 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9d38069de0 msgr2=0x7f9d3806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:54.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.250+0000 7f9d5d5ff700 1 --2- 172.21.15.111:0/2821593213 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9d38069de0 0x7f9d3806c290 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009b40 tx=0x7f9d48009210 comp rx=0 tx=0).stop 2023-03-29T10:57:54.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.250+0000 7f9d5d5ff700 1 -- 172.21.15.111:0/2821593213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d58160290 msgr2=0x7f9d58151ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:54.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.250+0000 7f9d5d5ff700 1 --2- 172.21.15.111:0/2821593213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d58160290 0x7f9d58151ea0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000caa0 tx=0x7f9d5000cdb0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.250+0000 7f9d5d5ff700 1 -- 172.21.15.111:0/2821593213 shutdown_connections 2023-03-29T10:57:54.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.250+0000 7f9d5d5ff700 1 --2- 172.21.15.111:0/2821593213 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9d38069de0 0x7f9d3806c290 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.250+0000 7f9d5d5ff700 1 --2- 172.21.15.111:0/2821593213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9d58160290 0x7f9d58151ea0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.250+0000 7f9d5d5ff700 1 --2- 172.21.15.111:0/2821593213 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9d580cf970 0x7f9d58153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.250+0000 7f9d5d5ff700 1 --2- 172.21.15.111:0/2821593213 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d580cfd40 0x7f9d58151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.250+0000 7f9d5d5ff700 1 -- 172.21.15.111:0/2821593213 >> 172.21.15.111:0/2821593213 conn(0x7f9d58071b10 msgr2=0x7f9d580d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:54.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.250+0000 7f9d5d5ff700 1 -- 172.21.15.111:0/2821593213 shutdown_connections 2023-03-29T10:57:54.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.250+0000 7f9d5d5ff700 1 -- 172.21.15.111:0/2821593213 wait complete. 2023-03-29T10:57:54.266 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-03-29T10:57:54.267 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T10:57:54.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.453+0000 7f8567dff700 1 Processor -- start 2023-03-29T10:57:54.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.454+0000 7f8567dff700 1 -- start start 2023-03-29T10:57:54.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.455+0000 7f8567dff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8560160c00 0x7f85600b5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:54.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.455+0000 7f8567dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85600cf190 0x7f85600cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:54.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.455+0000 7f8567dff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85601611d0 con 0x7f85600b88e0 2023-03-29T10:57:54.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.455+0000 7f8567dff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85600d2e60 con 0x7f8560160c00 2023-03-29T10:57:54.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.455+0000 7f8567dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85600cd830 con 0x7f85600cf190 2023-03-29T10:57:54.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.455+0000 7f856539a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8560160c00 0x7f85600b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:54.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.455+0000 7f8565b9b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f85600b88e0 0x7f85600b4a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41626/0 (socket says 172.21.15.111:41626) 2023-03-29T10:57:54.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.455+0000 7f8565b9b700 1 -- 172.21.15.111:0/3229465943 learned_addr learned my addr 172.21.15.111:0/3229465943 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:54.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.455+0000 7f856639c700 1 --2- 172.21.15.111:0/3229465943 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85600cf190 0x7f85600cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:54.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.456+0000 7f856539a700 1 -- 172.21.15.111:0/3229465943 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85600cf190 msgr2=0x7f85600cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:54.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.456+0000 7f856539a700 1 --2- 172.21.15.111:0/3229465943 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85600cf190 0x7f85600cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.456+0000 7f856539a700 1 -- 172.21.15.111:0/3229465943 >> v1:172.21.15.111:6789/0 conn(0x7f85600b88e0 legacy=0x7f85600b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:54.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.456+0000 7f856539a700 1 -- 172.21.15.111:0/3229465943 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85600cdc50 con 0x7f8560160c00 2023-03-29T10:57:54.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.456+0000 7f856539a700 1 --2- 172.21.15.111:0/3229465943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8560160c00 0x7f85600b5160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8550009840 tx=0x7f8550009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a46b0b5250e845 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:54.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.456+0000 7f8564b99700 1 -- 172.21.15.111:0/3229465943 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8550011830 con 0x7f8560160c00 2023-03-29T10:57:54.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.456+0000 7f8564b99700 1 -- 172.21.15.111:0/3229465943 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8550011990 con 0x7f8560160c00 2023-03-29T10:57:54.456 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.456+0000 7f8564b99700 1 -- 172.21.15.111:0/3229465943 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8550011d20 con 0x7f8560160c00 2023-03-29T10:57:54.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.457+0000 7f8567dff700 1 -- 172.21.15.111:0/3229465943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8560160c00 msgr2=0x7f85600b5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:54.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.457+0000 7f8567dff700 1 --2- 172.21.15.111:0/3229465943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8560160c00 0x7f85600b5160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8550009840 tx=0x7f8550009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:54.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.457+0000 7f8567dff700 1 -- 172.21.15.111:0/3229465943 shutdown_connections 2023-03-29T10:57:54.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.457+0000 7f8567dff700 1 --2- 172.21.15.111:0/3229465943 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85600cf190 0x7f85600cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.457+0000 7f8567dff700 1 --2- 172.21.15.111:0/3229465943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8560160c00 0x7f85600b5160 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.457+0000 7f8567dff700 1 -- 172.21.15.111:0/3229465943 >> 172.21.15.111:0/3229465943 conn(0x7f85600666b0 msgr2=0x7f8560071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:54.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.457+0000 7f8567dff700 1 -- 172.21.15.111:0/3229465943 shutdown_connections 2023-03-29T10:57:54.457 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.457+0000 7f8567dff700 1 -- 172.21.15.111:0/3229465943 wait complete. 2023-03-29T10:57:54.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.458+0000 7f8567dff700 1 Processor -- start 2023-03-29T10:57:54.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.458+0000 7f8567dff700 1 -- start start 2023-03-29T10:57:54.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.459+0000 7f8567dff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85600cf190 0x7f85601491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:54.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.459+0000 7f8567dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8560160c00 0x7f85600b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:54.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.459+0000 7f8567dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85601f63c0 0x7f85600cf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:54.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.459+0000 7f8567dff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85601496e0 con 0x7f85600cf190 2023-03-29T10:57:54.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.459+0000 7f8567dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85601f5690 con 0x7f8560160c00 2023-03-29T10:57:54.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.459+0000 7f8567dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85600b2e70 con 0x7f85601f63c0 2023-03-29T10:57:54.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.460+0000 7f8565b9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85600cf190 0x7f85601491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:54.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.460+0000 7f8565b9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85600cf190 0x7f85601491a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38212/0 (socket says 172.21.15.111:38212) 2023-03-29T10:57:54.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.460+0000 7f8565b9b700 1 -- 172.21.15.111:0/3704526726 learned_addr learned my addr 172.21.15.111:0/3704526726 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:54.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.460+0000 7f856639c700 1 --2- 172.21.15.111:0/3704526726 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85601f63c0 0x7f85600cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:54.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.460+0000 7f856539a700 1 --2- 172.21.15.111:0/3704526726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8560160c00 0x7f85600b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:54.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.460+0000 7f8565b9b700 1 -- 172.21.15.111:0/3704526726 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85601f63c0 msgr2=0x7f85600cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:54.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.460+0000 7f8565b9b700 1 --2- 172.21.15.111:0/3704526726 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85601f63c0 0x7f85600cf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.460+0000 7f8565b9b700 1 -- 172.21.15.111:0/3704526726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8560160c00 msgr2=0x7f85600b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:54.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.460+0000 7f8565b9b700 1 --2- 172.21.15.111:0/3704526726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8560160c00 0x7f85600b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.460+0000 7f8565b9b700 1 -- 172.21.15.111:0/3704526726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85500094a0 con 0x7f85600cf190 2023-03-29T10:57:54.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.460+0000 7f8565b9b700 1 --2- 172.21.15.111:0/3704526726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85600cf190 0x7f85601491a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f855c00c520 tx=0x7f855c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:54.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.461+0000 7f85567fc700 1 -- 172.21.15.111:0/3704526726 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c005d70 con 0x7f85600cf190 2023-03-29T10:57:54.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.461+0000 7f85567fc700 1 -- 172.21.15.111:0/3704526726 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f855c0070d0 con 0x7f85600cf190 2023-03-29T10:57:54.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.461+0000 7f85567fc700 1 -- 172.21.15.111:0/3704526726 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855c007440 con 0x7f85600cf190 2023-03-29T10:57:54.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.461+0000 7f8567dff700 1 -- 172.21.15.111:0/3704526726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85600d2b80 con 0x7f85600cf190 2023-03-29T10:57:54.461 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.461+0000 7f8567dff700 1 -- 172.21.15.111:0/3704526726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85600d28a0 con 0x7f85600cf190 2023-03-29T10:57:54.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.462+0000 7f8567dff700 1 -- 172.21.15.111:0/3704526726 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85600d0c30 con 0x7f85600cf190 2023-03-29T10:57:54.462 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.463+0000 7f85567fc700 1 -- 172.21.15.111:0/3704526726 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f855c0075a0 con 0x7f85600cf190 2023-03-29T10:57:54.463 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.463+0000 7f85567fc700 1 --2- 172.21.15.111:0/3704526726 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8544069de0 0x7f854406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:54.463 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.464+0000 7f85567fc700 1 -- 172.21.15.111:0/3704526726 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f855c0873c0 con 0x7f85600cf190 2023-03-29T10:57:54.463 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.464+0000 7f856539a700 1 --2- 172.21.15.111:0/3704526726 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8544069de0 0x7f854406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:54.463 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.464+0000 7f856539a700 1 --2- 172.21.15.111:0/3704526726 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8544069de0 0x7f854406c290 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f85500090b0 tx=0x7f8550008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:54.467 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.467+0000 7f85567fc700 1 -- 172.21.15.111:0/3704526726 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f855c04c160 con 0x7f85600cf190 2023-03-29T10:57:54.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.658+0000 7f8567dff700 1 -- 172.21.15.111:0/3704526726 --> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f85601611d0 con 0x7f8544069de0 2023-03-29T10:57:54.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.670+0000 7f85567fc700 1 -- 172.21.15.111:0/3704526726 <== mgr.4103 v2:172.21.15.111:6832/110141 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367280 (secure 0 0 0) 0x7f85601611d0 con 0x7f8544069de0 2023-03-29T10:57:54.675 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:54.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.675+0000 7f8567dff700 1 -- 172.21.15.111:0/3704526726 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8544069de0 msgr2=0x7f854406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:54.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.675+0000 7f8567dff700 1 --2- 172.21.15.111:0/3704526726 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8544069de0 0x7f854406c290 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f85500090b0 tx=0x7f8550008030 comp rx=0 tx=0).stop 2023-03-29T10:57:54.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.675+0000 7f8567dff700 1 -- 172.21.15.111:0/3704526726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85600cf190 msgr2=0x7f85601491a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:54.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.675+0000 7f8567dff700 1 --2- 172.21.15.111:0/3704526726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85600cf190 0x7f85601491a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f855c00c520 tx=0x7f855c00c830 comp rx=0 tx=0).stop 2023-03-29T10:57:54.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.675+0000 7f8567dff700 1 -- 172.21.15.111:0/3704526726 shutdown_connections 2023-03-29T10:57:54.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.675+0000 7f8567dff700 1 --2- 172.21.15.111:0/3704526726 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8544069de0 0x7f854406c290 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.675+0000 7f8567dff700 1 --2- 172.21.15.111:0/3704526726 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85600cf190 0x7f85601491a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.675+0000 7f8567dff700 1 --2- 172.21.15.111:0/3704526726 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85601f63c0 0x7f85600cf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.675+0000 7f8567dff700 1 --2- 172.21.15.111:0/3704526726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8560160c00 0x7f85600b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.677 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.675+0000 7f8567dff700 1 -- 172.21.15.111:0/3704526726 >> 172.21.15.111:0/3704526726 conn(0x7f85600666b0 msgr2=0x7f856015ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:54.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.675+0000 7f8567dff700 1 -- 172.21.15.111:0/3704526726 shutdown_connections 2023-03-29T10:57:54.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.675+0000 7f8567dff700 1 -- 172.21.15.111:0/3704526726 wait complete. 2023-03-29T10:57:54.678 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2023-03-29T10:57:54.692 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2023-03-29T10:57:54.131594+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594966,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67584,"kb_used_data":3584,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":754907136,"statfs":{"total":773094113280,"available":773024907264,"internally_reserved":0,"allocated":3670016,"data_stored":2185270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":142,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003130"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.845704+0000","last_change":"2023-03-29T10:57:27.854795+0000","last_active":"2023-03-29T10:57:31.845704+0000","last_peered":"2023-03-29T10:57:31.845704+0000","last_clean":"2023-03-29T10:57:31.845704+0000","last_became_active":"2023-03-29T10:57:27.854521+0000","last_became_peered":"2023-03-29T10:57:27.854521+0000","last_unstale":"2023-03-29T10:57:31.845704+0000","last_undegraded":"2023-03-29T10:57:31.845704+0000","last_fullsized":"2023-03-29T10:57:31.845704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:40:19.611213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.862071+0000","last_change":"2023-03-29T10:57:27.850091+0000","last_active":"2023-03-29T10:57:31.862071+0000","last_peered":"2023-03-29T10:57:31.862071+0000","last_clean":"2023-03-29T10:57:31.862071+0000","last_became_active":"2023-03-29T10:57:27.849795+0000","last_became_peered":"2023-03-29T10:57:27.849795+0000","last_unstale":"2023-03-29T10:57:31.862071+0000","last_undegraded":"2023-03-29T10:57:31.862071+0000","last_fullsized":"2023-03-29T10:57:31.862071+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:36:09.200571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853118+0000","last_change":"2023-03-29T10:57:27.864238+0000","last_active":"2023-03-29T10:57:31.853118+0000","last_peered":"2023-03-29T10:57:31.853118+0000","last_clean":"2023-03-29T10:57:31.853118+0000","last_became_active":"2023-03-29T10:57:27.863878+0000","last_became_peered":"2023-03-29T10:57:27.863878+0000","last_unstale":"2023-03-29T10:57:31.853118+0000","last_undegraded":"2023-03-29T10:57:31.853118+0000","last_fullsized":"2023-03-29T10:57:31.853118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:01:58.615750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854397+0000","last_change":"2023-03-29T10:57:27.861054+0000","last_active":"2023-03-29T10:57:31.854397+0000","last_peered":"2023-03-29T10:57:31.854397+0000","last_clean":"2023-03-29T10:57:31.854397+0000","last_became_active":"2023-03-29T10:57:27.859547+0000","last_became_peered":"2023-03-29T10:57:27.859547+0000","last_unstale":"2023-03-29T10:57:31.854397+0000","last_undegraded":"2023-03-29T10:57:31.854397+0000","last_fullsized":"2023-03-29T10:57:31.854397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:24:00.034416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854082+0000","last_change":"2023-03-29T10:57:27.855728+0000","last_active":"2023-03-29T10:57:31.854082+0000","last_peered":"2023-03-29T10:57:31.854082+0000","last_clean":"2023-03-29T10:57:31.854082+0000","last_became_active":"2023-03-29T10:57:27.855438+0000","last_became_peered":"2023-03-29T10:57:27.855438+0000","last_unstale":"2023-03-29T10:57:31.854082+0000","last_undegraded":"2023-03-29T10:57:31.854082+0000","last_fullsized":"2023-03-29T10:57:31.854082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:33:37.107795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852905+0000","last_change":"2023-03-29T10:57:27.861795+0000","last_active":"2023-03-29T10:57:31.852905+0000","last_peered":"2023-03-29T10:57:31.852905+0000","last_clean":"2023-03-29T10:57:31.852905+0000","last_became_active":"2023-03-29T10:57:27.860751+0000","last_became_peered":"2023-03-29T10:57:27.860751+0000","last_unstale":"2023-03-29T10:57:31.852905+0000","last_undegraded":"2023-03-29T10:57:31.852905+0000","last_fullsized":"2023-03-29T10:57:31.852905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:41.882164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847170+0000","last_change":"2023-03-29T10:57:27.855143+0000","last_active":"2023-03-29T10:57:31.847170+0000","last_peered":"2023-03-29T10:57:31.847170+0000","last_clean":"2023-03-29T10:57:31.847170+0000","last_became_active":"2023-03-29T10:57:27.854804+0000","last_became_peered":"2023-03-29T10:57:27.854804+0000","last_unstale":"2023-03-29T10:57:31.847170+0000","last_undegraded":"2023-03-29T10:57:31.847170+0000","last_fullsized":"2023-03-29T10:57:31.847170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:24:08.253556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861416+0000","last_change":"2023-03-29T10:57:27.864339+0000","last_active":"2023-03-29T10:57:31.861416+0000","last_peered":"2023-03-29T10:57:31.861416+0000","last_clean":"2023-03-29T10:57:31.861416+0000","last_became_active":"2023-03-29T10:57:27.864034+0000","last_became_peered":"2023-03-29T10:57:27.864034+0000","last_unstale":"2023-03-29T10:57:31.861416+0000","last_undegraded":"2023-03-29T10:57:31.861416+0000","last_fullsized":"2023-03-29T10:57:31.861416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:10:11.279944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851971+0000","last_change":"2023-03-29T10:57:27.849118+0000","last_active":"2023-03-29T10:57:31.851971+0000","last_peered":"2023-03-29T10:57:31.851971+0000","last_clean":"2023-03-29T10:57:31.851971+0000","last_became_active":"2023-03-29T10:57:27.848724+0000","last_became_peered":"2023-03-29T10:57:27.848724+0000","last_unstale":"2023-03-29T10:57:31.851971+0000","last_undegraded":"2023-03-29T10:57:31.851971+0000","last_fullsized":"2023-03-29T10:57:31.851971+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:36:09.200571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.862264+0000","last_change":"2023-03-29T10:57:27.848373+0000","last_active":"2023-03-29T10:57:31.862264+0000","last_peered":"2023-03-29T10:57:31.862264+0000","last_clean":"2023-03-29T10:57:31.862264+0000","last_became_active":"2023-03-29T10:57:27.847792+0000","last_became_peered":"2023-03-29T10:57:27.847792+0000","last_unstale":"2023-03-29T10:57:31.862264+0000","last_undegraded":"2023-03-29T10:57:31.862264+0000","last_fullsized":"2023-03-29T10:57:31.862264+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:46:29.403965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853207+0000","last_change":"2023-03-29T10:57:27.864182+0000","last_active":"2023-03-29T10:57:31.853207+0000","last_peered":"2023-03-29T10:57:31.853207+0000","last_clean":"2023-03-29T10:57:31.853207+0000","last_became_active":"2023-03-29T10:57:27.863567+0000","last_became_peered":"2023-03-29T10:57:27.863567+0000","last_unstale":"2023-03-29T10:57:31.853207+0000","last_undegraded":"2023-03-29T10:57:31.853207+0000","last_fullsized":"2023-03-29T10:57:31.853207+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:52:36.730410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.845640+0000","last_change":"2023-03-29T10:57:27.863745+0000","last_active":"2023-03-29T10:57:31.845640+0000","last_peered":"2023-03-29T10:57:31.845640+0000","last_clean":"2023-03-29T10:57:31.845640+0000","last_became_active":"2023-03-29T10:57:27.863377+0000","last_became_peered":"2023-03-29T10:57:27.863377+0000","last_unstale":"2023-03-29T10:57:31.845640+0000","last_undegraded":"2023-03-29T10:57:31.845640+0000","last_fullsized":"2023-03-29T10:57:31.845640+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:53:39.143611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846141+0000","last_change":"2023-03-29T10:57:27.856601+0000","last_active":"2023-03-29T10:57:31.846141+0000","last_peered":"2023-03-29T10:57:31.846141+0000","last_clean":"2023-03-29T10:57:31.846141+0000","last_became_active":"2023-03-29T10:57:27.856323+0000","last_became_peered":"2023-03-29T10:57:27.856323+0000","last_unstale":"2023-03-29T10:57:31.846141+0000","last_undegraded":"2023-03-29T10:57:31.846141+0000","last_fullsized":"2023-03-29T10:57:31.846141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:12:15.104266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.863135+0000","last_change":"2023-03-29T10:57:27.850723+0000","last_active":"2023-03-29T10:57:31.863135+0000","last_peered":"2023-03-29T10:57:31.863135+0000","last_clean":"2023-03-29T10:57:31.863135+0000","last_became_active":"2023-03-29T10:57:27.844480+0000","last_became_peered":"2023-03-29T10:57:27.844480+0000","last_unstale":"2023-03-29T10:57:31.863135+0000","last_undegraded":"2023-03-29T10:57:31.863135+0000","last_fullsized":"2023-03-29T10:57:31.863135+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:07:19.154573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847070+0000","last_change":"2023-03-29T10:57:27.855233+0000","last_active":"2023-03-29T10:57:31.847070+0000","last_peered":"2023-03-29T10:57:31.847070+0000","last_clean":"2023-03-29T10:57:31.847070+0000","last_became_active":"2023-03-29T10:57:27.854970+0000","last_became_peered":"2023-03-29T10:57:27.854970+0000","last_unstale":"2023-03-29T10:57:31.847070+0000","last_undegraded":"2023-03-29T10:57:31.847070+0000","last_fullsized":"2023-03-29T10:57:31.847070+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:46:29.403965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860076+0000","last_change":"2023-03-29T10:57:27.860037+0000","last_active":"2023-03-29T10:57:31.860076+0000","last_peered":"2023-03-29T10:57:31.860076+0000","last_clean":"2023-03-29T10:57:31.860076+0000","last_became_active":"2023-03-29T10:57:27.859764+0000","last_became_peered":"2023-03-29T10:57:27.859764+0000","last_unstale":"2023-03-29T10:57:31.860076+0000","last_undegraded":"2023-03-29T10:57:31.860076+0000","last_fullsized":"2023-03-29T10:57:31.860076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:25:48.508072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851770+0000","last_change":"2023-03-29T10:57:27.848952+0000","last_active":"2023-03-29T10:57:31.851770+0000","last_peered":"2023-03-29T10:57:31.851770+0000","last_clean":"2023-03-29T10:57:31.851770+0000","last_became_active":"2023-03-29T10:57:27.848390+0000","last_became_peered":"2023-03-29T10:57:27.848390+0000","last_unstale":"2023-03-29T10:57:31.851770+0000","last_undegraded":"2023-03-29T10:57:31.851770+0000","last_fullsized":"2023-03-29T10:57:31.851770+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:07:19.154573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854647+0000","last_change":"2023-03-29T10:57:27.862653+0000","last_active":"2023-03-29T10:57:31.854647+0000","last_peered":"2023-03-29T10:57:31.854647+0000","last_clean":"2023-03-29T10:57:31.854647+0000","last_became_active":"2023-03-29T10:57:27.862108+0000","last_became_peered":"2023-03-29T10:57:27.862108+0000","last_unstale":"2023-03-29T10:57:31.854647+0000","last_undegraded":"2023-03-29T10:57:31.854647+0000","last_fullsized":"2023-03-29T10:57:31.854647+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:06.878410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846508+0000","last_change":"2023-03-29T10:57:27.859887+0000","last_active":"2023-03-29T10:57:31.846508+0000","last_peered":"2023-03-29T10:57:31.846508+0000","last_clean":"2023-03-29T10:57:31.846508+0000","last_became_active":"2023-03-29T10:57:27.859632+0000","last_became_peered":"2023-03-29T10:57:27.859632+0000","last_unstale":"2023-03-29T10:57:31.846508+0000","last_undegraded":"2023-03-29T10:57:31.846508+0000","last_fullsized":"2023-03-29T10:57:31.846508+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:17.321564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854180+0000","last_change":"2023-03-29T10:57:27.865228+0000","last_active":"2023-03-29T10:57:31.854180+0000","last_peered":"2023-03-29T10:57:31.854180+0000","last_clean":"2023-03-29T10:57:31.854180+0000","last_became_active":"2023-03-29T10:57:27.864919+0000","last_became_peered":"2023-03-29T10:57:27.864919+0000","last_unstale":"2023-03-29T10:57:31.854180+0000","last_undegraded":"2023-03-29T10:57:31.854180+0000","last_fullsized":"2023-03-29T10:57:31.854180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:44:47.553332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846978+0000","last_change":"2023-03-29T10:57:27.853377+0000","last_active":"2023-03-29T10:57:31.846978+0000","last_peered":"2023-03-29T10:57:31.846978+0000","last_clean":"2023-03-29T10:57:31.846978+0000","last_became_active":"2023-03-29T10:57:27.853039+0000","last_became_peered":"2023-03-29T10:57:27.853039+0000","last_unstale":"2023-03-29T10:57:31.846978+0000","last_undegraded":"2023-03-29T10:57:31.846978+0000","last_fullsized":"2023-03-29T10:57:31.846978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:23:24.586546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851362+0000","last_change":"2023-03-29T10:57:27.849931+0000","last_active":"2023-03-29T10:57:31.851362+0000","last_peered":"2023-03-29T10:57:31.851362+0000","last_clean":"2023-03-29T10:57:31.851362+0000","last_became_active":"2023-03-29T10:57:27.849662+0000","last_became_peered":"2023-03-29T10:57:27.849662+0000","last_unstale":"2023-03-29T10:57:31.851362+0000","last_undegraded":"2023-03-29T10:57:31.851362+0000","last_fullsized":"2023-03-29T10:57:31.851362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:25:48.508072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852622+0000","last_change":"2023-03-29T10:57:27.850427+0000","last_active":"2023-03-29T10:57:31.852622+0000","last_peered":"2023-03-29T10:57:31.852622+0000","last_clean":"2023-03-29T10:57:31.852622+0000","last_became_active":"2023-03-29T10:57:27.850140+0000","last_became_peered":"2023-03-29T10:57:27.850140+0000","last_unstale":"2023-03-29T10:57:31.852622+0000","last_undegraded":"2023-03-29T10:57:31.852622+0000","last_fullsized":"2023-03-29T10:57:31.852622+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:01:58.615750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847710+0000","last_change":"2023-03-29T10:57:27.854437+0000","last_active":"2023-03-29T10:57:31.847710+0000","last_peered":"2023-03-29T10:57:31.847710+0000","last_clean":"2023-03-29T10:57:31.847710+0000","last_became_active":"2023-03-29T10:57:27.853876+0000","last_became_peered":"2023-03-29T10:57:27.853876+0000","last_unstale":"2023-03-29T10:57:31.847710+0000","last_undegraded":"2023-03-29T10:57:31.847710+0000","last_fullsized":"2023-03-29T10:57:31.847710+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:33:37.107795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851569+0000","last_change":"2023-03-29T10:57:27.856343+0000","last_active":"2023-03-29T10:57:31.851569+0000","last_peered":"2023-03-29T10:57:31.851569+0000","last_clean":"2023-03-29T10:57:31.851569+0000","last_became_active":"2023-03-29T10:57:27.855843+0000","last_became_peered":"2023-03-29T10:57:27.855843+0000","last_unstale":"2023-03-29T10:57:31.851569+0000","last_undegraded":"2023-03-29T10:57:31.851569+0000","last_fullsized":"2023-03-29T10:57:31.851569+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:46:29.403965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852481+0000","last_change":"2023-03-29T10:57:27.846853+0000","last_active":"2023-03-29T10:57:31.852481+0000","last_peered":"2023-03-29T10:57:31.852481+0000","last_clean":"2023-03-29T10:57:31.852481+0000","last_became_active":"2023-03-29T10:57:27.846503+0000","last_became_peered":"2023-03-29T10:57:27.846503+0000","last_unstale":"2023-03-29T10:57:31.852481+0000","last_undegraded":"2023-03-29T10:57:31.852481+0000","last_fullsized":"2023-03-29T10:57:31.852481+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:37:35.403032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.864802+0000","last_change":"2023-03-29T10:57:27.850277+0000","last_active":"2023-03-29T10:57:31.864802+0000","last_peered":"2023-03-29T10:57:31.864802+0000","last_clean":"2023-03-29T10:57:31.864802+0000","last_became_active":"2023-03-29T10:57:27.849726+0000","last_became_peered":"2023-03-29T10:57:27.849726+0000","last_unstale":"2023-03-29T10:57:31.864802+0000","last_undegraded":"2023-03-29T10:57:31.864802+0000","last_fullsized":"2023-03-29T10:57:31.864802+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:25:48.508072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340375+0000","last_change":"2023-03-29T10:57:31.874510+0000","last_active":"2023-03-29T10:57:32.340375+0000","last_peered":"2023-03-29T10:57:32.340375+0000","last_clean":"2023-03-29T10:57:32.340375+0000","last_became_active":"2023-03-29T10:57:31.874055+0000","last_became_peered":"2023-03-29T10:57:31.874055+0000","last_unstale":"2023-03-29T10:57:32.340375+0000","last_undegraded":"2023-03-29T10:57:32.340375+0000","last_fullsized":"2023-03-29T10:57:32.340375+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:53:20.999237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852831+0000","last_change":"2023-03-29T10:57:27.366688+0000","last_active":"2023-03-29T10:57:31.852831+0000","last_peered":"2023-03-29T10:57:31.852831+0000","last_clean":"2023-03-29T10:57:31.852831+0000","last_became_active":"2023-03-29T10:57:27.366396+0000","last_became_peered":"2023-03-29T10:57:27.366396+0000","last_unstale":"2023-03-29T10:57:31.852831+0000","last_undegraded":"2023-03-29T10:57:31.852831+0000","last_fullsized":"2023-03-29T10:57:31.852831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:16.312585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853217+0000","last_change":"2023-03-29T10:57:27.851695+0000","last_active":"2023-03-29T10:57:31.853217+0000","last_peered":"2023-03-29T10:57:31.853217+0000","last_clean":"2023-03-29T10:57:31.853217+0000","last_became_active":"2023-03-29T10:57:27.851344+0000","last_became_peered":"2023-03-29T10:57:27.851344+0000","last_unstale":"2023-03-29T10:57:31.853217+0000","last_undegraded":"2023-03-29T10:57:31.853217+0000","last_fullsized":"2023-03-29T10:57:31.853217+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:53:38.186916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606792+0000","last_change":"2023-03-29T10:57:31.878011+0000","last_active":"2023-03-29T10:57:32.606792+0000","last_peered":"2023-03-29T10:57:32.606792+0000","last_clean":"2023-03-29T10:57:32.606792+0000","last_became_active":"2023-03-29T10:57:31.877524+0000","last_became_peered":"2023-03-29T10:57:31.877524+0000","last_unstale":"2023-03-29T10:57:32.606792+0000","last_undegraded":"2023-03-29T10:57:32.606792+0000","last_fullsized":"2023-03-29T10:57:32.606792+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:44:51.056937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.850681+0000","last_change":"2023-03-29T10:57:27.368907+0000","last_active":"2023-03-29T10:57:31.850681+0000","last_peered":"2023-03-29T10:57:31.850681+0000","last_clean":"2023-03-29T10:57:31.850681+0000","last_became_active":"2023-03-29T10:57:27.367137+0000","last_became_peered":"2023-03-29T10:57:27.367137+0000","last_unstale":"2023-03-29T10:57:31.850681+0000","last_undegraded":"2023-03-29T10:57:31.850681+0000","last_fullsized":"2023-03-29T10:57:31.850681+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:12:14.095287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853885+0000","last_change":"2023-03-29T10:57:27.844281+0000","last_active":"2023-03-29T10:57:31.853885+0000","last_peered":"2023-03-29T10:57:31.853885+0000","last_clean":"2023-03-29T10:57:31.853885+0000","last_became_active":"2023-03-29T10:57:27.844012+0000","last_became_peered":"2023-03-29T10:57:27.844012+0000","last_unstale":"2023-03-29T10:57:31.853885+0000","last_undegraded":"2023-03-29T10:57:31.853885+0000","last_fullsized":"2023-03-29T10:57:31.853885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:23:24.586546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.336612+0000","last_change":"2023-03-29T10:57:31.872397+0000","last_active":"2023-03-29T10:57:32.336612+0000","last_peered":"2023-03-29T10:57:32.336612+0000","last_clean":"2023-03-29T10:57:32.336612+0000","last_became_active":"2023-03-29T10:57:31.872073+0000","last_became_peered":"2023-03-29T10:57:31.872073+0000","last_unstale":"2023-03-29T10:57:32.336612+0000","last_undegraded":"2023-03-29T10:57:32.336612+0000","last_fullsized":"2023-03-29T10:57:32.336612+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:24:14.450310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849730+0000","last_change":"2023-03-29T10:57:27.362811+0000","last_active":"2023-03-29T10:57:31.849730+0000","last_peered":"2023-03-29T10:57:31.849730+0000","last_clean":"2023-03-29T10:57:31.849730+0000","last_became_active":"2023-03-29T10:57:27.362313+0000","last_became_peered":"2023-03-29T10:57:27.362313+0000","last_unstale":"2023-03-29T10:57:31.849730+0000","last_undegraded":"2023-03-29T10:57:31.849730+0000","last_fullsized":"2023-03-29T10:57:31.849730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:57:18.786945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.857696+0000","last_change":"2023-03-29T10:57:27.853328+0000","last_active":"2023-03-29T10:57:31.857696+0000","last_peered":"2023-03-29T10:57:31.857696+0000","last_clean":"2023-03-29T10:57:31.857696+0000","last_became_active":"2023-03-29T10:57:27.852980+0000","last_became_peered":"2023-03-29T10:57:27.852980+0000","last_unstale":"2023-03-29T10:57:31.857696+0000","last_undegraded":"2023-03-29T10:57:31.857696+0000","last_fullsized":"2023-03-29T10:57:31.857696+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:35:25.612159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606385+0000","last_change":"2023-03-29T10:57:31.880406+0000","last_active":"2023-03-29T10:57:32.606385+0000","last_peered":"2023-03-29T10:57:32.606385+0000","last_clean":"2023-03-29T10:57:32.606385+0000","last_became_active":"2023-03-29T10:57:31.879910+0000","last_became_peered":"2023-03-29T10:57:31.879910+0000","last_unstale":"2023-03-29T10:57:32.606385+0000","last_undegraded":"2023-03-29T10:57:32.606385+0000","last_fullsized":"2023-03-29T10:57:32.606385+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:50:03.793655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852066+0000","last_change":"2023-03-29T10:57:27.361203+0000","last_active":"2023-03-29T10:57:31.852066+0000","last_peered":"2023-03-29T10:57:31.852066+0000","last_clean":"2023-03-29T10:57:31.852066+0000","last_became_active":"2023-03-29T10:57:27.360905+0000","last_became_peered":"2023-03-29T10:57:27.360905+0000","last_unstale":"2023-03-29T10:57:31.852066+0000","last_undegraded":"2023-03-29T10:57:31.852066+0000","last_fullsized":"2023-03-29T10:57:31.852066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:53:38.134633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849652+0000","last_change":"2023-03-29T10:57:27.848599+0000","last_active":"2023-03-29T10:57:31.849652+0000","last_peered":"2023-03-29T10:57:31.849652+0000","last_clean":"2023-03-29T10:57:31.849652+0000","last_became_active":"2023-03-29T10:57:27.848300+0000","last_became_peered":"2023-03-29T10:57:27.848300+0000","last_unstale":"2023-03-29T10:57:31.849652+0000","last_undegraded":"2023-03-29T10:57:31.849652+0000","last_fullsized":"2023-03-29T10:57:31.849652+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:44:47.553332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340886+0000","last_change":"2023-03-29T10:57:31.875820+0000","last_active":"2023-03-29T10:57:32.340886+0000","last_peered":"2023-03-29T10:57:32.340886+0000","last_clean":"2023-03-29T10:57:32.340886+0000","last_became_active":"2023-03-29T10:57:31.875285+0000","last_became_peered":"2023-03-29T10:57:31.875285+0000","last_unstale":"2023-03-29T10:57:32.340886+0000","last_undegraded":"2023-03-29T10:57:32.340886+0000","last_fullsized":"2023-03-29T10:57:32.340886+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:05:02.661508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.862158+0000","last_change":"2023-03-29T10:57:27.369635+0000","last_active":"2023-03-29T10:57:31.862158+0000","last_peered":"2023-03-29T10:57:31.862158+0000","last_clean":"2023-03-29T10:57:31.862158+0000","last_became_active":"2023-03-29T10:57:27.369232+0000","last_became_peered":"2023-03-29T10:57:27.369232+0000","last_unstale":"2023-03-29T10:57:31.862158+0000","last_undegraded":"2023-03-29T10:57:31.862158+0000","last_fullsized":"2023-03-29T10:57:31.862158+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:45:12.326563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852705+0000","last_change":"2023-03-29T10:57:27.841973+0000","last_active":"2023-03-29T10:57:31.852705+0000","last_peered":"2023-03-29T10:57:31.852705+0000","last_clean":"2023-03-29T10:57:31.852705+0000","last_became_active":"2023-03-29T10:57:27.841668+0000","last_became_peered":"2023-03-29T10:57:27.841668+0000","last_unstale":"2023-03-29T10:57:31.852705+0000","last_undegraded":"2023-03-29T10:57:31.852705+0000","last_fullsized":"2023-03-29T10:57:31.852705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:25:48.508072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.608208+0000","last_change":"2023-03-29T10:57:31.867435+0000","last_active":"2023-03-29T10:57:32.608208+0000","last_peered":"2023-03-29T10:57:32.608208+0000","last_clean":"2023-03-29T10:57:32.608208+0000","last_became_active":"2023-03-29T10:57:31.867024+0000","last_became_peered":"2023-03-29T10:57:31.867024+0000","last_unstale":"2023-03-29T10:57:32.608208+0000","last_undegraded":"2023-03-29T10:57:32.608208+0000","last_fullsized":"2023-03-29T10:57:32.608208+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:33:59.083704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846915+0000","last_change":"2023-03-29T10:57:27.367580+0000","last_active":"2023-03-29T10:57:31.846915+0000","last_peered":"2023-03-29T10:57:31.846915+0000","last_clean":"2023-03-29T10:57:31.846915+0000","last_became_active":"2023-03-29T10:57:27.367297+0000","last_became_peered":"2023-03-29T10:57:27.367297+0000","last_unstale":"2023-03-29T10:57:31.846915+0000","last_undegraded":"2023-03-29T10:57:31.846915+0000","last_fullsized":"2023-03-29T10:57:31.846915+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:35:24.603180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851381+0000","last_change":"2023-03-29T10:57:27.868674+0000","last_active":"2023-03-29T10:57:31.851381+0000","last_peered":"2023-03-29T10:57:31.851381+0000","last_clean":"2023-03-29T10:57:31.851381+0000","last_became_active":"2023-03-29T10:57:27.868480+0000","last_became_peered":"2023-03-29T10:57:27.868480+0000","last_unstale":"2023-03-29T10:57:31.851381+0000","last_undegraded":"2023-03-29T10:57:31.851381+0000","last_fullsized":"2023-03-29T10:57:31.851381+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:24:08.253556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.543504+0000","last_change":"2023-03-29T10:57:31.865358+0000","last_active":"2023-03-29T10:57:32.543504+0000","last_peered":"2023-03-29T10:57:32.543504+0000","last_clean":"2023-03-29T10:57:32.543504+0000","last_became_active":"2023-03-29T10:57:31.864834+0000","last_became_peered":"2023-03-29T10:57:31.864834+0000","last_unstale":"2023-03-29T10:57:32.543504+0000","last_undegraded":"2023-03-29T10:57:32.543504+0000","last_fullsized":"2023-03-29T10:57:32.543504+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:40:55.333034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:44.820956+0000","last_change":"2023-03-29T10:57:27.361696+0000","last_active":"2023-03-29T10:57:44.820956+0000","last_peered":"2023-03-29T10:57:44.820956+0000","last_clean":"2023-03-29T10:57:44.820956+0000","last_became_active":"2023-03-29T10:57:27.360021+0000","last_became_peered":"2023-03-29T10:57:27.360021+0000","last_unstale":"2023-03-29T10:57:44.820956+0000","last_undegraded":"2023-03-29T10:57:44.820956+0000","last_fullsized":"2023-03-29T10:57:44.820956+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:57:18.786945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853468+0000","last_change":"2023-03-29T10:57:27.860824+0000","last_active":"2023-03-29T10:57:31.853468+0000","last_peered":"2023-03-29T10:57:31.853468+0000","last_clean":"2023-03-29T10:57:31.853468+0000","last_became_active":"2023-03-29T10:57:27.860139+0000","last_became_peered":"2023-03-29T10:57:27.860139+0000","last_unstale":"2023-03-29T10:57:31.853468+0000","last_undegraded":"2023-03-29T10:57:31.853468+0000","last_fullsized":"2023-03-29T10:57:31.853468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:10:11.279944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606214+0000","last_change":"2023-03-29T10:57:31.869234+0000","last_active":"2023-03-29T10:57:32.606214+0000","last_peered":"2023-03-29T10:57:32.606214+0000","last_clean":"2023-03-29T10:57:32.606214+0000","last_became_active":"2023-03-29T10:57:31.868479+0000","last_became_peered":"2023-03-29T10:57:31.868479+0000","last_unstale":"2023-03-29T10:57:32.606214+0000","last_undegraded":"2023-03-29T10:57:32.606214+0000","last_fullsized":"2023-03-29T10:57:32.606214+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:29:07.043419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846875+0000","last_change":"2023-03-29T10:57:27.363811+0000","last_active":"2023-03-29T10:57:31.846875+0000","last_peered":"2023-03-29T10:57:31.846875+0000","last_clean":"2023-03-29T10:57:31.846875+0000","last_became_active":"2023-03-29T10:57:27.362755+0000","last_became_peered":"2023-03-29T10:57:27.362755+0000","last_unstale":"2023-03-29T10:57:31.846875+0000","last_undegraded":"2023-03-29T10:57:31.846875+0000","last_fullsized":"2023-03-29T10:57:31.846875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:45:12.326563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849249+0000","last_change":"2023-03-29T10:57:27.848024+0000","last_active":"2023-03-29T10:57:31.849249+0000","last_peered":"2023-03-29T10:57:31.849249+0000","last_clean":"2023-03-29T10:57:31.849249+0000","last_became_active":"2023-03-29T10:57:27.847742+0000","last_became_peered":"2023-03-29T10:57:27.847742+0000","last_unstale":"2023-03-29T10:57:31.849249+0000","last_undegraded":"2023-03-29T10:57:31.849249+0000","last_fullsized":"2023-03-29T10:57:31.849249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:07:19.154573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.608195+0000","last_change":"2023-03-29T10:57:31.874176+0000","last_active":"2023-03-29T10:57:32.608195+0000","last_peered":"2023-03-29T10:57:32.608195+0000","last_clean":"2023-03-29T10:57:32.608195+0000","last_became_active":"2023-03-29T10:57:31.873748+0000","last_became_peered":"2023-03-29T10:57:31.873748+0000","last_unstale":"2023-03-29T10:57:32.608195+0000","last_undegraded":"2023-03-29T10:57:32.608195+0000","last_fullsized":"2023-03-29T10:57:32.608195+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:29:07.043419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851278+0000","last_change":"2023-03-29T10:57:27.353170+0000","last_active":"2023-03-29T10:57:31.851278+0000","last_peered":"2023-03-29T10:57:31.851278+0000","last_clean":"2023-03-29T10:57:31.851278+0000","last_became_active":"2023-03-29T10:57:27.352891+0000","last_became_peered":"2023-03-29T10:57:27.352891+0000","last_unstale":"2023-03-29T10:57:31.851278+0000","last_undegraded":"2023-03-29T10:57:31.851278+0000","last_fullsized":"2023-03-29T10:57:31.851278+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:42:03.146529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.864725+0000","last_change":"2023-03-29T10:57:27.850083+0000","last_active":"2023-03-29T10:57:31.864725+0000","last_peered":"2023-03-29T10:57:31.864725+0000","last_clean":"2023-03-29T10:57:31.864725+0000","last_became_active":"2023-03-29T10:57:27.849341+0000","last_became_peered":"2023-03-29T10:57:27.849341+0000","last_unstale":"2023-03-29T10:57:31.864725+0000","last_undegraded":"2023-03-29T10:57:31.864725+0000","last_fullsized":"2023-03-29T10:57:31.864725+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:41.882164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.608429+0000","last_change":"2023-03-29T10:57:31.874626+0000","last_active":"2023-03-29T10:57:32.608429+0000","last_peered":"2023-03-29T10:57:32.608429+0000","last_clean":"2023-03-29T10:57:32.608429+0000","last_became_active":"2023-03-29T10:57:31.874288+0000","last_became_peered":"2023-03-29T10:57:31.874288+0000","last_unstale":"2023-03-29T10:57:32.608429+0000","last_undegraded":"2023-03-29T10:57:32.608429+0000","last_fullsized":"2023-03-29T10:57:32.608429+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:10:08.859801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847314+0000","last_change":"2023-03-29T10:57:27.371046+0000","last_active":"2023-03-29T10:57:31.847314+0000","last_peered":"2023-03-29T10:57:31.847314+0000","last_clean":"2023-03-29T10:57:31.847314+0000","last_became_active":"2023-03-29T10:57:27.370703+0000","last_became_peered":"2023-03-29T10:57:27.370703+0000","last_unstale":"2023-03-29T10:57:31.847314+0000","last_undegraded":"2023-03-29T10:57:31.847314+0000","last_fullsized":"2023-03-29T10:57:31.847314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:00:38.166723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853955+0000","last_change":"2023-03-29T10:57:27.857805+0000","last_active":"2023-03-29T10:57:31.853955+0000","last_peered":"2023-03-29T10:57:31.853955+0000","last_clean":"2023-03-29T10:57:31.853955+0000","last_became_active":"2023-03-29T10:57:27.857520+0000","last_became_peered":"2023-03-29T10:57:27.857520+0000","last_unstale":"2023-03-29T10:57:31.853955+0000","last_undegraded":"2023-03-29T10:57:31.853955+0000","last_fullsized":"2023-03-29T10:57:31.853955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:24:08.253556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605664+0000","last_change":"2023-03-29T10:57:31.868093+0000","last_active":"2023-03-29T10:57:32.605664+0000","last_peered":"2023-03-29T10:57:32.605664+0000","last_clean":"2023-03-29T10:57:32.605664+0000","last_became_active":"2023-03-29T10:57:31.867516+0000","last_became_peered":"2023-03-29T10:57:31.867516+0000","last_unstale":"2023-03-29T10:57:32.605664+0000","last_undegraded":"2023-03-29T10:57:32.605664+0000","last_fullsized":"2023-03-29T10:57:32.605664+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:56:12.713461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852187+0000","last_change":"2023-03-29T10:57:27.366626+0000","last_active":"2023-03-29T10:57:31.852187+0000","last_peered":"2023-03-29T10:57:31.852187+0000","last_clean":"2023-03-29T10:57:31.852187+0000","last_became_active":"2023-03-29T10:57:27.366259+0000","last_became_peered":"2023-03-29T10:57:27.366259+0000","last_unstale":"2023-03-29T10:57:31.852187+0000","last_undegraded":"2023-03-29T10:57:31.852187+0000","last_fullsized":"2023-03-29T10:57:31.852187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:01:05.024349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852636+0000","last_change":"2023-03-29T10:57:27.842305+0000","last_active":"2023-03-29T10:57:31.852636+0000","last_peered":"2023-03-29T10:57:31.852636+0000","last_clean":"2023-03-29T10:57:31.852636+0000","last_became_active":"2023-03-29T10:57:27.841993+0000","last_became_peered":"2023-03-29T10:57:27.841993+0000","last_unstale":"2023-03-29T10:57:31.852636+0000","last_undegraded":"2023-03-29T10:57:31.852636+0000","last_fullsized":"2023-03-29T10:57:31.852636+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:41.882164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340179+0000","last_change":"2023-03-29T10:57:31.874901+0000","last_active":"2023-03-29T10:57:32.340179+0000","last_peered":"2023-03-29T10:57:32.340179+0000","last_clean":"2023-03-29T10:57:32.340179+0000","last_became_active":"2023-03-29T10:57:31.874412+0000","last_became_peered":"2023-03-29T10:57:31.874412+0000","last_unstale":"2023-03-29T10:57:32.340179+0000","last_undegraded":"2023-03-29T10:57:32.340179+0000","last_fullsized":"2023-03-29T10:57:32.340179+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:19:12.936732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847695+0000","last_change":"2023-03-29T10:57:27.363942+0000","last_active":"2023-03-29T10:57:31.847695+0000","last_peered":"2023-03-29T10:57:31.847695+0000","last_clean":"2023-03-29T10:57:31.847695+0000","last_became_active":"2023-03-29T10:57:27.363509+0000","last_became_peered":"2023-03-29T10:57:27.363509+0000","last_unstale":"2023-03-29T10:57:31.847695+0000","last_undegraded":"2023-03-29T10:57:31.847695+0000","last_fullsized":"2023-03-29T10:57:31.847695+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:23:59.025438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861310+0000","last_change":"2023-03-29T10:57:27.849632+0000","last_active":"2023-03-29T10:57:31.861310+0000","last_peered":"2023-03-29T10:57:31.861310+0000","last_clean":"2023-03-29T10:57:31.861310+0000","last_became_active":"2023-03-29T10:57:27.849341+0000","last_became_peered":"2023-03-29T10:57:27.849341+0000","last_unstale":"2023-03-29T10:57:31.861310+0000","last_undegraded":"2023-03-29T10:57:31.861310+0000","last_fullsized":"2023-03-29T10:57:31.861310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:01:58.615750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.604803+0000","last_change":"2023-03-29T10:57:31.864416+0000","last_active":"2023-03-29T10:57:32.604803+0000","last_peered":"2023-03-29T10:57:32.604803+0000","last_clean":"2023-03-29T10:57:32.604803+0000","last_became_active":"2023-03-29T10:57:31.864021+0000","last_became_peered":"2023-03-29T10:57:31.864021+0000","last_unstale":"2023-03-29T10:57:32.604803+0000","last_undegraded":"2023-03-29T10:57:32.604803+0000","last_fullsized":"2023-03-29T10:57:32.604803+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:44:51.056937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847268+0000","last_change":"2023-03-29T10:57:27.365113+0000","last_active":"2023-03-29T10:57:31.847268+0000","last_peered":"2023-03-29T10:57:31.847268+0000","last_clean":"2023-03-29T10:57:31.847268+0000","last_became_active":"2023-03-29T10:57:27.363593+0000","last_became_peered":"2023-03-29T10:57:27.363593+0000","last_unstale":"2023-03-29T10:57:31.847268+0000","last_undegraded":"2023-03-29T10:57:31.847268+0000","last_fullsized":"2023-03-29T10:57:31.847268+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:10:38.094232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860900+0000","last_change":"2023-03-29T10:57:27.861487+0000","last_active":"2023-03-29T10:57:31.860900+0000","last_peered":"2023-03-29T10:57:31.860900+0000","last_clean":"2023-03-29T10:57:31.860900+0000","last_became_active":"2023-03-29T10:57:27.861218+0000","last_became_peered":"2023-03-29T10:57:27.861218+0000","last_unstale":"2023-03-29T10:57:31.860900+0000","last_undegraded":"2023-03-29T10:57:31.860900+0000","last_fullsized":"2023-03-29T10:57:31.860900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:04:59.157904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.604937+0000","last_change":"2023-03-29T10:57:31.871267+0000","last_active":"2023-03-29T10:57:32.604937+0000","last_peered":"2023-03-29T10:57:32.604937+0000","last_clean":"2023-03-29T10:57:32.604937+0000","last_became_active":"2023-03-29T10:57:31.870814+0000","last_became_peered":"2023-03-29T10:57:31.870814+0000","last_unstale":"2023-03-29T10:57:32.604937+0000","last_undegraded":"2023-03-29T10:57:32.604937+0000","last_fullsized":"2023-03-29T10:57:32.604937+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:57:20.392149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847436+0000","last_change":"2023-03-29T10:57:27.367428+0000","last_active":"2023-03-29T10:57:31.847436+0000","last_peered":"2023-03-29T10:57:31.847436+0000","last_clean":"2023-03-29T10:57:31.847436+0000","last_became_active":"2023-03-29T10:57:27.367100+0000","last_became_peered":"2023-03-29T10:57:27.367100+0000","last_unstale":"2023-03-29T10:57:31.847436+0000","last_undegraded":"2023-03-29T10:57:31.847436+0000","last_fullsized":"2023-03-29T10:57:31.847436+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:52:59.436152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.858073+0000","last_change":"2023-03-29T10:57:27.863156+0000","last_active":"2023-03-29T10:57:31.858073+0000","last_peered":"2023-03-29T10:57:31.858073+0000","last_clean":"2023-03-29T10:57:31.858073+0000","last_became_active":"2023-03-29T10:57:27.862805+0000","last_became_peered":"2023-03-29T10:57:27.862805+0000","last_unstale":"2023-03-29T10:57:31.858073+0000","last_undegraded":"2023-03-29T10:57:31.858073+0000","last_fullsized":"2023-03-29T10:57:31.858073+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:32:05.407357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605019+0000","last_change":"2023-03-29T10:57:31.878616+0000","last_active":"2023-03-29T10:57:32.605019+0000","last_peered":"2023-03-29T10:57:32.605019+0000","last_clean":"2023-03-29T10:57:32.605019+0000","last_became_active":"2023-03-29T10:57:31.878248+0000","last_became_peered":"2023-03-29T10:57:31.878248+0000","last_unstale":"2023-03-29T10:57:32.605019+0000","last_undegraded":"2023-03-29T10:57:32.605019+0000","last_fullsized":"2023-03-29T10:57:32.605019+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:53:41.690520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861723+0000","last_change":"2023-03-29T10:57:27.370499+0000","last_active":"2023-03-29T10:57:31.861723+0000","last_peered":"2023-03-29T10:57:31.861723+0000","last_clean":"2023-03-29T10:57:31.861723+0000","last_became_active":"2023-03-29T10:57:27.370048+0000","last_became_peered":"2023-03-29T10:57:27.370048+0000","last_unstale":"2023-03-29T10:57:31.861723+0000","last_undegraded":"2023-03-29T10:57:31.861723+0000","last_fullsized":"2023-03-29T10:57:31.861723+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:52:59.436152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854114+0000","last_change":"2023-03-29T10:57:27.848004+0000","last_active":"2023-03-29T10:57:31.854114+0000","last_peered":"2023-03-29T10:57:31.854114+0000","last_clean":"2023-03-29T10:57:31.854114+0000","last_became_active":"2023-03-29T10:57:27.847731+0000","last_became_peered":"2023-03-29T10:57:27.847731+0000","last_unstale":"2023-03-29T10:57:31.854114+0000","last_undegraded":"2023-03-29T10:57:31.854114+0000","last_fullsized":"2023-03-29T10:57:31.854114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:41.882164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605202+0000","last_change":"2023-03-29T10:57:31.868289+0000","last_active":"2023-03-29T10:57:32.605202+0000","last_peered":"2023-03-29T10:57:32.605202+0000","last_clean":"2023-03-29T10:57:32.605202+0000","last_became_active":"2023-03-29T10:57:31.867905+0000","last_became_peered":"2023-03-29T10:57:31.867905+0000","last_unstale":"2023-03-29T10:57:32.605202+0000","last_undegraded":"2023-03-29T10:57:32.605202+0000","last_fullsized":"2023-03-29T10:57:32.605202+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:23:28.090151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851809+0000","last_change":"2023-03-29T10:57:27.368350+0000","last_active":"2023-03-29T10:57:31.851809+0000","last_peered":"2023-03-29T10:57:31.851809+0000","last_clean":"2023-03-29T10:57:31.851809+0000","last_became_active":"2023-03-29T10:57:27.367743+0000","last_became_peered":"2023-03-29T10:57:27.367743+0000","last_unstale":"2023-03-29T10:57:31.851809+0000","last_undegraded":"2023-03-29T10:57:31.851809+0000","last_fullsized":"2023-03-29T10:57:31.851809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:16.312585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853838+0000","last_change":"2023-03-29T10:57:27.862273+0000","last_active":"2023-03-29T10:57:31.853838+0000","last_peered":"2023-03-29T10:57:31.853838+0000","last_clean":"2023-03-29T10:57:31.853838+0000","last_became_active":"2023-03-29T10:57:27.861893+0000","last_became_peered":"2023-03-29T10:57:27.861893+0000","last_unstale":"2023-03-29T10:57:31.853838+0000","last_undegraded":"2023-03-29T10:57:31.853838+0000","last_fullsized":"2023-03-29T10:57:31.853838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:34:37.113840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.543869+0000","last_change":"2023-03-29T10:57:31.877007+0000","last_active":"2023-03-29T10:57:32.543869+0000","last_peered":"2023-03-29T10:57:32.543869+0000","last_clean":"2023-03-29T10:57:32.543869+0000","last_became_active":"2023-03-29T10:57:31.876649+0000","last_became_peered":"2023-03-29T10:57:31.876649+0000","last_unstale":"2023-03-29T10:57:32.543869+0000","last_undegraded":"2023-03-29T10:57:32.543869+0000","last_fullsized":"2023-03-29T10:57:32.543869+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:19:12.936732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.858178+0000","last_change":"2023-03-29T10:57:27.370469+0000","last_active":"2023-03-29T10:57:31.858178+0000","last_peered":"2023-03-29T10:57:31.858178+0000","last_clean":"2023-03-29T10:57:31.858178+0000","last_became_active":"2023-03-29T10:57:27.370194+0000","last_became_peered":"2023-03-29T10:57:27.370194+0000","last_unstale":"2023-03-29T10:57:31.858178+0000","last_undegraded":"2023-03-29T10:57:31.858178+0000","last_fullsized":"2023-03-29T10:57:31.858178+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:42:03.146529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853028+0000","last_change":"2023-03-29T10:57:27.863980+0000","last_active":"2023-03-29T10:57:31.853028+0000","last_peered":"2023-03-29T10:57:31.853028+0000","last_clean":"2023-03-29T10:57:31.853028+0000","last_became_active":"2023-03-29T10:57:27.863503+0000","last_became_peered":"2023-03-29T10:57:27.863503+0000","last_unstale":"2023-03-29T10:57:31.853028+0000","last_undegraded":"2023-03-29T10:57:31.853028+0000","last_fullsized":"2023-03-29T10:57:31.853028+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:36:09.200571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606340+0000","last_change":"2023-03-29T10:57:31.880859+0000","last_active":"2023-03-29T10:57:32.606340+0000","last_peered":"2023-03-29T10:57:32.606340+0000","last_clean":"2023-03-29T10:57:32.606340+0000","last_became_active":"2023-03-29T10:57:31.879872+0000","last_became_peered":"2023-03-29T10:57:31.879872+0000","last_unstale":"2023-03-29T10:57:32.606340+0000","last_undegraded":"2023-03-29T10:57:32.606340+0000","last_fullsized":"2023-03-29T10:57:32.606340+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:40:04.971172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860797+0000","last_change":"2023-03-29T10:57:27.359363+0000","last_active":"2023-03-29T10:57:31.860797+0000","last_peered":"2023-03-29T10:57:31.860797+0000","last_clean":"2023-03-29T10:57:31.860797+0000","last_became_active":"2023-03-29T10:57:27.359054+0000","last_became_peered":"2023-03-29T10:57:27.359054+0000","last_unstale":"2023-03-29T10:57:31.860797+0000","last_undegraded":"2023-03-29T10:57:31.860797+0000","last_fullsized":"2023-03-29T10:57:31.860797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:42:03.146529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854683+0000","last_change":"2023-03-29T10:57:27.862443+0000","last_active":"2023-03-29T10:57:31.854683+0000","last_peered":"2023-03-29T10:57:31.854683+0000","last_clean":"2023-03-29T10:57:31.854683+0000","last_became_active":"2023-03-29T10:57:27.861796+0000","last_became_peered":"2023-03-29T10:57:27.861796+0000","last_unstale":"2023-03-29T10:57:31.854683+0000","last_undegraded":"2023-03-29T10:57:31.854683+0000","last_fullsized":"2023-03-29T10:57:31.854683+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:50:00.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340645+0000","last_change":"2023-03-29T10:57:31.875674+0000","last_active":"2023-03-29T10:57:32.340645+0000","last_peered":"2023-03-29T10:57:32.340645+0000","last_clean":"2023-03-29T10:57:32.340645+0000","last_became_active":"2023-03-29T10:57:31.875240+0000","last_became_peered":"2023-03-29T10:57:31.875240+0000","last_unstale":"2023-03-29T10:57:32.340645+0000","last_undegraded":"2023-03-29T10:57:32.340645+0000","last_fullsized":"2023-03-29T10:57:32.340645+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:36:10.088665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849570+0000","last_change":"2023-03-29T10:57:27.366426+0000","last_active":"2023-03-29T10:57:31.849570+0000","last_peered":"2023-03-29T10:57:31.849570+0000","last_clean":"2023-03-29T10:57:31.849570+0000","last_became_active":"2023-03-29T10:57:27.366129+0000","last_became_peered":"2023-03-29T10:57:27.366129+0000","last_unstale":"2023-03-29T10:57:31.849570+0000","last_undegraded":"2023-03-29T10:57:31.849570+0000","last_fullsized":"2023-03-29T10:57:31.849570+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:42:03.146529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851728+0000","last_change":"2023-03-29T10:57:27.856138+0000","last_active":"2023-03-29T10:57:31.851728+0000","last_peered":"2023-03-29T10:57:31.851728+0000","last_clean":"2023-03-29T10:57:31.851728+0000","last_became_active":"2023-03-29T10:57:27.855392+0000","last_became_peered":"2023-03-29T10:57:27.855392+0000","last_unstale":"2023-03-29T10:57:31.851728+0000","last_undegraded":"2023-03-29T10:57:31.851728+0000","last_fullsized":"2023-03-29T10:57:31.851728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:12:10.758718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.336574+0000","last_change":"2023-03-29T10:57:31.872511+0000","last_active":"2023-03-29T10:57:32.336574+0000","last_peered":"2023-03-29T10:57:32.336574+0000","last_clean":"2023-03-29T10:57:32.336574+0000","last_became_active":"2023-03-29T10:57:31.872088+0000","last_became_peered":"2023-03-29T10:57:31.872088+0000","last_unstale":"2023-03-29T10:57:32.336574+0000","last_undegraded":"2023-03-29T10:57:32.336574+0000","last_fullsized":"2023-03-29T10:57:32.336574+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:28:41.336878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852880+0000","last_change":"2023-03-29T10:57:22.318583+0000","last_active":"2023-03-29T10:57:31.852880+0000","last_peered":"2023-03-29T10:57:31.852880+0000","last_clean":"2023-03-29T10:57:31.852880+0000","last_became_active":"2023-03-29T10:57:22.318223+0000","last_became_peered":"2023-03-29T10:57:22.318223+0000","last_unstale":"2023-03-29T10:57:31.852880+0000","last_undegraded":"2023-03-29T10:57:31.852880+0000","last_fullsized":"2023-03-29T10:57:31.852880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:31:03.349241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.864963+0000","last_change":"2023-03-29T10:57:27.369314+0000","last_active":"2023-03-29T10:57:31.864963+0000","last_peered":"2023-03-29T10:57:31.864963+0000","last_clean":"2023-03-29T10:57:31.864963+0000","last_became_active":"2023-03-29T10:57:27.368989+0000","last_became_peered":"2023-03-29T10:57:27.368989+0000","last_unstale":"2023-03-29T10:57:31.864963+0000","last_undegraded":"2023-03-29T10:57:31.864963+0000","last_fullsized":"2023-03-29T10:57:31.864963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:35:24.603180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851286+0000","last_change":"2023-03-29T10:57:27.860136+0000","last_active":"2023-03-29T10:57:31.851286+0000","last_peered":"2023-03-29T10:57:31.851286+0000","last_clean":"2023-03-29T10:57:31.851286+0000","last_became_active":"2023-03-29T10:57:27.859888+0000","last_became_peered":"2023-03-29T10:57:27.859888+0000","last_unstale":"2023-03-29T10:57:31.851286+0000","last_undegraded":"2023-03-29T10:57:31.851286+0000","last_fullsized":"2023-03-29T10:57:31.851286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:23:24.586546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.336910+0000","last_change":"2023-03-29T10:57:31.877812+0000","last_active":"2023-03-29T10:57:32.336910+0000","last_peered":"2023-03-29T10:57:32.336910+0000","last_clean":"2023-03-29T10:57:32.336910+0000","last_became_active":"2023-03-29T10:57:31.877432+0000","last_became_peered":"2023-03-29T10:57:31.877432+0000","last_unstale":"2023-03-29T10:57:32.336910+0000","last_undegraded":"2023-03-29T10:57:32.336910+0000","last_fullsized":"2023-03-29T10:57:32.336910+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:19:12.936732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.863218+0000","last_change":"2023-03-29T10:57:22.320330+0000","last_active":"2023-03-29T10:57:31.863218+0000","last_peered":"2023-03-29T10:57:31.863218+0000","last_clean":"2023-03-29T10:57:31.863218+0000","last_became_active":"2023-03-29T10:57:22.320011+0000","last_became_peered":"2023-03-29T10:57:22.320011+0000","last_unstale":"2023-03-29T10:57:31.863218+0000","last_undegraded":"2023-03-29T10:57:31.863218+0000","last_fullsized":"2023-03-29T10:57:31.863218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:47:01.099069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853293+0000","last_change":"2023-03-29T10:57:27.361265+0000","last_active":"2023-03-29T10:57:31.853293+0000","last_peered":"2023-03-29T10:57:31.853293+0000","last_clean":"2023-03-29T10:57:31.853293+0000","last_became_active":"2023-03-29T10:57:27.360017+0000","last_became_peered":"2023-03-29T10:57:27.360017+0000","last_unstale":"2023-03-29T10:57:31.853293+0000","last_undegraded":"2023-03-29T10:57:31.853293+0000","last_fullsized":"2023-03-29T10:57:31.853293+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:12:14.095287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852801+0000","last_change":"2023-03-29T10:57:27.846016+0000","last_active":"2023-03-29T10:57:31.852801+0000","last_peered":"2023-03-29T10:57:31.852801+0000","last_clean":"2023-03-29T10:57:31.852801+0000","last_became_active":"2023-03-29T10:57:27.845677+0000","last_became_peered":"2023-03-29T10:57:27.845677+0000","last_unstale":"2023-03-29T10:57:31.852801+0000","last_undegraded":"2023-03-29T10:57:31.852801+0000","last_fullsized":"2023-03-29T10:57:31.852801+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T10:58:34.425561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605463+0000","last_change":"2023-03-29T10:57:31.863426+0000","last_active":"2023-03-29T10:57:32.605463+0000","last_peered":"2023-03-29T10:57:32.605463+0000","last_clean":"2023-03-29T10:57:32.605463+0000","last_became_active":"2023-03-29T10:57:31.862992+0000","last_became_peered":"2023-03-29T10:57:31.862992+0000","last_unstale":"2023-03-29T10:57:32.605463+0000","last_undegraded":"2023-03-29T10:57:32.605463+0000","last_fullsized":"2023-03-29T10:57:32.605463+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:24:11.757160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.845511+0000","last_change":"2023-03-29T10:57:22.467148+0000","last_active":"2023-03-29T10:57:31.845511+0000","last_peered":"2023-03-29T10:57:31.845511+0000","last_clean":"2023-03-29T10:57:31.845511+0000","last_became_active":"2023-03-29T10:57:22.466822+0000","last_became_peered":"2023-03-29T10:57:22.466822+0000","last_unstale":"2023-03-29T10:57:31.845511+0000","last_undegraded":"2023-03-29T10:57:31.845511+0000","last_fullsized":"2023-03-29T10:57:31.845511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:53:23.007205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851898+0000","last_change":"2023-03-29T10:57:27.364233+0000","last_active":"2023-03-29T10:57:31.851898+0000","last_peered":"2023-03-29T10:57:31.851898+0000","last_clean":"2023-03-29T10:57:31.851898+0000","last_became_active":"2023-03-29T10:57:27.363992+0000","last_became_peered":"2023-03-29T10:57:27.363992+0000","last_unstale":"2023-03-29T10:57:31.851898+0000","last_undegraded":"2023-03-29T10:57:31.851898+0000","last_fullsized":"2023-03-29T10:57:31.851898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:57:18.786945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851648+0000","last_change":"2023-03-29T10:57:27.859088+0000","last_active":"2023-03-29T10:57:31.851648+0000","last_peered":"2023-03-29T10:57:31.851648+0000","last_clean":"2023-03-29T10:57:31.851648+0000","last_became_active":"2023-03-29T10:57:27.858803+0000","last_became_peered":"2023-03-29T10:57:27.858803+0000","last_unstale":"2023-03-29T10:57:31.851648+0000","last_undegraded":"2023-03-29T10:57:31.851648+0000","last_fullsized":"2023-03-29T10:57:31.851648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:44:42.208513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.341079+0000","last_change":"2023-03-29T10:57:31.868378+0000","last_active":"2023-03-29T10:57:32.341079+0000","last_peered":"2023-03-29T10:57:32.341079+0000","last_clean":"2023-03-29T10:57:32.341079+0000","last_became_active":"2023-03-29T10:57:31.867988+0000","last_became_peered":"2023-03-29T10:57:31.867988+0000","last_unstale":"2023-03-29T10:57:32.341079+0000","last_undegraded":"2023-03-29T10:57:32.341079+0000","last_fullsized":"2023-03-29T10:57:32.341079+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:18:14.433000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861576+0000","last_change":"2023-03-29T10:57:22.321523+0000","last_active":"2023-03-29T10:57:31.861576+0000","last_peered":"2023-03-29T10:57:31.861576+0000","last_clean":"2023-03-29T10:57:31.861576+0000","last_became_active":"2023-03-29T10:57:22.321183+0000","last_became_peered":"2023-03-29T10:57:22.321183+0000","last_unstale":"2023-03-29T10:57:31.861576+0000","last_undegraded":"2023-03-29T10:57:31.861576+0000","last_fullsized":"2023-03-29T10:57:31.861576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:35:11.664657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846653+0000","last_change":"2023-03-29T10:57:18.176665+0000","last_active":"2023-03-29T10:57:31.846653+0000","last_peered":"2023-03-29T10:57:31.846653+0000","last_clean":"2023-03-29T10:57:31.846653+0000","last_became_active":"2023-03-29T10:57:18.175739+0000","last_became_peered":"2023-03-29T10:57:18.175739+0000","last_unstale":"2023-03-29T10:57:31.846653+0000","last_undegraded":"2023-03-29T10:57:31.846653+0000","last_fullsized":"2023-03-29T10:57:31.846653+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:16.275989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:16.275989+0000","last_clean_scrub_stamp":"2023-03-29T10:57:16.275989+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:06:32.872344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851023+0000","last_change":"2023-03-29T10:57:27.365796+0000","last_active":"2023-03-29T10:57:31.851023+0000","last_peered":"2023-03-29T10:57:31.851023+0000","last_clean":"2023-03-29T10:57:31.851023+0000","last_became_active":"2023-03-29T10:57:27.365435+0000","last_became_peered":"2023-03-29T10:57:27.365435+0000","last_unstale":"2023-03-29T10:57:31.851023+0000","last_undegraded":"2023-03-29T10:57:31.851023+0000","last_fullsized":"2023-03-29T10:57:31.851023+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:12:14.095287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.855069+0000","last_change":"2023-03-29T10:57:27.866559+0000","last_active":"2023-03-29T10:57:31.855069+0000","last_peered":"2023-03-29T10:57:31.855069+0000","last_clean":"2023-03-29T10:57:31.855069+0000","last_became_active":"2023-03-29T10:57:27.866250+0000","last_became_peered":"2023-03-29T10:57:27.866250+0000","last_unstale":"2023-03-29T10:57:31.855069+0000","last_undegraded":"2023-03-29T10:57:31.855069+0000","last_fullsized":"2023-03-29T10:57:31.855069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:53:00.445130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606048+0000","last_change":"2023-03-29T10:57:31.881080+0000","last_active":"2023-03-29T10:57:32.606048+0000","last_peered":"2023-03-29T10:57:32.606048+0000","last_clean":"2023-03-29T10:57:32.606048+0000","last_became_active":"2023-03-29T10:57:31.880427+0000","last_became_peered":"2023-03-29T10:57:31.880427+0000","last_unstale":"2023-03-29T10:57:32.606048+0000","last_undegraded":"2023-03-29T10:57:32.606048+0000","last_fullsized":"2023-03-29T10:57:32.606048+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:36:05.766940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847505+0000","last_change":"2023-03-29T10:57:22.322607+0000","last_active":"2023-03-29T10:57:31.847505+0000","last_peered":"2023-03-29T10:57:31.847505+0000","last_clean":"2023-03-29T10:57:31.847505+0000","last_became_active":"2023-03-29T10:57:22.321613+0000","last_became_peered":"2023-03-29T10:57:22.321613+0000","last_unstale":"2023-03-29T10:57:31.847505+0000","last_undegraded":"2023-03-29T10:57:31.847505+0000","last_fullsized":"2023-03-29T10:57:31.847505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:35:11.664657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851455+0000","last_change":"2023-03-29T10:57:27.366362+0000","last_active":"2023-03-29T10:57:31.851455+0000","last_peered":"2023-03-29T10:57:31.851455+0000","last_clean":"2023-03-29T10:57:31.851455+0000","last_became_active":"2023-03-29T10:57:27.366048+0000","last_became_peered":"2023-03-29T10:57:27.366048+0000","last_unstale":"2023-03-29T10:57:31.851455+0000","last_undegraded":"2023-03-29T10:57:31.851455+0000","last_fullsized":"2023-03-29T10:57:31.851455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:01:05.024349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851632+0000","last_change":"2023-03-29T10:57:27.847948+0000","last_active":"2023-03-29T10:57:31.851632+0000","last_peered":"2023-03-29T10:57:31.851632+0000","last_clean":"2023-03-29T10:57:31.851632+0000","last_became_active":"2023-03-29T10:57:27.847691+0000","last_became_peered":"2023-03-29T10:57:27.847691+0000","last_unstale":"2023-03-29T10:57:31.851632+0000","last_undegraded":"2023-03-29T10:57:31.851632+0000","last_fullsized":"2023-03-29T10:57:31.851632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:01:58.615750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605974+0000","last_change":"2023-03-29T10:57:31.873406+0000","last_active":"2023-03-29T10:57:32.605974+0000","last_peered":"2023-03-29T10:57:32.605974+0000","last_clean":"2023-03-29T10:57:32.605974+0000","last_became_active":"2023-03-29T10:57:31.873032+0000","last_became_peered":"2023-03-29T10:57:31.873032+0000","last_unstale":"2023-03-29T10:57:32.605974+0000","last_undegraded":"2023-03-29T10:57:32.605974+0000","last_fullsized":"2023-03-29T10:57:32.605974+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:53:41.690520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846972+0000","last_change":"2023-03-29T10:57:22.315905+0000","last_active":"2023-03-29T10:57:31.846972+0000","last_peered":"2023-03-29T10:57:31.846972+0000","last_clean":"2023-03-29T10:57:31.846972+0000","last_became_active":"2023-03-29T10:57:22.315603+0000","last_became_peered":"2023-03-29T10:57:22.315603+0000","last_unstale":"2023-03-29T10:57:31.846972+0000","last_undegraded":"2023-03-29T10:57:31.846972+0000","last_fullsized":"2023-03-29T10:57:31.846972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T10:59:44.030013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853041+0000","last_change":"2023-03-29T10:57:27.363544+0000","last_active":"2023-03-29T10:57:31.853041+0000","last_peered":"2023-03-29T10:57:31.853041+0000","last_clean":"2023-03-29T10:57:31.853041+0000","last_became_active":"2023-03-29T10:57:27.363284+0000","last_became_peered":"2023-03-29T10:57:27.363284+0000","last_unstale":"2023-03-29T10:57:31.853041+0000","last_undegraded":"2023-03-29T10:57:31.853041+0000","last_fullsized":"2023-03-29T10:57:31.853041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:16.312585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853937+0000","last_change":"2023-03-29T10:57:27.844093+0000","last_active":"2023-03-29T10:57:31.853937+0000","last_peered":"2023-03-29T10:57:31.853937+0000","last_clean":"2023-03-29T10:57:31.853937+0000","last_became_active":"2023-03-29T10:57:27.843647+0000","last_became_peered":"2023-03-29T10:57:27.843647+0000","last_unstale":"2023-03-29T10:57:31.853937+0000","last_undegraded":"2023-03-29T10:57:31.853937+0000","last_fullsized":"2023-03-29T10:57:31.853937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:46:29.403965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340198+0000","last_change":"2023-03-29T10:57:31.878229+0000","last_active":"2023-03-29T10:57:32.340198+0000","last_peered":"2023-03-29T10:57:32.340198+0000","last_clean":"2023-03-29T10:57:32.340198+0000","last_became_active":"2023-03-29T10:57:31.877846+0000","last_became_peered":"2023-03-29T10:57:31.877846+0000","last_unstale":"2023-03-29T10:57:32.340198+0000","last_undegraded":"2023-03-29T10:57:32.340198+0000","last_fullsized":"2023-03-29T10:57:32.340198+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:32:15.233141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852732+0000","last_change":"2023-03-29T10:57:27.366984+0000","last_active":"2023-03-29T10:57:31.852732+0000","last_peered":"2023-03-29T10:57:31.852732+0000","last_clean":"2023-03-29T10:57:31.852732+0000","last_became_active":"2023-03-29T10:57:27.366716+0000","last_became_peered":"2023-03-29T10:57:27.366716+0000","last_unstale":"2023-03-29T10:57:31.852732+0000","last_undegraded":"2023-03-29T10:57:31.852732+0000","last_fullsized":"2023-03-29T10:57:31.852732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:00:38.166723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852301+0000","last_change":"2023-03-29T10:57:22.319554+0000","last_active":"2023-03-29T10:57:31.852301+0000","last_peered":"2023-03-29T10:57:31.852301+0000","last_clean":"2023-03-29T10:57:31.852301+0000","last_became_active":"2023-03-29T10:57:22.319198+0000","last_became_peered":"2023-03-29T10:57:22.319198+0000","last_unstale":"2023-03-29T10:57:31.852301+0000","last_undegraded":"2023-03-29T10:57:31.852301+0000","last_fullsized":"2023-03-29T10:57:31.852301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:47:01.099069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849473+0000","last_change":"2023-03-29T10:57:27.839233+0000","last_active":"2023-03-29T10:57:31.849473+0000","last_peered":"2023-03-29T10:57:31.849473+0000","last_clean":"2023-03-29T10:57:31.849473+0000","last_became_active":"2023-03-29T10:57:27.838936+0000","last_became_peered":"2023-03-29T10:57:27.838936+0000","last_unstale":"2023-03-29T10:57:31.849473+0000","last_undegraded":"2023-03-29T10:57:31.849473+0000","last_fullsized":"2023-03-29T10:57:31.849473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:10:11.279944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.607718+0000","last_change":"2023-03-29T10:57:31.876988+0000","last_active":"2023-03-29T10:57:32.607718+0000","last_peered":"2023-03-29T10:57:32.607718+0000","last_clean":"2023-03-29T10:57:32.607718+0000","last_became_active":"2023-03-29T10:57:31.876521+0000","last_became_peered":"2023-03-29T10:57:31.876521+0000","last_unstale":"2023-03-29T10:57:32.607718+0000","last_undegraded":"2023-03-29T10:57:32.607718+0000","last_fullsized":"2023-03-29T10:57:32.607718+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:40:04.971172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.858506+0000","last_change":"2023-03-29T10:57:22.322319+0000","last_active":"2023-03-29T10:57:31.858506+0000","last_peered":"2023-03-29T10:57:31.858506+0000","last_clean":"2023-03-29T10:57:31.858506+0000","last_became_active":"2023-03-29T10:57:22.321943+0000","last_became_peered":"2023-03-29T10:57:22.321943+0000","last_unstale":"2023-03-29T10:57:31.858506+0000","last_undegraded":"2023-03-29T10:57:31.858506+0000","last_fullsized":"2023-03-29T10:57:31.858506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:58:04.969453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"28'1","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.336470+0000","last_change":"2023-03-29T10:57:27.364507+0000","last_active":"2023-03-29T10:57:32.336470+0000","last_peered":"2023-03-29T10:57:32.336470+0000","last_clean":"2023-03-29T10:57:32.336470+0000","last_became_active":"2023-03-29T10:57:27.363959+0000","last_became_peered":"2023-03-29T10:57:27.363959+0000","last_unstale":"2023-03-29T10:57:32.336470+0000","last_undegraded":"2023-03-29T10:57:32.336470+0000","last_fullsized":"2023-03-29T10:57:32.336470+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:16.312585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860854+0000","last_change":"2023-03-29T10:57:27.861040+0000","last_active":"2023-03-29T10:57:31.860854+0000","last_peered":"2023-03-29T10:57:31.860854+0000","last_clean":"2023-03-29T10:57:31.860854+0000","last_became_active":"2023-03-29T10:57:27.860750+0000","last_became_peered":"2023-03-29T10:57:27.860750+0000","last_unstale":"2023-03-29T10:57:31.860854+0000","last_undegraded":"2023-03-29T10:57:31.860854+0000","last_fullsized":"2023-03-29T10:57:31.860854+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:40:51.829430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.543747+0000","last_change":"2023-03-29T10:57:31.882689+0000","last_active":"2023-03-29T10:57:32.543747+0000","last_peered":"2023-03-29T10:57:32.543747+0000","last_clean":"2023-03-29T10:57:32.543747+0000","last_became_active":"2023-03-29T10:57:31.882298+0000","last_became_peered":"2023-03-29T10:57:31.882298+0000","last_unstale":"2023-03-29T10:57:32.543747+0000","last_undegraded":"2023-03-29T10:57:32.543747+0000","last_fullsized":"2023-03-29T10:57:32.543747+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:18:14.433000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853851+0000","last_change":"2023-03-29T10:57:27.363293+0000","last_active":"2023-03-29T10:57:31.853851+0000","last_peered":"2023-03-29T10:57:31.853851+0000","last_clean":"2023-03-29T10:57:31.853851+0000","last_became_active":"2023-03-29T10:57:27.362993+0000","last_became_peered":"2023-03-29T10:57:27.362993+0000","last_unstale":"2023-03-29T10:57:31.853851+0000","last_undegraded":"2023-03-29T10:57:31.853851+0000","last_fullsized":"2023-03-29T10:57:31.853851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:57:18.786945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.845743+0000","last_change":"2023-03-29T10:57:27.849850+0000","last_active":"2023-03-29T10:57:31.845743+0000","last_peered":"2023-03-29T10:57:31.845743+0000","last_clean":"2023-03-29T10:57:31.845743+0000","last_became_active":"2023-03-29T10:57:27.849429+0000","last_became_peered":"2023-03-29T10:57:27.849429+0000","last_unstale":"2023-03-29T10:57:31.845743+0000","last_undegraded":"2023-03-29T10:57:31.845743+0000","last_fullsized":"2023-03-29T10:57:31.845743+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:00:39.175701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.341198+0000","last_change":"2023-03-29T10:57:31.868416+0000","last_active":"2023-03-29T10:57:32.341198+0000","last_peered":"2023-03-29T10:57:32.341198+0000","last_clean":"2023-03-29T10:57:32.341198+0000","last_became_active":"2023-03-29T10:57:31.868042+0000","last_became_peered":"2023-03-29T10:57:31.868042+0000","last_unstale":"2023-03-29T10:57:32.341198+0000","last_undegraded":"2023-03-29T10:57:32.341198+0000","last_fullsized":"2023-03-29T10:57:32.341198+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:28:41.336878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861823+0000","last_change":"2023-03-29T10:57:27.373078+0000","last_active":"2023-03-29T10:57:31.861823+0000","last_peered":"2023-03-29T10:57:31.861823+0000","last_clean":"2023-03-29T10:57:31.861823+0000","last_became_active":"2023-03-29T10:57:27.372747+0000","last_became_peered":"2023-03-29T10:57:27.372747+0000","last_unstale":"2023-03-29T10:57:31.861823+0000","last_undegraded":"2023-03-29T10:57:31.861823+0000","last_fullsized":"2023-03-29T10:57:31.861823+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:23:59.025438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853187+0000","last_change":"2023-03-29T10:57:27.861043+0000","last_active":"2023-03-29T10:57:31.853187+0000","last_peered":"2023-03-29T10:57:31.853187+0000","last_clean":"2023-03-29T10:57:31.853187+0000","last_became_active":"2023-03-29T10:57:27.860522+0000","last_became_peered":"2023-03-29T10:57:27.860522+0000","last_unstale":"2023-03-29T10:57:31.853187+0000","last_undegraded":"2023-03-29T10:57:31.853187+0000","last_fullsized":"2023-03-29T10:57:31.853187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:49:24.886407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606089+0000","last_change":"2023-03-29T10:57:31.879767+0000","last_active":"2023-03-29T10:57:32.606089+0000","last_peered":"2023-03-29T10:57:32.606089+0000","last_clean":"2023-03-29T10:57:32.606089+0000","last_became_active":"2023-03-29T10:57:31.879330+0000","last_became_peered":"2023-03-29T10:57:31.879330+0000","last_unstale":"2023-03-29T10:57:32.606089+0000","last_undegraded":"2023-03-29T10:57:32.606089+0000","last_fullsized":"2023-03-29T10:57:32.606089+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:10:08.859801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.864594+0000","last_change":"2023-03-29T10:57:27.370488+0000","last_active":"2023-03-29T10:57:31.864594+0000","last_peered":"2023-03-29T10:57:31.864594+0000","last_clean":"2023-03-29T10:57:31.864594+0000","last_became_active":"2023-03-29T10:57:27.370181+0000","last_became_peered":"2023-03-29T10:57:27.370181+0000","last_unstale":"2023-03-29T10:57:31.864594+0000","last_undegraded":"2023-03-29T10:57:31.864594+0000","last_fullsized":"2023-03-29T10:57:31.864594+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:10:38.094232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852668+0000","last_change":"2023-03-29T10:57:27.860955+0000","last_active":"2023-03-29T10:57:31.852668+0000","last_peered":"2023-03-29T10:57:31.852668+0000","last_clean":"2023-03-29T10:57:31.852668+0000","last_became_active":"2023-03-29T10:57:27.860396+0000","last_became_peered":"2023-03-29T10:57:27.860396+0000","last_unstale":"2023-03-29T10:57:31.852668+0000","last_undegraded":"2023-03-29T10:57:31.852668+0000","last_fullsized":"2023-03-29T10:57:31.852668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:44:40.017537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.339309+0000","last_change":"2023-03-29T10:57:31.874679+0000","last_active":"2023-03-29T10:57:32.339309+0000","last_peered":"2023-03-29T10:57:32.339309+0000","last_clean":"2023-03-29T10:57:32.339309+0000","last_became_active":"2023-03-29T10:57:31.874124+0000","last_became_peered":"2023-03-29T10:57:31.874124+0000","last_unstale":"2023-03-29T10:57:32.339309+0000","last_undegraded":"2023-03-29T10:57:32.339309+0000","last_fullsized":"2023-03-29T10:57:32.339309+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:36:05.766940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853964+0000","last_change":"2023-03-29T10:57:27.366500+0000","last_active":"2023-03-29T10:57:31.853964+0000","last_peered":"2023-03-29T10:57:31.853964+0000","last_clean":"2023-03-29T10:57:31.853964+0000","last_became_active":"2023-03-29T10:57:27.366215+0000","last_became_peered":"2023-03-29T10:57:27.366215+0000","last_unstale":"2023-03-29T10:57:31.853964+0000","last_undegraded":"2023-03-29T10:57:31.853964+0000","last_fullsized":"2023-03-29T10:57:31.853964+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:45:12.326563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852786+0000","last_change":"2023-03-29T10:57:27.838577+0000","last_active":"2023-03-29T10:57:31.852786+0000","last_peered":"2023-03-29T10:57:31.852786+0000","last_clean":"2023-03-29T10:57:31.852786+0000","last_became_active":"2023-03-29T10:57:27.838301+0000","last_became_peered":"2023-03-29T10:57:27.838301+0000","last_unstale":"2023-03-29T10:57:31.852786+0000","last_undegraded":"2023-03-29T10:57:31.852786+0000","last_fullsized":"2023-03-29T10:57:31.852786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:36:09.200571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.543500+0000","last_change":"2023-03-29T10:57:31.874474+0000","last_active":"2023-03-29T10:57:32.543500+0000","last_peered":"2023-03-29T10:57:32.543500+0000","last_clean":"2023-03-29T10:57:32.543500+0000","last_became_active":"2023-03-29T10:57:31.874137+0000","last_became_peered":"2023-03-29T10:57:31.874137+0000","last_unstale":"2023-03-29T10:57:32.543500+0000","last_undegraded":"2023-03-29T10:57:32.543500+0000","last_fullsized":"2023-03-29T10:57:32.543500+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:10.382014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.857710+0000","last_change":"2023-03-29T10:57:27.365343+0000","last_active":"2023-03-29T10:57:31.857710+0000","last_peered":"2023-03-29T10:57:31.857710+0000","last_clean":"2023-03-29T10:57:31.857710+0000","last_became_active":"2023-03-29T10:57:27.365066+0000","last_became_peered":"2023-03-29T10:57:27.365066+0000","last_unstale":"2023-03-29T10:57:31.857710+0000","last_undegraded":"2023-03-29T10:57:31.857710+0000","last_fullsized":"2023-03-29T10:57:31.857710+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:40:18.602235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860006+0000","last_change":"2023-03-29T10:57:27.860635+0000","last_active":"2023-03-29T10:57:31.860006+0000","last_peered":"2023-03-29T10:57:31.860006+0000","last_clean":"2023-03-29T10:57:31.860006+0000","last_became_active":"2023-03-29T10:57:27.860287+0000","last_became_peered":"2023-03-29T10:57:27.860287+0000","last_unstale":"2023-03-29T10:57:31.860006+0000","last_undegraded":"2023-03-29T10:57:31.860006+0000","last_fullsized":"2023-03-29T10:57:31.860006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:57:16.888544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852270+0000","last_change":"2023-03-29T10:57:27.858882+0000","last_active":"2023-03-29T10:57:31.852270+0000","last_peered":"2023-03-29T10:57:31.852270+0000","last_clean":"2023-03-29T10:57:31.852270+0000","last_became_active":"2023-03-29T10:57:27.858471+0000","last_became_peered":"2023-03-29T10:57:27.858471+0000","last_unstale":"2023-03-29T10:57:31.852270+0000","last_undegraded":"2023-03-29T10:57:31.852270+0000","last_fullsized":"2023-03-29T10:57:31.852270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:10:39.103210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852924+0000","last_change":"2023-03-29T10:57:27.848104+0000","last_active":"2023-03-29T10:57:31.852924+0000","last_peered":"2023-03-29T10:57:31.852924+0000","last_clean":"2023-03-29T10:57:31.852924+0000","last_became_active":"2023-03-29T10:57:27.847812+0000","last_became_peered":"2023-03-29T10:57:27.847812+0000","last_unstale":"2023-03-29T10:57:31.852924+0000","last_undegraded":"2023-03-29T10:57:31.852924+0000","last_fullsized":"2023-03-29T10:57:31.852924+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:50:00.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852955+0000","last_change":"2023-03-29T10:57:27.868878+0000","last_active":"2023-03-29T10:57:31.852955+0000","last_peered":"2023-03-29T10:57:31.852955+0000","last_clean":"2023-03-29T10:57:31.852955+0000","last_became_active":"2023-03-29T10:57:27.868603+0000","last_became_peered":"2023-03-29T10:57:27.868603+0000","last_unstale":"2023-03-29T10:57:31.852955+0000","last_undegraded":"2023-03-29T10:57:31.852955+0000","last_fullsized":"2023-03-29T10:57:31.852955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:53:38.186916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851718+0000","last_change":"2023-03-29T10:57:27.853683+0000","last_active":"2023-03-29T10:57:31.851718+0000","last_peered":"2023-03-29T10:57:31.851718+0000","last_clean":"2023-03-29T10:57:31.851718+0000","last_became_active":"2023-03-29T10:57:27.853396+0000","last_became_peered":"2023-03-29T10:57:27.853396+0000","last_unstale":"2023-03-29T10:57:31.851718+0000","last_undegraded":"2023-03-29T10:57:31.851718+0000","last_fullsized":"2023-03-29T10:57:31.851718+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:57:16.888544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852962+0000","last_change":"2023-03-29T10:57:27.856369+0000","last_active":"2023-03-29T10:57:31.852962+0000","last_peered":"2023-03-29T10:57:31.852962+0000","last_clean":"2023-03-29T10:57:31.852962+0000","last_became_active":"2023-03-29T10:57:27.856118+0000","last_became_peered":"2023-03-29T10:57:27.856118+0000","last_unstale":"2023-03-29T10:57:31.852962+0000","last_undegraded":"2023-03-29T10:57:31.852962+0000","last_fullsized":"2023-03-29T10:57:31.852962+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:33:37.107795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4598,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9296,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362544,"statfs":{"total":96636764160,"available":96627245056,"internally_reserved":0,"allocated":925696,"data_stored":718956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8112,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363728,"statfs":{"total":96636764160,"available":96628457472,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8176,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363664,"statfs":{"total":96636764160,"available":96628391936,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9268,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362572,"statfs":{"total":96636764160,"available":96627273728,"internally_reserved":0,"allocated":897024,"data_stored":714903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8108,"kb_used_data":292,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363732,"statfs":{"total":96636764160,"available":96628461568,"internally_reserved":0,"allocated":299008,"data_stored":123953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8180,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363660,"statfs":{"total":96636764160,"available":96628387840,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8260,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363580,"statfs":{"total":96636764160,"available":96628305920,"internally_reserved":0,"allocated":323584,"data_stored":128428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8184,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363656,"statfs":{"total":96636764160,"available":96628383744,"internally_reserved":0,"allocated":311296,"data_stored":125224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-03-29T10:57:54.717 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T10:57:54.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.916+0000 7f70443ff700 1 Processor -- start 2023-03-29T10:57:54.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.916+0000 7f70443ff700 1 -- start start 2023-03-29T10:57:54.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.917+0000 7f70443ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f703c0cf970 0x7f703c0cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:54.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.917+0000 7f70443ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f703c0cfd40 0x7f703c0d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:54.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.917+0000 7f70443ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f703c0cdc10 con 0x7f703c160290 2023-03-29T10:57:54.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.917+0000 7f70443ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f703c0b6c60 con 0x7f703c0cfd40 2023-03-29T10:57:54.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.917+0000 7f70443ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703c0cd650 con 0x7f703c0cf970 2023-03-29T10:57:54.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.917+0000 7f704299c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f703c160290 0x7f703c1606d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41638/0 (socket says 172.21.15.111:41638) 2023-03-29T10:57:54.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.917+0000 7f704199a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f703c0cfd40 0x7f703c0d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:54.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.917+0000 7f704299c700 1 -- 172.21.15.111:0/3688664146 learned_addr learned my addr 172.21.15.111:0/3688664146 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:54.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.918+0000 7f704219b700 1 --2- 172.21.15.111:0/3688664146 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f703c0cf970 0x7f703c0cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:54.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.918+0000 7f704199a700 1 -- 172.21.15.111:0/3688664146 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f703c0cf970 msgr2=0x7f703c0cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:54.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.918+0000 7f704199a700 1 --2- 172.21.15.111:0/3688664146 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f703c0cf970 0x7f703c0cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.918+0000 7f704199a700 1 -- 172.21.15.111:0/3688664146 >> v1:172.21.15.111:6789/0 conn(0x7f703c160290 legacy=0x7f703c1606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:54.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.918+0000 7f704199a700 1 -- 172.21.15.111:0/3688664146 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703c0cd930 con 0x7f703c0cfd40 2023-03-29T10:57:54.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.918+0000 7f704199a700 1 --2- 172.21.15.111:0/3688664146 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f703c0cfd40 0x7f703c0d01b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f702c009840 tx=0x7f702c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc92a6082d976d3 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:54.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.918+0000 7f7041199700 1 -- 172.21.15.111:0/3688664146 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c011830 con 0x7f703c0cfd40 2023-03-29T10:57:54.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.918+0000 7f7041199700 1 -- 172.21.15.111:0/3688664146 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702c011990 con 0x7f703c0cfd40 2023-03-29T10:57:54.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.918+0000 7f7041199700 1 -- 172.21.15.111:0/3688664146 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c011d20 con 0x7f703c0cfd40 2023-03-29T10:57:54.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.919+0000 7f70443ff700 1 -- 172.21.15.111:0/3688664146 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f703c0cfd40 msgr2=0x7f703c0d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:54.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.919+0000 7f70443ff700 1 --2- 172.21.15.111:0/3688664146 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f703c0cfd40 0x7f703c0d01b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f702c009840 tx=0x7f702c009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:54.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.919+0000 7f70443ff700 1 -- 172.21.15.111:0/3688664146 shutdown_connections 2023-03-29T10:57:54.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.919+0000 7f70443ff700 1 --2- 172.21.15.111:0/3688664146 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f703c0cf970 0x7f703c0cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.919+0000 7f70443ff700 1 --2- 172.21.15.111:0/3688664146 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f703c0cfd40 0x7f703c0d01b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.919+0000 7f70443ff700 1 -- 172.21.15.111:0/3688664146 >> 172.21.15.111:0/3688664146 conn(0x7f703c071b10 msgr2=0x7f703c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:54.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.919+0000 7f70443ff700 1 -- 172.21.15.111:0/3688664146 shutdown_connections 2023-03-29T10:57:54.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.919+0000 7f70443ff700 1 -- 172.21.15.111:0/3688664146 wait complete. 2023-03-29T10:57:54.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.921+0000 7f70443ff700 1 Processor -- start 2023-03-29T10:57:54.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.921+0000 7f70443ff700 1 -- start start 2023-03-29T10:57:54.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.922+0000 7f70443ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f703c0cf970 0x7f703c0c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:54.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.922+0000 7f70443ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f703c0cfd40 0x7f703c14a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:54.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.922+0000 7f70443ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f703c160290 0x7f703c0c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:54.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.922+0000 7f70443ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c0b2570 con 0x7f703c160290 2023-03-29T10:57:54.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.922+0000 7f70443ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703c14a580 con 0x7f703c0cfd40 2023-03-29T10:57:54.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.922+0000 7f70443ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703c1f30b0 con 0x7f703c0cf970 2023-03-29T10:57:54.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.922+0000 7f704299c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f703c160290 0x7f703c0c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:54.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.922+0000 7f704299c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f703c160290 0x7f703c0c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38236/0 (socket says 172.21.15.111:38236) 2023-03-29T10:57:54.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.922+0000 7f704299c700 1 -- 172.21.15.111:0/420774813 learned_addr learned my addr 172.21.15.111:0/420774813 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:54.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.922+0000 7f704219b700 1 --2- 172.21.15.111:0/420774813 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f703c0cf970 0x7f703c0c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:54.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.922+0000 7f704199a700 1 --2- 172.21.15.111:0/420774813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f703c0cfd40 0x7f703c14a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:54.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.922+0000 7f704299c700 1 -- 172.21.15.111:0/420774813 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f703c0cf970 msgr2=0x7f703c0c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:54.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.922+0000 7f704299c700 1 --2- 172.21.15.111:0/420774813 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f703c0cf970 0x7f703c0c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.922+0000 7f704299c700 1 -- 172.21.15.111:0/420774813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f703c0cfd40 msgr2=0x7f703c14a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:54.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.922+0000 7f704299c700 1 --2- 172.21.15.111:0/420774813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f703c0cfd40 0x7f703c14a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:54.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.922+0000 7f704299c700 1 -- 172.21.15.111:0/420774813 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702c0094a0 con 0x7f703c160290 2023-03-29T10:57:54.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.922+0000 7f704199a700 1 --2- 172.21.15.111:0/420774813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f703c0cfd40 0x7f703c14a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:57:54.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.923+0000 7f704299c700 1 --2- 172.21.15.111:0/420774813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f703c160290 0x7f703c0c6a70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f703400cab0 tx=0x7f703400cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:54.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.923+0000 7f7032ffd700 1 -- 172.21.15.111:0/420774813 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703400a9a0 con 0x7f703c160290 2023-03-29T10:57:54.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.923+0000 7f7032ffd700 1 -- 172.21.15.111:0/420774813 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703400ab00 con 0x7f703c160290 2023-03-29T10:57:54.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.923+0000 7f70443ff700 1 -- 172.21.15.111:0/420774813 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f703c0cf650 con 0x7f703c160290 2023-03-29T10:57:54.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.923+0000 7f7032ffd700 1 -- 172.21.15.111:0/420774813 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703400ae70 con 0x7f703c160290 2023-03-29T10:57:54.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.923+0000 7f70443ff700 1 -- 172.21.15.111:0/420774813 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f703c0b68f0 con 0x7f703c160290 2023-03-29T10:57:54.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.924+0000 7f70443ff700 1 -- 172.21.15.111:0/420774813 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f703c0d06f0 con 0x7f703c160290 2023-03-29T10:57:54.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.925+0000 7f7032ffd700 1 -- 172.21.15.111:0/420774813 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f703401b3b0 con 0x7f703c160290 2023-03-29T10:57:54.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.926+0000 7f7032ffd700 1 --2- 172.21.15.111:0/420774813 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7020069de0 0x7f702006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:54.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.926+0000 7f7032ffd700 1 -- 172.21.15.111:0/420774813 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7034087dc0 con 0x7f703c160290 2023-03-29T10:57:54.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.926+0000 7f704219b700 1 --2- 172.21.15.111:0/420774813 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7020069de0 0x7f702006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:54.926 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.926+0000 7f704219b700 1 --2- 172.21.15.111:0/420774813 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7020069de0 0x7f702006c290 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f703c0ca970 tx=0x7f7038009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:54.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:54.930+0000 7f7032ffd700 1 -- 172.21.15.111:0/420774813 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f703404c160 con 0x7f703c160290 2023-03-29T10:57:55.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.118+0000 7f70443ff700 1 -- 172.21.15.111:0/420774813 --> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f703c074430 con 0x7f7020069de0 2023-03-29T10:57:55.128 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.128+0000 7f7032ffd700 1 -- 172.21.15.111:0/420774813 <== mgr.4103 v2:172.21.15.111:6832/110141 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367280 (secure 0 0 0) 0x7f703c074430 con 0x7f7020069de0 2023-03-29T10:57:55.132 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:55.134 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.133+0000 7f70443ff700 1 -- 172.21.15.111:0/420774813 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7020069de0 msgr2=0x7f702006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:55.134 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.133+0000 7f70443ff700 1 --2- 172.21.15.111:0/420774813 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7020069de0 0x7f702006c290 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f703c0ca970 tx=0x7f7038009210 comp rx=0 tx=0).stop 2023-03-29T10:57:55.134 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.133+0000 7f70443ff700 1 -- 172.21.15.111:0/420774813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f703c160290 msgr2=0x7f703c0c6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:55.134 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.133+0000 7f70443ff700 1 --2- 172.21.15.111:0/420774813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f703c160290 0x7f703c0c6a70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f703400cab0 tx=0x7f703400cdc0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.133+0000 7f70443ff700 1 -- 172.21.15.111:0/420774813 shutdown_connections 2023-03-29T10:57:55.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.133+0000 7f70443ff700 1 --2- 172.21.15.111:0/420774813 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7020069de0 0x7f702006c290 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.133+0000 7f70443ff700 1 --2- 172.21.15.111:0/420774813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f703c160290 0x7f703c0c6a70 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.133+0000 7f70443ff700 1 --2- 172.21.15.111:0/420774813 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f703c0cf970 0x7f703c0c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.133+0000 7f70443ff700 1 --2- 172.21.15.111:0/420774813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f703c0cfd40 0x7f703c14a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.133+0000 7f70443ff700 1 -- 172.21.15.111:0/420774813 >> 172.21.15.111:0/420774813 conn(0x7f703c071b10 msgr2=0x7f703c0d1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:55.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.133+0000 7f70443ff700 1 -- 172.21.15.111:0/420774813 shutdown_connections 2023-03-29T10:57:55.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.133+0000 7f70443ff700 1 -- 172.21.15.111:0/420774813 wait complete. 2023-03-29T10:57:55.135 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2023-03-29T10:57:55.149 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2023-03-29T10:57:54.131594+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594966,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67584,"kb_used_data":3584,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":754907136,"statfs":{"total":773094113280,"available":773024907264,"internally_reserved":0,"allocated":3670016,"data_stored":2185270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":142,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003130"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.845704+0000","last_change":"2023-03-29T10:57:27.854795+0000","last_active":"2023-03-29T10:57:31.845704+0000","last_peered":"2023-03-29T10:57:31.845704+0000","last_clean":"2023-03-29T10:57:31.845704+0000","last_became_active":"2023-03-29T10:57:27.854521+0000","last_became_peered":"2023-03-29T10:57:27.854521+0000","last_unstale":"2023-03-29T10:57:31.845704+0000","last_undegraded":"2023-03-29T10:57:31.845704+0000","last_fullsized":"2023-03-29T10:57:31.845704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:40:19.611213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.862071+0000","last_change":"2023-03-29T10:57:27.850091+0000","last_active":"2023-03-29T10:57:31.862071+0000","last_peered":"2023-03-29T10:57:31.862071+0000","last_clean":"2023-03-29T10:57:31.862071+0000","last_became_active":"2023-03-29T10:57:27.849795+0000","last_became_peered":"2023-03-29T10:57:27.849795+0000","last_unstale":"2023-03-29T10:57:31.862071+0000","last_undegraded":"2023-03-29T10:57:31.862071+0000","last_fullsized":"2023-03-29T10:57:31.862071+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:36:09.200571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853118+0000","last_change":"2023-03-29T10:57:27.864238+0000","last_active":"2023-03-29T10:57:31.853118+0000","last_peered":"2023-03-29T10:57:31.853118+0000","last_clean":"2023-03-29T10:57:31.853118+0000","last_became_active":"2023-03-29T10:57:27.863878+0000","last_became_peered":"2023-03-29T10:57:27.863878+0000","last_unstale":"2023-03-29T10:57:31.853118+0000","last_undegraded":"2023-03-29T10:57:31.853118+0000","last_fullsized":"2023-03-29T10:57:31.853118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:01:58.615750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854397+0000","last_change":"2023-03-29T10:57:27.861054+0000","last_active":"2023-03-29T10:57:31.854397+0000","last_peered":"2023-03-29T10:57:31.854397+0000","last_clean":"2023-03-29T10:57:31.854397+0000","last_became_active":"2023-03-29T10:57:27.859547+0000","last_became_peered":"2023-03-29T10:57:27.859547+0000","last_unstale":"2023-03-29T10:57:31.854397+0000","last_undegraded":"2023-03-29T10:57:31.854397+0000","last_fullsized":"2023-03-29T10:57:31.854397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:24:00.034416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854082+0000","last_change":"2023-03-29T10:57:27.855728+0000","last_active":"2023-03-29T10:57:31.854082+0000","last_peered":"2023-03-29T10:57:31.854082+0000","last_clean":"2023-03-29T10:57:31.854082+0000","last_became_active":"2023-03-29T10:57:27.855438+0000","last_became_peered":"2023-03-29T10:57:27.855438+0000","last_unstale":"2023-03-29T10:57:31.854082+0000","last_undegraded":"2023-03-29T10:57:31.854082+0000","last_fullsized":"2023-03-29T10:57:31.854082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:33:37.107795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852905+0000","last_change":"2023-03-29T10:57:27.861795+0000","last_active":"2023-03-29T10:57:31.852905+0000","last_peered":"2023-03-29T10:57:31.852905+0000","last_clean":"2023-03-29T10:57:31.852905+0000","last_became_active":"2023-03-29T10:57:27.860751+0000","last_became_peered":"2023-03-29T10:57:27.860751+0000","last_unstale":"2023-03-29T10:57:31.852905+0000","last_undegraded":"2023-03-29T10:57:31.852905+0000","last_fullsized":"2023-03-29T10:57:31.852905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:41.882164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847170+0000","last_change":"2023-03-29T10:57:27.855143+0000","last_active":"2023-03-29T10:57:31.847170+0000","last_peered":"2023-03-29T10:57:31.847170+0000","last_clean":"2023-03-29T10:57:31.847170+0000","last_became_active":"2023-03-29T10:57:27.854804+0000","last_became_peered":"2023-03-29T10:57:27.854804+0000","last_unstale":"2023-03-29T10:57:31.847170+0000","last_undegraded":"2023-03-29T10:57:31.847170+0000","last_fullsized":"2023-03-29T10:57:31.847170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:24:08.253556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861416+0000","last_change":"2023-03-29T10:57:27.864339+0000","last_active":"2023-03-29T10:57:31.861416+0000","last_peered":"2023-03-29T10:57:31.861416+0000","last_clean":"2023-03-29T10:57:31.861416+0000","last_became_active":"2023-03-29T10:57:27.864034+0000","last_became_peered":"2023-03-29T10:57:27.864034+0000","last_unstale":"2023-03-29T10:57:31.861416+0000","last_undegraded":"2023-03-29T10:57:31.861416+0000","last_fullsized":"2023-03-29T10:57:31.861416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:10:11.279944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851971+0000","last_change":"2023-03-29T10:57:27.849118+0000","last_active":"2023-03-29T10:57:31.851971+0000","last_peered":"2023-03-29T10:57:31.851971+0000","last_clean":"2023-03-29T10:57:31.851971+0000","last_became_active":"2023-03-29T10:57:27.848724+0000","last_became_peered":"2023-03-29T10:57:27.848724+0000","last_unstale":"2023-03-29T10:57:31.851971+0000","last_undegraded":"2023-03-29T10:57:31.851971+0000","last_fullsized":"2023-03-29T10:57:31.851971+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:36:09.200571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.862264+0000","last_change":"2023-03-29T10:57:27.848373+0000","last_active":"2023-03-29T10:57:31.862264+0000","last_peered":"2023-03-29T10:57:31.862264+0000","last_clean":"2023-03-29T10:57:31.862264+0000","last_became_active":"2023-03-29T10:57:27.847792+0000","last_became_peered":"2023-03-29T10:57:27.847792+0000","last_unstale":"2023-03-29T10:57:31.862264+0000","last_undegraded":"2023-03-29T10:57:31.862264+0000","last_fullsized":"2023-03-29T10:57:31.862264+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:46:29.403965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853207+0000","last_change":"2023-03-29T10:57:27.864182+0000","last_active":"2023-03-29T10:57:31.853207+0000","last_peered":"2023-03-29T10:57:31.853207+0000","last_clean":"2023-03-29T10:57:31.853207+0000","last_became_active":"2023-03-29T10:57:27.863567+0000","last_became_peered":"2023-03-29T10:57:27.863567+0000","last_unstale":"2023-03-29T10:57:31.853207+0000","last_undegraded":"2023-03-29T10:57:31.853207+0000","last_fullsized":"2023-03-29T10:57:31.853207+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:52:36.730410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.845640+0000","last_change":"2023-03-29T10:57:27.863745+0000","last_active":"2023-03-29T10:57:31.845640+0000","last_peered":"2023-03-29T10:57:31.845640+0000","last_clean":"2023-03-29T10:57:31.845640+0000","last_became_active":"2023-03-29T10:57:27.863377+0000","last_became_peered":"2023-03-29T10:57:27.863377+0000","last_unstale":"2023-03-29T10:57:31.845640+0000","last_undegraded":"2023-03-29T10:57:31.845640+0000","last_fullsized":"2023-03-29T10:57:31.845640+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:53:39.143611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846141+0000","last_change":"2023-03-29T10:57:27.856601+0000","last_active":"2023-03-29T10:57:31.846141+0000","last_peered":"2023-03-29T10:57:31.846141+0000","last_clean":"2023-03-29T10:57:31.846141+0000","last_became_active":"2023-03-29T10:57:27.856323+0000","last_became_peered":"2023-03-29T10:57:27.856323+0000","last_unstale":"2023-03-29T10:57:31.846141+0000","last_undegraded":"2023-03-29T10:57:31.846141+0000","last_fullsized":"2023-03-29T10:57:31.846141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:12:15.104266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.863135+0000","last_change":"2023-03-29T10:57:27.850723+0000","last_active":"2023-03-29T10:57:31.863135+0000","last_peered":"2023-03-29T10:57:31.863135+0000","last_clean":"2023-03-29T10:57:31.863135+0000","last_became_active":"2023-03-29T10:57:27.844480+0000","last_became_peered":"2023-03-29T10:57:27.844480+0000","last_unstale":"2023-03-29T10:57:31.863135+0000","last_undegraded":"2023-03-29T10:57:31.863135+0000","last_fullsized":"2023-03-29T10:57:31.863135+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:07:19.154573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847070+0000","last_change":"2023-03-29T10:57:27.855233+0000","last_active":"2023-03-29T10:57:31.847070+0000","last_peered":"2023-03-29T10:57:31.847070+0000","last_clean":"2023-03-29T10:57:31.847070+0000","last_became_active":"2023-03-29T10:57:27.854970+0000","last_became_peered":"2023-03-29T10:57:27.854970+0000","last_unstale":"2023-03-29T10:57:31.847070+0000","last_undegraded":"2023-03-29T10:57:31.847070+0000","last_fullsized":"2023-03-29T10:57:31.847070+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:46:29.403965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860076+0000","last_change":"2023-03-29T10:57:27.860037+0000","last_active":"2023-03-29T10:57:31.860076+0000","last_peered":"2023-03-29T10:57:31.860076+0000","last_clean":"2023-03-29T10:57:31.860076+0000","last_became_active":"2023-03-29T10:57:27.859764+0000","last_became_peered":"2023-03-29T10:57:27.859764+0000","last_unstale":"2023-03-29T10:57:31.860076+0000","last_undegraded":"2023-03-29T10:57:31.860076+0000","last_fullsized":"2023-03-29T10:57:31.860076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:25:48.508072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851770+0000","last_change":"2023-03-29T10:57:27.848952+0000","last_active":"2023-03-29T10:57:31.851770+0000","last_peered":"2023-03-29T10:57:31.851770+0000","last_clean":"2023-03-29T10:57:31.851770+0000","last_became_active":"2023-03-29T10:57:27.848390+0000","last_became_peered":"2023-03-29T10:57:27.848390+0000","last_unstale":"2023-03-29T10:57:31.851770+0000","last_undegraded":"2023-03-29T10:57:31.851770+0000","last_fullsized":"2023-03-29T10:57:31.851770+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:07:19.154573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854647+0000","last_change":"2023-03-29T10:57:27.862653+0000","last_active":"2023-03-29T10:57:31.854647+0000","last_peered":"2023-03-29T10:57:31.854647+0000","last_clean":"2023-03-29T10:57:31.854647+0000","last_became_active":"2023-03-29T10:57:27.862108+0000","last_became_peered":"2023-03-29T10:57:27.862108+0000","last_unstale":"2023-03-29T10:57:31.854647+0000","last_undegraded":"2023-03-29T10:57:31.854647+0000","last_fullsized":"2023-03-29T10:57:31.854647+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:06.878410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846508+0000","last_change":"2023-03-29T10:57:27.859887+0000","last_active":"2023-03-29T10:57:31.846508+0000","last_peered":"2023-03-29T10:57:31.846508+0000","last_clean":"2023-03-29T10:57:31.846508+0000","last_became_active":"2023-03-29T10:57:27.859632+0000","last_became_peered":"2023-03-29T10:57:27.859632+0000","last_unstale":"2023-03-29T10:57:31.846508+0000","last_undegraded":"2023-03-29T10:57:31.846508+0000","last_fullsized":"2023-03-29T10:57:31.846508+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:17.321564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854180+0000","last_change":"2023-03-29T10:57:27.865228+0000","last_active":"2023-03-29T10:57:31.854180+0000","last_peered":"2023-03-29T10:57:31.854180+0000","last_clean":"2023-03-29T10:57:31.854180+0000","last_became_active":"2023-03-29T10:57:27.864919+0000","last_became_peered":"2023-03-29T10:57:27.864919+0000","last_unstale":"2023-03-29T10:57:31.854180+0000","last_undegraded":"2023-03-29T10:57:31.854180+0000","last_fullsized":"2023-03-29T10:57:31.854180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:44:47.553332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846978+0000","last_change":"2023-03-29T10:57:27.853377+0000","last_active":"2023-03-29T10:57:31.846978+0000","last_peered":"2023-03-29T10:57:31.846978+0000","last_clean":"2023-03-29T10:57:31.846978+0000","last_became_active":"2023-03-29T10:57:27.853039+0000","last_became_peered":"2023-03-29T10:57:27.853039+0000","last_unstale":"2023-03-29T10:57:31.846978+0000","last_undegraded":"2023-03-29T10:57:31.846978+0000","last_fullsized":"2023-03-29T10:57:31.846978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:23:24.586546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851362+0000","last_change":"2023-03-29T10:57:27.849931+0000","last_active":"2023-03-29T10:57:31.851362+0000","last_peered":"2023-03-29T10:57:31.851362+0000","last_clean":"2023-03-29T10:57:31.851362+0000","last_became_active":"2023-03-29T10:57:27.849662+0000","last_became_peered":"2023-03-29T10:57:27.849662+0000","last_unstale":"2023-03-29T10:57:31.851362+0000","last_undegraded":"2023-03-29T10:57:31.851362+0000","last_fullsized":"2023-03-29T10:57:31.851362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:25:48.508072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852622+0000","last_change":"2023-03-29T10:57:27.850427+0000","last_active":"2023-03-29T10:57:31.852622+0000","last_peered":"2023-03-29T10:57:31.852622+0000","last_clean":"2023-03-29T10:57:31.852622+0000","last_became_active":"2023-03-29T10:57:27.850140+0000","last_became_peered":"2023-03-29T10:57:27.850140+0000","last_unstale":"2023-03-29T10:57:31.852622+0000","last_undegraded":"2023-03-29T10:57:31.852622+0000","last_fullsized":"2023-03-29T10:57:31.852622+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:01:58.615750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847710+0000","last_change":"2023-03-29T10:57:27.854437+0000","last_active":"2023-03-29T10:57:31.847710+0000","last_peered":"2023-03-29T10:57:31.847710+0000","last_clean":"2023-03-29T10:57:31.847710+0000","last_became_active":"2023-03-29T10:57:27.853876+0000","last_became_peered":"2023-03-29T10:57:27.853876+0000","last_unstale":"2023-03-29T10:57:31.847710+0000","last_undegraded":"2023-03-29T10:57:31.847710+0000","last_fullsized":"2023-03-29T10:57:31.847710+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:33:37.107795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851569+0000","last_change":"2023-03-29T10:57:27.856343+0000","last_active":"2023-03-29T10:57:31.851569+0000","last_peered":"2023-03-29T10:57:31.851569+0000","last_clean":"2023-03-29T10:57:31.851569+0000","last_became_active":"2023-03-29T10:57:27.855843+0000","last_became_peered":"2023-03-29T10:57:27.855843+0000","last_unstale":"2023-03-29T10:57:31.851569+0000","last_undegraded":"2023-03-29T10:57:31.851569+0000","last_fullsized":"2023-03-29T10:57:31.851569+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:46:29.403965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852481+0000","last_change":"2023-03-29T10:57:27.846853+0000","last_active":"2023-03-29T10:57:31.852481+0000","last_peered":"2023-03-29T10:57:31.852481+0000","last_clean":"2023-03-29T10:57:31.852481+0000","last_became_active":"2023-03-29T10:57:27.846503+0000","last_became_peered":"2023-03-29T10:57:27.846503+0000","last_unstale":"2023-03-29T10:57:31.852481+0000","last_undegraded":"2023-03-29T10:57:31.852481+0000","last_fullsized":"2023-03-29T10:57:31.852481+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:37:35.403032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.864802+0000","last_change":"2023-03-29T10:57:27.850277+0000","last_active":"2023-03-29T10:57:31.864802+0000","last_peered":"2023-03-29T10:57:31.864802+0000","last_clean":"2023-03-29T10:57:31.864802+0000","last_became_active":"2023-03-29T10:57:27.849726+0000","last_became_peered":"2023-03-29T10:57:27.849726+0000","last_unstale":"2023-03-29T10:57:31.864802+0000","last_undegraded":"2023-03-29T10:57:31.864802+0000","last_fullsized":"2023-03-29T10:57:31.864802+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:25:48.508072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340375+0000","last_change":"2023-03-29T10:57:31.874510+0000","last_active":"2023-03-29T10:57:32.340375+0000","last_peered":"2023-03-29T10:57:32.340375+0000","last_clean":"2023-03-29T10:57:32.340375+0000","last_became_active":"2023-03-29T10:57:31.874055+0000","last_became_peered":"2023-03-29T10:57:31.874055+0000","last_unstale":"2023-03-29T10:57:32.340375+0000","last_undegraded":"2023-03-29T10:57:32.340375+0000","last_fullsized":"2023-03-29T10:57:32.340375+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:53:20.999237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852831+0000","last_change":"2023-03-29T10:57:27.366688+0000","last_active":"2023-03-29T10:57:31.852831+0000","last_peered":"2023-03-29T10:57:31.852831+0000","last_clean":"2023-03-29T10:57:31.852831+0000","last_became_active":"2023-03-29T10:57:27.366396+0000","last_became_peered":"2023-03-29T10:57:27.366396+0000","last_unstale":"2023-03-29T10:57:31.852831+0000","last_undegraded":"2023-03-29T10:57:31.852831+0000","last_fullsized":"2023-03-29T10:57:31.852831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:16.312585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853217+0000","last_change":"2023-03-29T10:57:27.851695+0000","last_active":"2023-03-29T10:57:31.853217+0000","last_peered":"2023-03-29T10:57:31.853217+0000","last_clean":"2023-03-29T10:57:31.853217+0000","last_became_active":"2023-03-29T10:57:27.851344+0000","last_became_peered":"2023-03-29T10:57:27.851344+0000","last_unstale":"2023-03-29T10:57:31.853217+0000","last_undegraded":"2023-03-29T10:57:31.853217+0000","last_fullsized":"2023-03-29T10:57:31.853217+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:53:38.186916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606792+0000","last_change":"2023-03-29T10:57:31.878011+0000","last_active":"2023-03-29T10:57:32.606792+0000","last_peered":"2023-03-29T10:57:32.606792+0000","last_clean":"2023-03-29T10:57:32.606792+0000","last_became_active":"2023-03-29T10:57:31.877524+0000","last_became_peered":"2023-03-29T10:57:31.877524+0000","last_unstale":"2023-03-29T10:57:32.606792+0000","last_undegraded":"2023-03-29T10:57:32.606792+0000","last_fullsized":"2023-03-29T10:57:32.606792+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:44:51.056937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.850681+0000","last_change":"2023-03-29T10:57:27.368907+0000","last_active":"2023-03-29T10:57:31.850681+0000","last_peered":"2023-03-29T10:57:31.850681+0000","last_clean":"2023-03-29T10:57:31.850681+0000","last_became_active":"2023-03-29T10:57:27.367137+0000","last_became_peered":"2023-03-29T10:57:27.367137+0000","last_unstale":"2023-03-29T10:57:31.850681+0000","last_undegraded":"2023-03-29T10:57:31.850681+0000","last_fullsized":"2023-03-29T10:57:31.850681+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:12:14.095287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853885+0000","last_change":"2023-03-29T10:57:27.844281+0000","last_active":"2023-03-29T10:57:31.853885+0000","last_peered":"2023-03-29T10:57:31.853885+0000","last_clean":"2023-03-29T10:57:31.853885+0000","last_became_active":"2023-03-29T10:57:27.844012+0000","last_became_peered":"2023-03-29T10:57:27.844012+0000","last_unstale":"2023-03-29T10:57:31.853885+0000","last_undegraded":"2023-03-29T10:57:31.853885+0000","last_fullsized":"2023-03-29T10:57:31.853885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:23:24.586546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.336612+0000","last_change":"2023-03-29T10:57:31.872397+0000","last_active":"2023-03-29T10:57:32.336612+0000","last_peered":"2023-03-29T10:57:32.336612+0000","last_clean":"2023-03-29T10:57:32.336612+0000","last_became_active":"2023-03-29T10:57:31.872073+0000","last_became_peered":"2023-03-29T10:57:31.872073+0000","last_unstale":"2023-03-29T10:57:32.336612+0000","last_undegraded":"2023-03-29T10:57:32.336612+0000","last_fullsized":"2023-03-29T10:57:32.336612+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:24:14.450310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849730+0000","last_change":"2023-03-29T10:57:27.362811+0000","last_active":"2023-03-29T10:57:31.849730+0000","last_peered":"2023-03-29T10:57:31.849730+0000","last_clean":"2023-03-29T10:57:31.849730+0000","last_became_active":"2023-03-29T10:57:27.362313+0000","last_became_peered":"2023-03-29T10:57:27.362313+0000","last_unstale":"2023-03-29T10:57:31.849730+0000","last_undegraded":"2023-03-29T10:57:31.849730+0000","last_fullsized":"2023-03-29T10:57:31.849730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:57:18.786945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.857696+0000","last_change":"2023-03-29T10:57:27.853328+0000","last_active":"2023-03-29T10:57:31.857696+0000","last_peered":"2023-03-29T10:57:31.857696+0000","last_clean":"2023-03-29T10:57:31.857696+0000","last_became_active":"2023-03-29T10:57:27.852980+0000","last_became_peered":"2023-03-29T10:57:27.852980+0000","last_unstale":"2023-03-29T10:57:31.857696+0000","last_undegraded":"2023-03-29T10:57:31.857696+0000","last_fullsized":"2023-03-29T10:57:31.857696+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:35:25.612159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606385+0000","last_change":"2023-03-29T10:57:31.880406+0000","last_active":"2023-03-29T10:57:32.606385+0000","last_peered":"2023-03-29T10:57:32.606385+0000","last_clean":"2023-03-29T10:57:32.606385+0000","last_became_active":"2023-03-29T10:57:31.879910+0000","last_became_peered":"2023-03-29T10:57:31.879910+0000","last_unstale":"2023-03-29T10:57:32.606385+0000","last_undegraded":"2023-03-29T10:57:32.606385+0000","last_fullsized":"2023-03-29T10:57:32.606385+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:50:03.793655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852066+0000","last_change":"2023-03-29T10:57:27.361203+0000","last_active":"2023-03-29T10:57:31.852066+0000","last_peered":"2023-03-29T10:57:31.852066+0000","last_clean":"2023-03-29T10:57:31.852066+0000","last_became_active":"2023-03-29T10:57:27.360905+0000","last_became_peered":"2023-03-29T10:57:27.360905+0000","last_unstale":"2023-03-29T10:57:31.852066+0000","last_undegraded":"2023-03-29T10:57:31.852066+0000","last_fullsized":"2023-03-29T10:57:31.852066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:53:38.134633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849652+0000","last_change":"2023-03-29T10:57:27.848599+0000","last_active":"2023-03-29T10:57:31.849652+0000","last_peered":"2023-03-29T10:57:31.849652+0000","last_clean":"2023-03-29T10:57:31.849652+0000","last_became_active":"2023-03-29T10:57:27.848300+0000","last_became_peered":"2023-03-29T10:57:27.848300+0000","last_unstale":"2023-03-29T10:57:31.849652+0000","last_undegraded":"2023-03-29T10:57:31.849652+0000","last_fullsized":"2023-03-29T10:57:31.849652+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:44:47.553332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340886+0000","last_change":"2023-03-29T10:57:31.875820+0000","last_active":"2023-03-29T10:57:32.340886+0000","last_peered":"2023-03-29T10:57:32.340886+0000","last_clean":"2023-03-29T10:57:32.340886+0000","last_became_active":"2023-03-29T10:57:31.875285+0000","last_became_peered":"2023-03-29T10:57:31.875285+0000","last_unstale":"2023-03-29T10:57:32.340886+0000","last_undegraded":"2023-03-29T10:57:32.340886+0000","last_fullsized":"2023-03-29T10:57:32.340886+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:05:02.661508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.862158+0000","last_change":"2023-03-29T10:57:27.369635+0000","last_active":"2023-03-29T10:57:31.862158+0000","last_peered":"2023-03-29T10:57:31.862158+0000","last_clean":"2023-03-29T10:57:31.862158+0000","last_became_active":"2023-03-29T10:57:27.369232+0000","last_became_peered":"2023-03-29T10:57:27.369232+0000","last_unstale":"2023-03-29T10:57:31.862158+0000","last_undegraded":"2023-03-29T10:57:31.862158+0000","last_fullsized":"2023-03-29T10:57:31.862158+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:45:12.326563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852705+0000","last_change":"2023-03-29T10:57:27.841973+0000","last_active":"2023-03-29T10:57:31.852705+0000","last_peered":"2023-03-29T10:57:31.852705+0000","last_clean":"2023-03-29T10:57:31.852705+0000","last_became_active":"2023-03-29T10:57:27.841668+0000","last_became_peered":"2023-03-29T10:57:27.841668+0000","last_unstale":"2023-03-29T10:57:31.852705+0000","last_undegraded":"2023-03-29T10:57:31.852705+0000","last_fullsized":"2023-03-29T10:57:31.852705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:25:48.508072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.608208+0000","last_change":"2023-03-29T10:57:31.867435+0000","last_active":"2023-03-29T10:57:32.608208+0000","last_peered":"2023-03-29T10:57:32.608208+0000","last_clean":"2023-03-29T10:57:32.608208+0000","last_became_active":"2023-03-29T10:57:31.867024+0000","last_became_peered":"2023-03-29T10:57:31.867024+0000","last_unstale":"2023-03-29T10:57:32.608208+0000","last_undegraded":"2023-03-29T10:57:32.608208+0000","last_fullsized":"2023-03-29T10:57:32.608208+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:33:59.083704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846915+0000","last_change":"2023-03-29T10:57:27.367580+0000","last_active":"2023-03-29T10:57:31.846915+0000","last_peered":"2023-03-29T10:57:31.846915+0000","last_clean":"2023-03-29T10:57:31.846915+0000","last_became_active":"2023-03-29T10:57:27.367297+0000","last_became_peered":"2023-03-29T10:57:27.367297+0000","last_unstale":"2023-03-29T10:57:31.846915+0000","last_undegraded":"2023-03-29T10:57:31.846915+0000","last_fullsized":"2023-03-29T10:57:31.846915+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:35:24.603180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851381+0000","last_change":"2023-03-29T10:57:27.868674+0000","last_active":"2023-03-29T10:57:31.851381+0000","last_peered":"2023-03-29T10:57:31.851381+0000","last_clean":"2023-03-29T10:57:31.851381+0000","last_became_active":"2023-03-29T10:57:27.868480+0000","last_became_peered":"2023-03-29T10:57:27.868480+0000","last_unstale":"2023-03-29T10:57:31.851381+0000","last_undegraded":"2023-03-29T10:57:31.851381+0000","last_fullsized":"2023-03-29T10:57:31.851381+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:24:08.253556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.543504+0000","last_change":"2023-03-29T10:57:31.865358+0000","last_active":"2023-03-29T10:57:32.543504+0000","last_peered":"2023-03-29T10:57:32.543504+0000","last_clean":"2023-03-29T10:57:32.543504+0000","last_became_active":"2023-03-29T10:57:31.864834+0000","last_became_peered":"2023-03-29T10:57:31.864834+0000","last_unstale":"2023-03-29T10:57:32.543504+0000","last_undegraded":"2023-03-29T10:57:32.543504+0000","last_fullsized":"2023-03-29T10:57:32.543504+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:40:55.333034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:44.820956+0000","last_change":"2023-03-29T10:57:27.361696+0000","last_active":"2023-03-29T10:57:44.820956+0000","last_peered":"2023-03-29T10:57:44.820956+0000","last_clean":"2023-03-29T10:57:44.820956+0000","last_became_active":"2023-03-29T10:57:27.360021+0000","last_became_peered":"2023-03-29T10:57:27.360021+0000","last_unstale":"2023-03-29T10:57:44.820956+0000","last_undegraded":"2023-03-29T10:57:44.820956+0000","last_fullsized":"2023-03-29T10:57:44.820956+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:57:18.786945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853468+0000","last_change":"2023-03-29T10:57:27.860824+0000","last_active":"2023-03-29T10:57:31.853468+0000","last_peered":"2023-03-29T10:57:31.853468+0000","last_clean":"2023-03-29T10:57:31.853468+0000","last_became_active":"2023-03-29T10:57:27.860139+0000","last_became_peered":"2023-03-29T10:57:27.860139+0000","last_unstale":"2023-03-29T10:57:31.853468+0000","last_undegraded":"2023-03-29T10:57:31.853468+0000","last_fullsized":"2023-03-29T10:57:31.853468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:10:11.279944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606214+0000","last_change":"2023-03-29T10:57:31.869234+0000","last_active":"2023-03-29T10:57:32.606214+0000","last_peered":"2023-03-29T10:57:32.606214+0000","last_clean":"2023-03-29T10:57:32.606214+0000","last_became_active":"2023-03-29T10:57:31.868479+0000","last_became_peered":"2023-03-29T10:57:31.868479+0000","last_unstale":"2023-03-29T10:57:32.606214+0000","last_undegraded":"2023-03-29T10:57:32.606214+0000","last_fullsized":"2023-03-29T10:57:32.606214+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:29:07.043419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846875+0000","last_change":"2023-03-29T10:57:27.363811+0000","last_active":"2023-03-29T10:57:31.846875+0000","last_peered":"2023-03-29T10:57:31.846875+0000","last_clean":"2023-03-29T10:57:31.846875+0000","last_became_active":"2023-03-29T10:57:27.362755+0000","last_became_peered":"2023-03-29T10:57:27.362755+0000","last_unstale":"2023-03-29T10:57:31.846875+0000","last_undegraded":"2023-03-29T10:57:31.846875+0000","last_fullsized":"2023-03-29T10:57:31.846875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:45:12.326563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849249+0000","last_change":"2023-03-29T10:57:27.848024+0000","last_active":"2023-03-29T10:57:31.849249+0000","last_peered":"2023-03-29T10:57:31.849249+0000","last_clean":"2023-03-29T10:57:31.849249+0000","last_became_active":"2023-03-29T10:57:27.847742+0000","last_became_peered":"2023-03-29T10:57:27.847742+0000","last_unstale":"2023-03-29T10:57:31.849249+0000","last_undegraded":"2023-03-29T10:57:31.849249+0000","last_fullsized":"2023-03-29T10:57:31.849249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:07:19.154573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.608195+0000","last_change":"2023-03-29T10:57:31.874176+0000","last_active":"2023-03-29T10:57:32.608195+0000","last_peered":"2023-03-29T10:57:32.608195+0000","last_clean":"2023-03-29T10:57:32.608195+0000","last_became_active":"2023-03-29T10:57:31.873748+0000","last_became_peered":"2023-03-29T10:57:31.873748+0000","last_unstale":"2023-03-29T10:57:32.608195+0000","last_undegraded":"2023-03-29T10:57:32.608195+0000","last_fullsized":"2023-03-29T10:57:32.608195+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:29:07.043419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851278+0000","last_change":"2023-03-29T10:57:27.353170+0000","last_active":"2023-03-29T10:57:31.851278+0000","last_peered":"2023-03-29T10:57:31.851278+0000","last_clean":"2023-03-29T10:57:31.851278+0000","last_became_active":"2023-03-29T10:57:27.352891+0000","last_became_peered":"2023-03-29T10:57:27.352891+0000","last_unstale":"2023-03-29T10:57:31.851278+0000","last_undegraded":"2023-03-29T10:57:31.851278+0000","last_fullsized":"2023-03-29T10:57:31.851278+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:42:03.146529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.864725+0000","last_change":"2023-03-29T10:57:27.850083+0000","last_active":"2023-03-29T10:57:31.864725+0000","last_peered":"2023-03-29T10:57:31.864725+0000","last_clean":"2023-03-29T10:57:31.864725+0000","last_became_active":"2023-03-29T10:57:27.849341+0000","last_became_peered":"2023-03-29T10:57:27.849341+0000","last_unstale":"2023-03-29T10:57:31.864725+0000","last_undegraded":"2023-03-29T10:57:31.864725+0000","last_fullsized":"2023-03-29T10:57:31.864725+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:41.882164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.608429+0000","last_change":"2023-03-29T10:57:31.874626+0000","last_active":"2023-03-29T10:57:32.608429+0000","last_peered":"2023-03-29T10:57:32.608429+0000","last_clean":"2023-03-29T10:57:32.608429+0000","last_became_active":"2023-03-29T10:57:31.874288+0000","last_became_peered":"2023-03-29T10:57:31.874288+0000","last_unstale":"2023-03-29T10:57:32.608429+0000","last_undegraded":"2023-03-29T10:57:32.608429+0000","last_fullsized":"2023-03-29T10:57:32.608429+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:10:08.859801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847314+0000","last_change":"2023-03-29T10:57:27.371046+0000","last_active":"2023-03-29T10:57:31.847314+0000","last_peered":"2023-03-29T10:57:31.847314+0000","last_clean":"2023-03-29T10:57:31.847314+0000","last_became_active":"2023-03-29T10:57:27.370703+0000","last_became_peered":"2023-03-29T10:57:27.370703+0000","last_unstale":"2023-03-29T10:57:31.847314+0000","last_undegraded":"2023-03-29T10:57:31.847314+0000","last_fullsized":"2023-03-29T10:57:31.847314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:00:38.166723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853955+0000","last_change":"2023-03-29T10:57:27.857805+0000","last_active":"2023-03-29T10:57:31.853955+0000","last_peered":"2023-03-29T10:57:31.853955+0000","last_clean":"2023-03-29T10:57:31.853955+0000","last_became_active":"2023-03-29T10:57:27.857520+0000","last_became_peered":"2023-03-29T10:57:27.857520+0000","last_unstale":"2023-03-29T10:57:31.853955+0000","last_undegraded":"2023-03-29T10:57:31.853955+0000","last_fullsized":"2023-03-29T10:57:31.853955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:24:08.253556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605664+0000","last_change":"2023-03-29T10:57:31.868093+0000","last_active":"2023-03-29T10:57:32.605664+0000","last_peered":"2023-03-29T10:57:32.605664+0000","last_clean":"2023-03-29T10:57:32.605664+0000","last_became_active":"2023-03-29T10:57:31.867516+0000","last_became_peered":"2023-03-29T10:57:31.867516+0000","last_unstale":"2023-03-29T10:57:32.605664+0000","last_undegraded":"2023-03-29T10:57:32.605664+0000","last_fullsized":"2023-03-29T10:57:32.605664+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:56:12.713461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852187+0000","last_change":"2023-03-29T10:57:27.366626+0000","last_active":"2023-03-29T10:57:31.852187+0000","last_peered":"2023-03-29T10:57:31.852187+0000","last_clean":"2023-03-29T10:57:31.852187+0000","last_became_active":"2023-03-29T10:57:27.366259+0000","last_became_peered":"2023-03-29T10:57:27.366259+0000","last_unstale":"2023-03-29T10:57:31.852187+0000","last_undegraded":"2023-03-29T10:57:31.852187+0000","last_fullsized":"2023-03-29T10:57:31.852187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:01:05.024349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852636+0000","last_change":"2023-03-29T10:57:27.842305+0000","last_active":"2023-03-29T10:57:31.852636+0000","last_peered":"2023-03-29T10:57:31.852636+0000","last_clean":"2023-03-29T10:57:31.852636+0000","last_became_active":"2023-03-29T10:57:27.841993+0000","last_became_peered":"2023-03-29T10:57:27.841993+0000","last_unstale":"2023-03-29T10:57:31.852636+0000","last_undegraded":"2023-03-29T10:57:31.852636+0000","last_fullsized":"2023-03-29T10:57:31.852636+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:41.882164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340179+0000","last_change":"2023-03-29T10:57:31.874901+0000","last_active":"2023-03-29T10:57:32.340179+0000","last_peered":"2023-03-29T10:57:32.340179+0000","last_clean":"2023-03-29T10:57:32.340179+0000","last_became_active":"2023-03-29T10:57:31.874412+0000","last_became_peered":"2023-03-29T10:57:31.874412+0000","last_unstale":"2023-03-29T10:57:32.340179+0000","last_undegraded":"2023-03-29T10:57:32.340179+0000","last_fullsized":"2023-03-29T10:57:32.340179+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:19:12.936732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847695+0000","last_change":"2023-03-29T10:57:27.363942+0000","last_active":"2023-03-29T10:57:31.847695+0000","last_peered":"2023-03-29T10:57:31.847695+0000","last_clean":"2023-03-29T10:57:31.847695+0000","last_became_active":"2023-03-29T10:57:27.363509+0000","last_became_peered":"2023-03-29T10:57:27.363509+0000","last_unstale":"2023-03-29T10:57:31.847695+0000","last_undegraded":"2023-03-29T10:57:31.847695+0000","last_fullsized":"2023-03-29T10:57:31.847695+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:23:59.025438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861310+0000","last_change":"2023-03-29T10:57:27.849632+0000","last_active":"2023-03-29T10:57:31.861310+0000","last_peered":"2023-03-29T10:57:31.861310+0000","last_clean":"2023-03-29T10:57:31.861310+0000","last_became_active":"2023-03-29T10:57:27.849341+0000","last_became_peered":"2023-03-29T10:57:27.849341+0000","last_unstale":"2023-03-29T10:57:31.861310+0000","last_undegraded":"2023-03-29T10:57:31.861310+0000","last_fullsized":"2023-03-29T10:57:31.861310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:01:58.615750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.604803+0000","last_change":"2023-03-29T10:57:31.864416+0000","last_active":"2023-03-29T10:57:32.604803+0000","last_peered":"2023-03-29T10:57:32.604803+0000","last_clean":"2023-03-29T10:57:32.604803+0000","last_became_active":"2023-03-29T10:57:31.864021+0000","last_became_peered":"2023-03-29T10:57:31.864021+0000","last_unstale":"2023-03-29T10:57:32.604803+0000","last_undegraded":"2023-03-29T10:57:32.604803+0000","last_fullsized":"2023-03-29T10:57:32.604803+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:44:51.056937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847268+0000","last_change":"2023-03-29T10:57:27.365113+0000","last_active":"2023-03-29T10:57:31.847268+0000","last_peered":"2023-03-29T10:57:31.847268+0000","last_clean":"2023-03-29T10:57:31.847268+0000","last_became_active":"2023-03-29T10:57:27.363593+0000","last_became_peered":"2023-03-29T10:57:27.363593+0000","last_unstale":"2023-03-29T10:57:31.847268+0000","last_undegraded":"2023-03-29T10:57:31.847268+0000","last_fullsized":"2023-03-29T10:57:31.847268+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:10:38.094232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860900+0000","last_change":"2023-03-29T10:57:27.861487+0000","last_active":"2023-03-29T10:57:31.860900+0000","last_peered":"2023-03-29T10:57:31.860900+0000","last_clean":"2023-03-29T10:57:31.860900+0000","last_became_active":"2023-03-29T10:57:27.861218+0000","last_became_peered":"2023-03-29T10:57:27.861218+0000","last_unstale":"2023-03-29T10:57:31.860900+0000","last_undegraded":"2023-03-29T10:57:31.860900+0000","last_fullsized":"2023-03-29T10:57:31.860900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:04:59.157904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.604937+0000","last_change":"2023-03-29T10:57:31.871267+0000","last_active":"2023-03-29T10:57:32.604937+0000","last_peered":"2023-03-29T10:57:32.604937+0000","last_clean":"2023-03-29T10:57:32.604937+0000","last_became_active":"2023-03-29T10:57:31.870814+0000","last_became_peered":"2023-03-29T10:57:31.870814+0000","last_unstale":"2023-03-29T10:57:32.604937+0000","last_undegraded":"2023-03-29T10:57:32.604937+0000","last_fullsized":"2023-03-29T10:57:32.604937+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:57:20.392149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847436+0000","last_change":"2023-03-29T10:57:27.367428+0000","last_active":"2023-03-29T10:57:31.847436+0000","last_peered":"2023-03-29T10:57:31.847436+0000","last_clean":"2023-03-29T10:57:31.847436+0000","last_became_active":"2023-03-29T10:57:27.367100+0000","last_became_peered":"2023-03-29T10:57:27.367100+0000","last_unstale":"2023-03-29T10:57:31.847436+0000","last_undegraded":"2023-03-29T10:57:31.847436+0000","last_fullsized":"2023-03-29T10:57:31.847436+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:52:59.436152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.858073+0000","last_change":"2023-03-29T10:57:27.863156+0000","last_active":"2023-03-29T10:57:31.858073+0000","last_peered":"2023-03-29T10:57:31.858073+0000","last_clean":"2023-03-29T10:57:31.858073+0000","last_became_active":"2023-03-29T10:57:27.862805+0000","last_became_peered":"2023-03-29T10:57:27.862805+0000","last_unstale":"2023-03-29T10:57:31.858073+0000","last_undegraded":"2023-03-29T10:57:31.858073+0000","last_fullsized":"2023-03-29T10:57:31.858073+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:32:05.407357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605019+0000","last_change":"2023-03-29T10:57:31.878616+0000","last_active":"2023-03-29T10:57:32.605019+0000","last_peered":"2023-03-29T10:57:32.605019+0000","last_clean":"2023-03-29T10:57:32.605019+0000","last_became_active":"2023-03-29T10:57:31.878248+0000","last_became_peered":"2023-03-29T10:57:31.878248+0000","last_unstale":"2023-03-29T10:57:32.605019+0000","last_undegraded":"2023-03-29T10:57:32.605019+0000","last_fullsized":"2023-03-29T10:57:32.605019+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:53:41.690520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861723+0000","last_change":"2023-03-29T10:57:27.370499+0000","last_active":"2023-03-29T10:57:31.861723+0000","last_peered":"2023-03-29T10:57:31.861723+0000","last_clean":"2023-03-29T10:57:31.861723+0000","last_became_active":"2023-03-29T10:57:27.370048+0000","last_became_peered":"2023-03-29T10:57:27.370048+0000","last_unstale":"2023-03-29T10:57:31.861723+0000","last_undegraded":"2023-03-29T10:57:31.861723+0000","last_fullsized":"2023-03-29T10:57:31.861723+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:52:59.436152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854114+0000","last_change":"2023-03-29T10:57:27.848004+0000","last_active":"2023-03-29T10:57:31.854114+0000","last_peered":"2023-03-29T10:57:31.854114+0000","last_clean":"2023-03-29T10:57:31.854114+0000","last_became_active":"2023-03-29T10:57:27.847731+0000","last_became_peered":"2023-03-29T10:57:27.847731+0000","last_unstale":"2023-03-29T10:57:31.854114+0000","last_undegraded":"2023-03-29T10:57:31.854114+0000","last_fullsized":"2023-03-29T10:57:31.854114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:41.882164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605202+0000","last_change":"2023-03-29T10:57:31.868289+0000","last_active":"2023-03-29T10:57:32.605202+0000","last_peered":"2023-03-29T10:57:32.605202+0000","last_clean":"2023-03-29T10:57:32.605202+0000","last_became_active":"2023-03-29T10:57:31.867905+0000","last_became_peered":"2023-03-29T10:57:31.867905+0000","last_unstale":"2023-03-29T10:57:32.605202+0000","last_undegraded":"2023-03-29T10:57:32.605202+0000","last_fullsized":"2023-03-29T10:57:32.605202+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:23:28.090151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851809+0000","last_change":"2023-03-29T10:57:27.368350+0000","last_active":"2023-03-29T10:57:31.851809+0000","last_peered":"2023-03-29T10:57:31.851809+0000","last_clean":"2023-03-29T10:57:31.851809+0000","last_became_active":"2023-03-29T10:57:27.367743+0000","last_became_peered":"2023-03-29T10:57:27.367743+0000","last_unstale":"2023-03-29T10:57:31.851809+0000","last_undegraded":"2023-03-29T10:57:31.851809+0000","last_fullsized":"2023-03-29T10:57:31.851809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:16.312585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853838+0000","last_change":"2023-03-29T10:57:27.862273+0000","last_active":"2023-03-29T10:57:31.853838+0000","last_peered":"2023-03-29T10:57:31.853838+0000","last_clean":"2023-03-29T10:57:31.853838+0000","last_became_active":"2023-03-29T10:57:27.861893+0000","last_became_peered":"2023-03-29T10:57:27.861893+0000","last_unstale":"2023-03-29T10:57:31.853838+0000","last_undegraded":"2023-03-29T10:57:31.853838+0000","last_fullsized":"2023-03-29T10:57:31.853838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:34:37.113840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.543869+0000","last_change":"2023-03-29T10:57:31.877007+0000","last_active":"2023-03-29T10:57:32.543869+0000","last_peered":"2023-03-29T10:57:32.543869+0000","last_clean":"2023-03-29T10:57:32.543869+0000","last_became_active":"2023-03-29T10:57:31.876649+0000","last_became_peered":"2023-03-29T10:57:31.876649+0000","last_unstale":"2023-03-29T10:57:32.543869+0000","last_undegraded":"2023-03-29T10:57:32.543869+0000","last_fullsized":"2023-03-29T10:57:32.543869+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:19:12.936732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.858178+0000","last_change":"2023-03-29T10:57:27.370469+0000","last_active":"2023-03-29T10:57:31.858178+0000","last_peered":"2023-03-29T10:57:31.858178+0000","last_clean":"2023-03-29T10:57:31.858178+0000","last_became_active":"2023-03-29T10:57:27.370194+0000","last_became_peered":"2023-03-29T10:57:27.370194+0000","last_unstale":"2023-03-29T10:57:31.858178+0000","last_undegraded":"2023-03-29T10:57:31.858178+0000","last_fullsized":"2023-03-29T10:57:31.858178+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:42:03.146529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853028+0000","last_change":"2023-03-29T10:57:27.863980+0000","last_active":"2023-03-29T10:57:31.853028+0000","last_peered":"2023-03-29T10:57:31.853028+0000","last_clean":"2023-03-29T10:57:31.853028+0000","last_became_active":"2023-03-29T10:57:27.863503+0000","last_became_peered":"2023-03-29T10:57:27.863503+0000","last_unstale":"2023-03-29T10:57:31.853028+0000","last_undegraded":"2023-03-29T10:57:31.853028+0000","last_fullsized":"2023-03-29T10:57:31.853028+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:36:09.200571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606340+0000","last_change":"2023-03-29T10:57:31.880859+0000","last_active":"2023-03-29T10:57:32.606340+0000","last_peered":"2023-03-29T10:57:32.606340+0000","last_clean":"2023-03-29T10:57:32.606340+0000","last_became_active":"2023-03-29T10:57:31.879872+0000","last_became_peered":"2023-03-29T10:57:31.879872+0000","last_unstale":"2023-03-29T10:57:32.606340+0000","last_undegraded":"2023-03-29T10:57:32.606340+0000","last_fullsized":"2023-03-29T10:57:32.606340+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:40:04.971172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860797+0000","last_change":"2023-03-29T10:57:27.359363+0000","last_active":"2023-03-29T10:57:31.860797+0000","last_peered":"2023-03-29T10:57:31.860797+0000","last_clean":"2023-03-29T10:57:31.860797+0000","last_became_active":"2023-03-29T10:57:27.359054+0000","last_became_peered":"2023-03-29T10:57:27.359054+0000","last_unstale":"2023-03-29T10:57:31.860797+0000","last_undegraded":"2023-03-29T10:57:31.860797+0000","last_fullsized":"2023-03-29T10:57:31.860797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:42:03.146529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854683+0000","last_change":"2023-03-29T10:57:27.862443+0000","last_active":"2023-03-29T10:57:31.854683+0000","last_peered":"2023-03-29T10:57:31.854683+0000","last_clean":"2023-03-29T10:57:31.854683+0000","last_became_active":"2023-03-29T10:57:27.861796+0000","last_became_peered":"2023-03-29T10:57:27.861796+0000","last_unstale":"2023-03-29T10:57:31.854683+0000","last_undegraded":"2023-03-29T10:57:31.854683+0000","last_fullsized":"2023-03-29T10:57:31.854683+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:50:00.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340645+0000","last_change":"2023-03-29T10:57:31.875674+0000","last_active":"2023-03-29T10:57:32.340645+0000","last_peered":"2023-03-29T10:57:32.340645+0000","last_clean":"2023-03-29T10:57:32.340645+0000","last_became_active":"2023-03-29T10:57:31.875240+0000","last_became_peered":"2023-03-29T10:57:31.875240+0000","last_unstale":"2023-03-29T10:57:32.340645+0000","last_undegraded":"2023-03-29T10:57:32.340645+0000","last_fullsized":"2023-03-29T10:57:32.340645+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:36:10.088665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849570+0000","last_change":"2023-03-29T10:57:27.366426+0000","last_active":"2023-03-29T10:57:31.849570+0000","last_peered":"2023-03-29T10:57:31.849570+0000","last_clean":"2023-03-29T10:57:31.849570+0000","last_became_active":"2023-03-29T10:57:27.366129+0000","last_became_peered":"2023-03-29T10:57:27.366129+0000","last_unstale":"2023-03-29T10:57:31.849570+0000","last_undegraded":"2023-03-29T10:57:31.849570+0000","last_fullsized":"2023-03-29T10:57:31.849570+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:42:03.146529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851728+0000","last_change":"2023-03-29T10:57:27.856138+0000","last_active":"2023-03-29T10:57:31.851728+0000","last_peered":"2023-03-29T10:57:31.851728+0000","last_clean":"2023-03-29T10:57:31.851728+0000","last_became_active":"2023-03-29T10:57:27.855392+0000","last_became_peered":"2023-03-29T10:57:27.855392+0000","last_unstale":"2023-03-29T10:57:31.851728+0000","last_undegraded":"2023-03-29T10:57:31.851728+0000","last_fullsized":"2023-03-29T10:57:31.851728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:12:10.758718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.336574+0000","last_change":"2023-03-29T10:57:31.872511+0000","last_active":"2023-03-29T10:57:32.336574+0000","last_peered":"2023-03-29T10:57:32.336574+0000","last_clean":"2023-03-29T10:57:32.336574+0000","last_became_active":"2023-03-29T10:57:31.872088+0000","last_became_peered":"2023-03-29T10:57:31.872088+0000","last_unstale":"2023-03-29T10:57:32.336574+0000","last_undegraded":"2023-03-29T10:57:32.336574+0000","last_fullsized":"2023-03-29T10:57:32.336574+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:28:41.336878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852880+0000","last_change":"2023-03-29T10:57:22.318583+0000","last_active":"2023-03-29T10:57:31.852880+0000","last_peered":"2023-03-29T10:57:31.852880+0000","last_clean":"2023-03-29T10:57:31.852880+0000","last_became_active":"2023-03-29T10:57:22.318223+0000","last_became_peered":"2023-03-29T10:57:22.318223+0000","last_unstale":"2023-03-29T10:57:31.852880+0000","last_undegraded":"2023-03-29T10:57:31.852880+0000","last_fullsized":"2023-03-29T10:57:31.852880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:31:03.349241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.864963+0000","last_change":"2023-03-29T10:57:27.369314+0000","last_active":"2023-03-29T10:57:31.864963+0000","last_peered":"2023-03-29T10:57:31.864963+0000","last_clean":"2023-03-29T10:57:31.864963+0000","last_became_active":"2023-03-29T10:57:27.368989+0000","last_became_peered":"2023-03-29T10:57:27.368989+0000","last_unstale":"2023-03-29T10:57:31.864963+0000","last_undegraded":"2023-03-29T10:57:31.864963+0000","last_fullsized":"2023-03-29T10:57:31.864963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:35:24.603180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851286+0000","last_change":"2023-03-29T10:57:27.860136+0000","last_active":"2023-03-29T10:57:31.851286+0000","last_peered":"2023-03-29T10:57:31.851286+0000","last_clean":"2023-03-29T10:57:31.851286+0000","last_became_active":"2023-03-29T10:57:27.859888+0000","last_became_peered":"2023-03-29T10:57:27.859888+0000","last_unstale":"2023-03-29T10:57:31.851286+0000","last_undegraded":"2023-03-29T10:57:31.851286+0000","last_fullsized":"2023-03-29T10:57:31.851286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:23:24.586546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.336910+0000","last_change":"2023-03-29T10:57:31.877812+0000","last_active":"2023-03-29T10:57:32.336910+0000","last_peered":"2023-03-29T10:57:32.336910+0000","last_clean":"2023-03-29T10:57:32.336910+0000","last_became_active":"2023-03-29T10:57:31.877432+0000","last_became_peered":"2023-03-29T10:57:31.877432+0000","last_unstale":"2023-03-29T10:57:32.336910+0000","last_undegraded":"2023-03-29T10:57:32.336910+0000","last_fullsized":"2023-03-29T10:57:32.336910+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:19:12.936732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.863218+0000","last_change":"2023-03-29T10:57:22.320330+0000","last_active":"2023-03-29T10:57:31.863218+0000","last_peered":"2023-03-29T10:57:31.863218+0000","last_clean":"2023-03-29T10:57:31.863218+0000","last_became_active":"2023-03-29T10:57:22.320011+0000","last_became_peered":"2023-03-29T10:57:22.320011+0000","last_unstale":"2023-03-29T10:57:31.863218+0000","last_undegraded":"2023-03-29T10:57:31.863218+0000","last_fullsized":"2023-03-29T10:57:31.863218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:47:01.099069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853293+0000","last_change":"2023-03-29T10:57:27.361265+0000","last_active":"2023-03-29T10:57:31.853293+0000","last_peered":"2023-03-29T10:57:31.853293+0000","last_clean":"2023-03-29T10:57:31.853293+0000","last_became_active":"2023-03-29T10:57:27.360017+0000","last_became_peered":"2023-03-29T10:57:27.360017+0000","last_unstale":"2023-03-29T10:57:31.853293+0000","last_undegraded":"2023-03-29T10:57:31.853293+0000","last_fullsized":"2023-03-29T10:57:31.853293+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:12:14.095287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852801+0000","last_change":"2023-03-29T10:57:27.846016+0000","last_active":"2023-03-29T10:57:31.852801+0000","last_peered":"2023-03-29T10:57:31.852801+0000","last_clean":"2023-03-29T10:57:31.852801+0000","last_became_active":"2023-03-29T10:57:27.845677+0000","last_became_peered":"2023-03-29T10:57:27.845677+0000","last_unstale":"2023-03-29T10:57:31.852801+0000","last_undegraded":"2023-03-29T10:57:31.852801+0000","last_fullsized":"2023-03-29T10:57:31.852801+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T10:58:34.425561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605463+0000","last_change":"2023-03-29T10:57:31.863426+0000","last_active":"2023-03-29T10:57:32.605463+0000","last_peered":"2023-03-29T10:57:32.605463+0000","last_clean":"2023-03-29T10:57:32.605463+0000","last_became_active":"2023-03-29T10:57:31.862992+0000","last_became_peered":"2023-03-29T10:57:31.862992+0000","last_unstale":"2023-03-29T10:57:32.605463+0000","last_undegraded":"2023-03-29T10:57:32.605463+0000","last_fullsized":"2023-03-29T10:57:32.605463+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:24:11.757160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.845511+0000","last_change":"2023-03-29T10:57:22.467148+0000","last_active":"2023-03-29T10:57:31.845511+0000","last_peered":"2023-03-29T10:57:31.845511+0000","last_clean":"2023-03-29T10:57:31.845511+0000","last_became_active":"2023-03-29T10:57:22.466822+0000","last_became_peered":"2023-03-29T10:57:22.466822+0000","last_unstale":"2023-03-29T10:57:31.845511+0000","last_undegraded":"2023-03-29T10:57:31.845511+0000","last_fullsized":"2023-03-29T10:57:31.845511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:53:23.007205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851898+0000","last_change":"2023-03-29T10:57:27.364233+0000","last_active":"2023-03-29T10:57:31.851898+0000","last_peered":"2023-03-29T10:57:31.851898+0000","last_clean":"2023-03-29T10:57:31.851898+0000","last_became_active":"2023-03-29T10:57:27.363992+0000","last_became_peered":"2023-03-29T10:57:27.363992+0000","last_unstale":"2023-03-29T10:57:31.851898+0000","last_undegraded":"2023-03-29T10:57:31.851898+0000","last_fullsized":"2023-03-29T10:57:31.851898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:57:18.786945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851648+0000","last_change":"2023-03-29T10:57:27.859088+0000","last_active":"2023-03-29T10:57:31.851648+0000","last_peered":"2023-03-29T10:57:31.851648+0000","last_clean":"2023-03-29T10:57:31.851648+0000","last_became_active":"2023-03-29T10:57:27.858803+0000","last_became_peered":"2023-03-29T10:57:27.858803+0000","last_unstale":"2023-03-29T10:57:31.851648+0000","last_undegraded":"2023-03-29T10:57:31.851648+0000","last_fullsized":"2023-03-29T10:57:31.851648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:44:42.208513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.341079+0000","last_change":"2023-03-29T10:57:31.868378+0000","last_active":"2023-03-29T10:57:32.341079+0000","last_peered":"2023-03-29T10:57:32.341079+0000","last_clean":"2023-03-29T10:57:32.341079+0000","last_became_active":"2023-03-29T10:57:31.867988+0000","last_became_peered":"2023-03-29T10:57:31.867988+0000","last_unstale":"2023-03-29T10:57:32.341079+0000","last_undegraded":"2023-03-29T10:57:32.341079+0000","last_fullsized":"2023-03-29T10:57:32.341079+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:18:14.433000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861576+0000","last_change":"2023-03-29T10:57:22.321523+0000","last_active":"2023-03-29T10:57:31.861576+0000","last_peered":"2023-03-29T10:57:31.861576+0000","last_clean":"2023-03-29T10:57:31.861576+0000","last_became_active":"2023-03-29T10:57:22.321183+0000","last_became_peered":"2023-03-29T10:57:22.321183+0000","last_unstale":"2023-03-29T10:57:31.861576+0000","last_undegraded":"2023-03-29T10:57:31.861576+0000","last_fullsized":"2023-03-29T10:57:31.861576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:35:11.664657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846653+0000","last_change":"2023-03-29T10:57:18.176665+0000","last_active":"2023-03-29T10:57:31.846653+0000","last_peered":"2023-03-29T10:57:31.846653+0000","last_clean":"2023-03-29T10:57:31.846653+0000","last_became_active":"2023-03-29T10:57:18.175739+0000","last_became_peered":"2023-03-29T10:57:18.175739+0000","last_unstale":"2023-03-29T10:57:31.846653+0000","last_undegraded":"2023-03-29T10:57:31.846653+0000","last_fullsized":"2023-03-29T10:57:31.846653+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:16.275989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:16.275989+0000","last_clean_scrub_stamp":"2023-03-29T10:57:16.275989+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:06:32.872344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851023+0000","last_change":"2023-03-29T10:57:27.365796+0000","last_active":"2023-03-29T10:57:31.851023+0000","last_peered":"2023-03-29T10:57:31.851023+0000","last_clean":"2023-03-29T10:57:31.851023+0000","last_became_active":"2023-03-29T10:57:27.365435+0000","last_became_peered":"2023-03-29T10:57:27.365435+0000","last_unstale":"2023-03-29T10:57:31.851023+0000","last_undegraded":"2023-03-29T10:57:31.851023+0000","last_fullsized":"2023-03-29T10:57:31.851023+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:12:14.095287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.855069+0000","last_change":"2023-03-29T10:57:27.866559+0000","last_active":"2023-03-29T10:57:31.855069+0000","last_peered":"2023-03-29T10:57:31.855069+0000","last_clean":"2023-03-29T10:57:31.855069+0000","last_became_active":"2023-03-29T10:57:27.866250+0000","last_became_peered":"2023-03-29T10:57:27.866250+0000","last_unstale":"2023-03-29T10:57:31.855069+0000","last_undegraded":"2023-03-29T10:57:31.855069+0000","last_fullsized":"2023-03-29T10:57:31.855069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:53:00.445130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606048+0000","last_change":"2023-03-29T10:57:31.881080+0000","last_active":"2023-03-29T10:57:32.606048+0000","last_peered":"2023-03-29T10:57:32.606048+0000","last_clean":"2023-03-29T10:57:32.606048+0000","last_became_active":"2023-03-29T10:57:31.880427+0000","last_became_peered":"2023-03-29T10:57:31.880427+0000","last_unstale":"2023-03-29T10:57:32.606048+0000","last_undegraded":"2023-03-29T10:57:32.606048+0000","last_fullsized":"2023-03-29T10:57:32.606048+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:36:05.766940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847505+0000","last_change":"2023-03-29T10:57:22.322607+0000","last_active":"2023-03-29T10:57:31.847505+0000","last_peered":"2023-03-29T10:57:31.847505+0000","last_clean":"2023-03-29T10:57:31.847505+0000","last_became_active":"2023-03-29T10:57:22.321613+0000","last_became_peered":"2023-03-29T10:57:22.321613+0000","last_unstale":"2023-03-29T10:57:31.847505+0000","last_undegraded":"2023-03-29T10:57:31.847505+0000","last_fullsized":"2023-03-29T10:57:31.847505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:35:11.664657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851455+0000","last_change":"2023-03-29T10:57:27.366362+0000","last_active":"2023-03-29T10:57:31.851455+0000","last_peered":"2023-03-29T10:57:31.851455+0000","last_clean":"2023-03-29T10:57:31.851455+0000","last_became_active":"2023-03-29T10:57:27.366048+0000","last_became_peered":"2023-03-29T10:57:27.366048+0000","last_unstale":"2023-03-29T10:57:31.851455+0000","last_undegraded":"2023-03-29T10:57:31.851455+0000","last_fullsized":"2023-03-29T10:57:31.851455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:01:05.024349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851632+0000","last_change":"2023-03-29T10:57:27.847948+0000","last_active":"2023-03-29T10:57:31.851632+0000","last_peered":"2023-03-29T10:57:31.851632+0000","last_clean":"2023-03-29T10:57:31.851632+0000","last_became_active":"2023-03-29T10:57:27.847691+0000","last_became_peered":"2023-03-29T10:57:27.847691+0000","last_unstale":"2023-03-29T10:57:31.851632+0000","last_undegraded":"2023-03-29T10:57:31.851632+0000","last_fullsized":"2023-03-29T10:57:31.851632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:01:58.615750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605974+0000","last_change":"2023-03-29T10:57:31.873406+0000","last_active":"2023-03-29T10:57:32.605974+0000","last_peered":"2023-03-29T10:57:32.605974+0000","last_clean":"2023-03-29T10:57:32.605974+0000","last_became_active":"2023-03-29T10:57:31.873032+0000","last_became_peered":"2023-03-29T10:57:31.873032+0000","last_unstale":"2023-03-29T10:57:32.605974+0000","last_undegraded":"2023-03-29T10:57:32.605974+0000","last_fullsized":"2023-03-29T10:57:32.605974+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:53:41.690520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846972+0000","last_change":"2023-03-29T10:57:22.315905+0000","last_active":"2023-03-29T10:57:31.846972+0000","last_peered":"2023-03-29T10:57:31.846972+0000","last_clean":"2023-03-29T10:57:31.846972+0000","last_became_active":"2023-03-29T10:57:22.315603+0000","last_became_peered":"2023-03-29T10:57:22.315603+0000","last_unstale":"2023-03-29T10:57:31.846972+0000","last_undegraded":"2023-03-29T10:57:31.846972+0000","last_fullsized":"2023-03-29T10:57:31.846972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T10:59:44.030013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853041+0000","last_change":"2023-03-29T10:57:27.363544+0000","last_active":"2023-03-29T10:57:31.853041+0000","last_peered":"2023-03-29T10:57:31.853041+0000","last_clean":"2023-03-29T10:57:31.853041+0000","last_became_active":"2023-03-29T10:57:27.363284+0000","last_became_peered":"2023-03-29T10:57:27.363284+0000","last_unstale":"2023-03-29T10:57:31.853041+0000","last_undegraded":"2023-03-29T10:57:31.853041+0000","last_fullsized":"2023-03-29T10:57:31.853041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:16.312585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853937+0000","last_change":"2023-03-29T10:57:27.844093+0000","last_active":"2023-03-29T10:57:31.853937+0000","last_peered":"2023-03-29T10:57:31.853937+0000","last_clean":"2023-03-29T10:57:31.853937+0000","last_became_active":"2023-03-29T10:57:27.843647+0000","last_became_peered":"2023-03-29T10:57:27.843647+0000","last_unstale":"2023-03-29T10:57:31.853937+0000","last_undegraded":"2023-03-29T10:57:31.853937+0000","last_fullsized":"2023-03-29T10:57:31.853937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:46:29.403965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340198+0000","last_change":"2023-03-29T10:57:31.878229+0000","last_active":"2023-03-29T10:57:32.340198+0000","last_peered":"2023-03-29T10:57:32.340198+0000","last_clean":"2023-03-29T10:57:32.340198+0000","last_became_active":"2023-03-29T10:57:31.877846+0000","last_became_peered":"2023-03-29T10:57:31.877846+0000","last_unstale":"2023-03-29T10:57:32.340198+0000","last_undegraded":"2023-03-29T10:57:32.340198+0000","last_fullsized":"2023-03-29T10:57:32.340198+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:32:15.233141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852732+0000","last_change":"2023-03-29T10:57:27.366984+0000","last_active":"2023-03-29T10:57:31.852732+0000","last_peered":"2023-03-29T10:57:31.852732+0000","last_clean":"2023-03-29T10:57:31.852732+0000","last_became_active":"2023-03-29T10:57:27.366716+0000","last_became_peered":"2023-03-29T10:57:27.366716+0000","last_unstale":"2023-03-29T10:57:31.852732+0000","last_undegraded":"2023-03-29T10:57:31.852732+0000","last_fullsized":"2023-03-29T10:57:31.852732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:00:38.166723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852301+0000","last_change":"2023-03-29T10:57:22.319554+0000","last_active":"2023-03-29T10:57:31.852301+0000","last_peered":"2023-03-29T10:57:31.852301+0000","last_clean":"2023-03-29T10:57:31.852301+0000","last_became_active":"2023-03-29T10:57:22.319198+0000","last_became_peered":"2023-03-29T10:57:22.319198+0000","last_unstale":"2023-03-29T10:57:31.852301+0000","last_undegraded":"2023-03-29T10:57:31.852301+0000","last_fullsized":"2023-03-29T10:57:31.852301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:47:01.099069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849473+0000","last_change":"2023-03-29T10:57:27.839233+0000","last_active":"2023-03-29T10:57:31.849473+0000","last_peered":"2023-03-29T10:57:31.849473+0000","last_clean":"2023-03-29T10:57:31.849473+0000","last_became_active":"2023-03-29T10:57:27.838936+0000","last_became_peered":"2023-03-29T10:57:27.838936+0000","last_unstale":"2023-03-29T10:57:31.849473+0000","last_undegraded":"2023-03-29T10:57:31.849473+0000","last_fullsized":"2023-03-29T10:57:31.849473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:10:11.279944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.607718+0000","last_change":"2023-03-29T10:57:31.876988+0000","last_active":"2023-03-29T10:57:32.607718+0000","last_peered":"2023-03-29T10:57:32.607718+0000","last_clean":"2023-03-29T10:57:32.607718+0000","last_became_active":"2023-03-29T10:57:31.876521+0000","last_became_peered":"2023-03-29T10:57:31.876521+0000","last_unstale":"2023-03-29T10:57:32.607718+0000","last_undegraded":"2023-03-29T10:57:32.607718+0000","last_fullsized":"2023-03-29T10:57:32.607718+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:40:04.971172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.858506+0000","last_change":"2023-03-29T10:57:22.322319+0000","last_active":"2023-03-29T10:57:31.858506+0000","last_peered":"2023-03-29T10:57:31.858506+0000","last_clean":"2023-03-29T10:57:31.858506+0000","last_became_active":"2023-03-29T10:57:22.321943+0000","last_became_peered":"2023-03-29T10:57:22.321943+0000","last_unstale":"2023-03-29T10:57:31.858506+0000","last_undegraded":"2023-03-29T10:57:31.858506+0000","last_fullsized":"2023-03-29T10:57:31.858506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:58:04.969453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"28'1","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.336470+0000","last_change":"2023-03-29T10:57:27.364507+0000","last_active":"2023-03-29T10:57:32.336470+0000","last_peered":"2023-03-29T10:57:32.336470+0000","last_clean":"2023-03-29T10:57:32.336470+0000","last_became_active":"2023-03-29T10:57:27.363959+0000","last_became_peered":"2023-03-29T10:57:27.363959+0000","last_unstale":"2023-03-29T10:57:32.336470+0000","last_undegraded":"2023-03-29T10:57:32.336470+0000","last_fullsized":"2023-03-29T10:57:32.336470+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:16.312585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860854+0000","last_change":"2023-03-29T10:57:27.861040+0000","last_active":"2023-03-29T10:57:31.860854+0000","last_peered":"2023-03-29T10:57:31.860854+0000","last_clean":"2023-03-29T10:57:31.860854+0000","last_became_active":"2023-03-29T10:57:27.860750+0000","last_became_peered":"2023-03-29T10:57:27.860750+0000","last_unstale":"2023-03-29T10:57:31.860854+0000","last_undegraded":"2023-03-29T10:57:31.860854+0000","last_fullsized":"2023-03-29T10:57:31.860854+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:40:51.829430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.543747+0000","last_change":"2023-03-29T10:57:31.882689+0000","last_active":"2023-03-29T10:57:32.543747+0000","last_peered":"2023-03-29T10:57:32.543747+0000","last_clean":"2023-03-29T10:57:32.543747+0000","last_became_active":"2023-03-29T10:57:31.882298+0000","last_became_peered":"2023-03-29T10:57:31.882298+0000","last_unstale":"2023-03-29T10:57:32.543747+0000","last_undegraded":"2023-03-29T10:57:32.543747+0000","last_fullsized":"2023-03-29T10:57:32.543747+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:18:14.433000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853851+0000","last_change":"2023-03-29T10:57:27.363293+0000","last_active":"2023-03-29T10:57:31.853851+0000","last_peered":"2023-03-29T10:57:31.853851+0000","last_clean":"2023-03-29T10:57:31.853851+0000","last_became_active":"2023-03-29T10:57:27.362993+0000","last_became_peered":"2023-03-29T10:57:27.362993+0000","last_unstale":"2023-03-29T10:57:31.853851+0000","last_undegraded":"2023-03-29T10:57:31.853851+0000","last_fullsized":"2023-03-29T10:57:31.853851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:57:18.786945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.845743+0000","last_change":"2023-03-29T10:57:27.849850+0000","last_active":"2023-03-29T10:57:31.845743+0000","last_peered":"2023-03-29T10:57:31.845743+0000","last_clean":"2023-03-29T10:57:31.845743+0000","last_became_active":"2023-03-29T10:57:27.849429+0000","last_became_peered":"2023-03-29T10:57:27.849429+0000","last_unstale":"2023-03-29T10:57:31.845743+0000","last_undegraded":"2023-03-29T10:57:31.845743+0000","last_fullsized":"2023-03-29T10:57:31.845743+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:00:39.175701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.341198+0000","last_change":"2023-03-29T10:57:31.868416+0000","last_active":"2023-03-29T10:57:32.341198+0000","last_peered":"2023-03-29T10:57:32.341198+0000","last_clean":"2023-03-29T10:57:32.341198+0000","last_became_active":"2023-03-29T10:57:31.868042+0000","last_became_peered":"2023-03-29T10:57:31.868042+0000","last_unstale":"2023-03-29T10:57:32.341198+0000","last_undegraded":"2023-03-29T10:57:32.341198+0000","last_fullsized":"2023-03-29T10:57:32.341198+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:28:41.336878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861823+0000","last_change":"2023-03-29T10:57:27.373078+0000","last_active":"2023-03-29T10:57:31.861823+0000","last_peered":"2023-03-29T10:57:31.861823+0000","last_clean":"2023-03-29T10:57:31.861823+0000","last_became_active":"2023-03-29T10:57:27.372747+0000","last_became_peered":"2023-03-29T10:57:27.372747+0000","last_unstale":"2023-03-29T10:57:31.861823+0000","last_undegraded":"2023-03-29T10:57:31.861823+0000","last_fullsized":"2023-03-29T10:57:31.861823+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:23:59.025438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853187+0000","last_change":"2023-03-29T10:57:27.861043+0000","last_active":"2023-03-29T10:57:31.853187+0000","last_peered":"2023-03-29T10:57:31.853187+0000","last_clean":"2023-03-29T10:57:31.853187+0000","last_became_active":"2023-03-29T10:57:27.860522+0000","last_became_peered":"2023-03-29T10:57:27.860522+0000","last_unstale":"2023-03-29T10:57:31.853187+0000","last_undegraded":"2023-03-29T10:57:31.853187+0000","last_fullsized":"2023-03-29T10:57:31.853187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:49:24.886407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606089+0000","last_change":"2023-03-29T10:57:31.879767+0000","last_active":"2023-03-29T10:57:32.606089+0000","last_peered":"2023-03-29T10:57:32.606089+0000","last_clean":"2023-03-29T10:57:32.606089+0000","last_became_active":"2023-03-29T10:57:31.879330+0000","last_became_peered":"2023-03-29T10:57:31.879330+0000","last_unstale":"2023-03-29T10:57:32.606089+0000","last_undegraded":"2023-03-29T10:57:32.606089+0000","last_fullsized":"2023-03-29T10:57:32.606089+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:10:08.859801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.864594+0000","last_change":"2023-03-29T10:57:27.370488+0000","last_active":"2023-03-29T10:57:31.864594+0000","last_peered":"2023-03-29T10:57:31.864594+0000","last_clean":"2023-03-29T10:57:31.864594+0000","last_became_active":"2023-03-29T10:57:27.370181+0000","last_became_peered":"2023-03-29T10:57:27.370181+0000","last_unstale":"2023-03-29T10:57:31.864594+0000","last_undegraded":"2023-03-29T10:57:31.864594+0000","last_fullsized":"2023-03-29T10:57:31.864594+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:10:38.094232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852668+0000","last_change":"2023-03-29T10:57:27.860955+0000","last_active":"2023-03-29T10:57:31.852668+0000","last_peered":"2023-03-29T10:57:31.852668+0000","last_clean":"2023-03-29T10:57:31.852668+0000","last_became_active":"2023-03-29T10:57:27.860396+0000","last_became_peered":"2023-03-29T10:57:27.860396+0000","last_unstale":"2023-03-29T10:57:31.852668+0000","last_undegraded":"2023-03-29T10:57:31.852668+0000","last_fullsized":"2023-03-29T10:57:31.852668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:44:40.017537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.339309+0000","last_change":"2023-03-29T10:57:31.874679+0000","last_active":"2023-03-29T10:57:32.339309+0000","last_peered":"2023-03-29T10:57:32.339309+0000","last_clean":"2023-03-29T10:57:32.339309+0000","last_became_active":"2023-03-29T10:57:31.874124+0000","last_became_peered":"2023-03-29T10:57:31.874124+0000","last_unstale":"2023-03-29T10:57:32.339309+0000","last_undegraded":"2023-03-29T10:57:32.339309+0000","last_fullsized":"2023-03-29T10:57:32.339309+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:36:05.766940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853964+0000","last_change":"2023-03-29T10:57:27.366500+0000","last_active":"2023-03-29T10:57:31.853964+0000","last_peered":"2023-03-29T10:57:31.853964+0000","last_clean":"2023-03-29T10:57:31.853964+0000","last_became_active":"2023-03-29T10:57:27.366215+0000","last_became_peered":"2023-03-29T10:57:27.366215+0000","last_unstale":"2023-03-29T10:57:31.853964+0000","last_undegraded":"2023-03-29T10:57:31.853964+0000","last_fullsized":"2023-03-29T10:57:31.853964+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:45:12.326563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852786+0000","last_change":"2023-03-29T10:57:27.838577+0000","last_active":"2023-03-29T10:57:31.852786+0000","last_peered":"2023-03-29T10:57:31.852786+0000","last_clean":"2023-03-29T10:57:31.852786+0000","last_became_active":"2023-03-29T10:57:27.838301+0000","last_became_peered":"2023-03-29T10:57:27.838301+0000","last_unstale":"2023-03-29T10:57:31.852786+0000","last_undegraded":"2023-03-29T10:57:31.852786+0000","last_fullsized":"2023-03-29T10:57:31.852786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:36:09.200571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.543500+0000","last_change":"2023-03-29T10:57:31.874474+0000","last_active":"2023-03-29T10:57:32.543500+0000","last_peered":"2023-03-29T10:57:32.543500+0000","last_clean":"2023-03-29T10:57:32.543500+0000","last_became_active":"2023-03-29T10:57:31.874137+0000","last_became_peered":"2023-03-29T10:57:31.874137+0000","last_unstale":"2023-03-29T10:57:32.543500+0000","last_undegraded":"2023-03-29T10:57:32.543500+0000","last_fullsized":"2023-03-29T10:57:32.543500+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:10.382014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.857710+0000","last_change":"2023-03-29T10:57:27.365343+0000","last_active":"2023-03-29T10:57:31.857710+0000","last_peered":"2023-03-29T10:57:31.857710+0000","last_clean":"2023-03-29T10:57:31.857710+0000","last_became_active":"2023-03-29T10:57:27.365066+0000","last_became_peered":"2023-03-29T10:57:27.365066+0000","last_unstale":"2023-03-29T10:57:31.857710+0000","last_undegraded":"2023-03-29T10:57:31.857710+0000","last_fullsized":"2023-03-29T10:57:31.857710+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:40:18.602235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860006+0000","last_change":"2023-03-29T10:57:27.860635+0000","last_active":"2023-03-29T10:57:31.860006+0000","last_peered":"2023-03-29T10:57:31.860006+0000","last_clean":"2023-03-29T10:57:31.860006+0000","last_became_active":"2023-03-29T10:57:27.860287+0000","last_became_peered":"2023-03-29T10:57:27.860287+0000","last_unstale":"2023-03-29T10:57:31.860006+0000","last_undegraded":"2023-03-29T10:57:31.860006+0000","last_fullsized":"2023-03-29T10:57:31.860006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:57:16.888544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852270+0000","last_change":"2023-03-29T10:57:27.858882+0000","last_active":"2023-03-29T10:57:31.852270+0000","last_peered":"2023-03-29T10:57:31.852270+0000","last_clean":"2023-03-29T10:57:31.852270+0000","last_became_active":"2023-03-29T10:57:27.858471+0000","last_became_peered":"2023-03-29T10:57:27.858471+0000","last_unstale":"2023-03-29T10:57:31.852270+0000","last_undegraded":"2023-03-29T10:57:31.852270+0000","last_fullsized":"2023-03-29T10:57:31.852270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:10:39.103210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852924+0000","last_change":"2023-03-29T10:57:27.848104+0000","last_active":"2023-03-29T10:57:31.852924+0000","last_peered":"2023-03-29T10:57:31.852924+0000","last_clean":"2023-03-29T10:57:31.852924+0000","last_became_active":"2023-03-29T10:57:27.847812+0000","last_became_peered":"2023-03-29T10:57:27.847812+0000","last_unstale":"2023-03-29T10:57:31.852924+0000","last_undegraded":"2023-03-29T10:57:31.852924+0000","last_fullsized":"2023-03-29T10:57:31.852924+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:50:00.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852955+0000","last_change":"2023-03-29T10:57:27.868878+0000","last_active":"2023-03-29T10:57:31.852955+0000","last_peered":"2023-03-29T10:57:31.852955+0000","last_clean":"2023-03-29T10:57:31.852955+0000","last_became_active":"2023-03-29T10:57:27.868603+0000","last_became_peered":"2023-03-29T10:57:27.868603+0000","last_unstale":"2023-03-29T10:57:31.852955+0000","last_undegraded":"2023-03-29T10:57:31.852955+0000","last_fullsized":"2023-03-29T10:57:31.852955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:53:38.186916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851718+0000","last_change":"2023-03-29T10:57:27.853683+0000","last_active":"2023-03-29T10:57:31.851718+0000","last_peered":"2023-03-29T10:57:31.851718+0000","last_clean":"2023-03-29T10:57:31.851718+0000","last_became_active":"2023-03-29T10:57:27.853396+0000","last_became_peered":"2023-03-29T10:57:27.853396+0000","last_unstale":"2023-03-29T10:57:31.851718+0000","last_undegraded":"2023-03-29T10:57:31.851718+0000","last_fullsized":"2023-03-29T10:57:31.851718+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:57:16.888544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852962+0000","last_change":"2023-03-29T10:57:27.856369+0000","last_active":"2023-03-29T10:57:31.852962+0000","last_peered":"2023-03-29T10:57:31.852962+0000","last_clean":"2023-03-29T10:57:31.852962+0000","last_became_active":"2023-03-29T10:57:27.856118+0000","last_became_peered":"2023-03-29T10:57:27.856118+0000","last_unstale":"2023-03-29T10:57:31.852962+0000","last_undegraded":"2023-03-29T10:57:31.852962+0000","last_fullsized":"2023-03-29T10:57:31.852962+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:33:37.107795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4598,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9296,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362544,"statfs":{"total":96636764160,"available":96627245056,"internally_reserved":0,"allocated":925696,"data_stored":718956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8112,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363728,"statfs":{"total":96636764160,"available":96628457472,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8176,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363664,"statfs":{"total":96636764160,"available":96628391936,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9268,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362572,"statfs":{"total":96636764160,"available":96627273728,"internally_reserved":0,"allocated":897024,"data_stored":714903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8108,"kb_used_data":292,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363732,"statfs":{"total":96636764160,"available":96628461568,"internally_reserved":0,"allocated":299008,"data_stored":123953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8180,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363660,"statfs":{"total":96636764160,"available":96628387840,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8260,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363580,"statfs":{"total":96636764160,"available":96628305920,"internally_reserved":0,"allocated":323584,"data_stored":128428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8184,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363656,"statfs":{"total":96636764160,"available":96628383744,"internally_reserved":0,"allocated":311296,"data_stored":125224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-03-29T10:57:55.167 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-03-29T10:57:55.168 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-03-29T10:57:55.168 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-03-29T10:57:55.168 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-03-29T10:57:55.168 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-03-29T10:57:55.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.372+0000 7fd042bff700 1 Processor -- start 2023-03-29T10:57:55.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.372+0000 7fd042bff700 1 -- start start 2023-03-29T10:57:55.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.373+0000 7fd042bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd03c0b88e0 0x7fd03c0b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:55.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.373+0000 7fd042bff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd03c0cf190 0x7fd03c0cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:55.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.374+0000 7fd042bff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd03c1611d0 con 0x7fd03c160c00 2023-03-29T10:57:55.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.374+0000 7fd042bff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03c0d2e60 con 0x7fd03c0cf190 2023-03-29T10:57:55.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.374+0000 7fd042bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03c0cd830 con 0x7fd03c0b88e0 2023-03-29T10:57:55.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.374+0000 7fd03bfff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fd03c160c00 0x7fd03c0b4f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41642/0 (socket says 172.21.15.111:41642) 2023-03-29T10:57:55.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.374+0000 7fd03bfff700 1 -- 172.21.15.111:0/3804809748 learned_addr learned my addr 172.21.15.111:0/3804809748 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:55.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.374+0000 7fd04119c700 1 --2- 172.21.15.111:0/3804809748 >> v2:172.21.15.111:3300/0 conn(0x7fd03c0cf190 0x7fd03c0cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:55.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.374+0000 7fd04099b700 1 --2- 172.21.15.111:0/3804809748 >> v2:172.21.15.119:3300/0 conn(0x7fd03c0b88e0 0x7fd03c0b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:55.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.374+0000 7fd03b7fe700 1 -- 172.21.15.111:0/3804809748 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940482588 0 0) 0x7fd028001110 con 0x7fd03c160c00 2023-03-29T10:57:55.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.374+0000 7fd03b7fe700 1 -- 172.21.15.111:0/3804809748 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd02c0035b0 con 0x7fd03c160c00 2023-03-29T10:57:55.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.374+0000 7fd04119c700 1 -- 172.21.15.111:0/3804809748 >> v2:172.21.15.119:3300/0 conn(0x7fd03c0b88e0 msgr2=0x7fd03c0b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:55.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.374+0000 7fd04119c700 1 --2- 172.21.15.111:0/3804809748 >> v2:172.21.15.119:3300/0 conn(0x7fd03c0b88e0 0x7fd03c0b4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.374+0000 7fd04119c700 1 -- 172.21.15.111:0/3804809748 >> v1:172.21.15.111:6789/0 conn(0x7fd03c160c00 legacy=0x7fd03c0b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:55.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.374+0000 7fd04119c700 1 -- 172.21.15.111:0/3804809748 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c0cdc50 con 0x7fd03c0cf190 2023-03-29T10:57:55.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.375+0000 7fd04119c700 1 --2- 172.21.15.111:0/3804809748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c0cf190 0x7fd03c0cf610 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd03c08bec0 tx=0x7fd03400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7bbb9058c182dd server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:55.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.375+0000 7fd03b7fe700 1 -- 172.21.15.111:0/3804809748 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03400e070 con 0x7fd03c0cf190 2023-03-29T10:57:55.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.375+0000 7fd03b7fe700 1 -- 172.21.15.111:0/3804809748 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03400bb20 con 0x7fd03c0cf190 2023-03-29T10:57:55.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.375+0000 7fd03b7fe700 1 -- 172.21.15.111:0/3804809748 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034012430 con 0x7fd03c0cf190 2023-03-29T10:57:55.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.375+0000 7fd042bff700 1 -- 172.21.15.111:0/3804809748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c0cf190 msgr2=0x7fd03c0cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:55.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.375+0000 7fd042bff700 1 --2- 172.21.15.111:0/3804809748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c0cf190 0x7fd03c0cf610 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd03c08bec0 tx=0x7fd03400b330 comp rx=0 tx=0).stop 2023-03-29T10:57:55.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.375+0000 7fd042bff700 1 -- 172.21.15.111:0/3804809748 shutdown_connections 2023-03-29T10:57:55.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.375+0000 7fd042bff700 1 --2- 172.21.15.111:0/3804809748 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c0cf190 0x7fd03c0cf610 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.375+0000 7fd042bff700 1 --2- 172.21.15.111:0/3804809748 >> v2:172.21.15.119:3300/0 conn(0x7fd03c0b88e0 0x7fd03c0b4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.375+0000 7fd042bff700 1 -- 172.21.15.111:0/3804809748 >> 172.21.15.111:0/3804809748 conn(0x7fd03c0666b0 msgr2=0x7fd03c071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:55.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.376+0000 7fd042bff700 1 -- 172.21.15.111:0/3804809748 shutdown_connections 2023-03-29T10:57:55.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.376+0000 7fd042bff700 1 -- 172.21.15.111:0/3804809748 wait complete. 2023-03-29T10:57:55.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.377+0000 7fd042bff700 1 Processor -- start 2023-03-29T10:57:55.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.377+0000 7fd042bff700 1 -- start start 2023-03-29T10:57:55.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.378+0000 7fd042bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd03c0cf190 0x7fd03c0b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:55.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.378+0000 7fd042bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd03c160c00 0x7fd03c0c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:55.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.378+0000 7fd042bff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c150310 0x7fd03c0b5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:55.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.378+0000 7fd042bff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c0b40b0 con 0x7fd03c150310 2023-03-29T10:57:55.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.378+0000 7fd042bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c0b4390 con 0x7fd03c0cf190 2023-03-29T10:57:55.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.378+0000 7fd042bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c0b4670 con 0x7fd03c160c00 2023-03-29T10:57:55.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.378+0000 7fd04119c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c150310 0x7fd03c0b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:55.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.379+0000 7fd04119c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c150310 0x7fd03c0b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38252/0 (socket says 172.21.15.111:38252) 2023-03-29T10:57:55.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.379+0000 7fd04119c700 1 -- 172.21.15.111:0/2709109071 learned_addr learned my addr 172.21.15.111:0/2709109071 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:55.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.379+0000 7fd03bfff700 1 --2- 172.21.15.111:0/2709109071 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd03c160c00 0x7fd03c0c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:55.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.379+0000 7fd04099b700 1 --2- 172.21.15.111:0/2709109071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd03c0cf190 0x7fd03c0b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:55.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.379+0000 7fd04119c700 1 -- 172.21.15.111:0/2709109071 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd03c160c00 msgr2=0x7fd03c0c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:55.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.379+0000 7fd04119c700 1 --2- 172.21.15.111:0/2709109071 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd03c160c00 0x7fd03c0c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.379+0000 7fd04119c700 1 -- 172.21.15.111:0/2709109071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd03c0cf190 msgr2=0x7fd03c0b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:55.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.379+0000 7fd04119c700 1 --2- 172.21.15.111:0/2709109071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd03c0cf190 0x7fd03c0b3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.379+0000 7fd04119c700 1 -- 172.21.15.111:0/2709109071 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd034009a70 con 0x7fd03c150310 2023-03-29T10:57:55.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.379+0000 7fd04119c700 1 --2- 172.21.15.111:0/2709109071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c150310 0x7fd03c0b5150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd034014040 tx=0x7fd034004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:55.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.379+0000 7fd0397fa700 1 -- 172.21.15.111:0/2709109071 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0340129f0 con 0x7fd03c150310 2023-03-29T10:57:55.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.380+0000 7fd0397fa700 1 -- 172.21.15.111:0/2709109071 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd034020070 con 0x7fd03c150310 2023-03-29T10:57:55.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.380+0000 7fd0397fa700 1 -- 172.21.15.111:0/2709109071 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034012d90 con 0x7fd03c150310 2023-03-29T10:57:55.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.380+0000 7fd042bff700 1 -- 172.21.15.111:0/2709109071 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03c0d2b80 con 0x7fd03c150310 2023-03-29T10:57:55.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.380+0000 7fd042bff700 1 -- 172.21.15.111:0/2709109071 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03c0d28a0 con 0x7fd03c150310 2023-03-29T10:57:55.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.381+0000 7fd042bff700 1 -- 172.21.15.111:0/2709109071 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03c0d0c30 con 0x7fd03c150310 2023-03-29T10:57:55.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.382+0000 7fd0397fa700 1 -- 172.21.15.111:0/2709109071 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fd034004040 con 0x7fd03c150310 2023-03-29T10:57:55.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.383+0000 7fd0397fa700 1 --2- 172.21.15.111:0/2709109071 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd020069de0 0x7fd02006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:55.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.383+0000 7fd0397fa700 1 -- 172.21.15.111:0/2709109071 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd034022070 con 0x7fd03c150310 2023-03-29T10:57:55.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.383+0000 7fd04099b700 1 --2- 172.21.15.111:0/2709109071 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd020069de0 0x7fd02006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:55.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.384+0000 7fd04099b700 1 --2- 172.21.15.111:0/2709109071 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd020069de0 0x7fd02006c290 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd03c06c1a0 tx=0x7fd030009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:55.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.389+0000 7fd0397fa700 1 -- 172.21.15.111:0/2709109071 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fd034055570 con 0x7fd03c150310 2023-03-29T10:57:55.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.635+0000 7fd042bff700 1 -- 172.21.15.111:0/2709109071 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd03c0ceb90 con 0x7fd03c150310 2023-03-29T10:57:55.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.638+0000 7fd0397fa700 1 -- 172.21.15.111:0/2709109071 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+168749 (secure 0 0 0) 0x7fd03402c020 con 0x7fd03c150310 2023-03-29T10:57:55.640 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:55.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.642+0000 7fd042bff700 1 -- 172.21.15.111:0/2709109071 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd020069de0 msgr2=0x7fd02006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:55.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.642+0000 7fd042bff700 1 --2- 172.21.15.111:0/2709109071 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd020069de0 0x7fd02006c290 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd03c06c1a0 tx=0x7fd030009210 comp rx=0 tx=0).stop 2023-03-29T10:57:55.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.642+0000 7fd042bff700 1 -- 172.21.15.111:0/2709109071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c150310 msgr2=0x7fd03c0b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:55.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.642+0000 7fd042bff700 1 --2- 172.21.15.111:0/2709109071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c150310 0x7fd03c0b5150 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd034014040 tx=0x7fd034004ad0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.642+0000 7fd042bff700 1 -- 172.21.15.111:0/2709109071 shutdown_connections 2023-03-29T10:57:55.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.642+0000 7fd042bff700 1 --2- 172.21.15.111:0/2709109071 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd020069de0 0x7fd02006c290 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.642+0000 7fd042bff700 1 --2- 172.21.15.111:0/2709109071 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd03c150310 0x7fd03c0b5150 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.642+0000 7fd042bff700 1 --2- 172.21.15.111:0/2709109071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd03c0cf190 0x7fd03c0b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.642+0000 7fd042bff700 1 --2- 172.21.15.111:0/2709109071 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd03c160c00 0x7fd03c0c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.642+0000 7fd042bff700 1 -- 172.21.15.111:0/2709109071 >> 172.21.15.111:0/2709109071 conn(0x7fd03c0666b0 msgr2=0x7fd03c15cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:55.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.642+0000 7fd042bff700 1 -- 172.21.15.111:0/2709109071 shutdown_connections 2023-03-29T10:57:55.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.642+0000 7fd042bff700 1 -- 172.21.15.111:0/2709109071 wait complete. 2023-03-29T10:57:55.659 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":5,"active_gid":4103,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6832","nonce":110141},{"type":"v1","addr":"172.21.15.111:6833","nonce":110141}]},"active_addr":"172.21.15.111:6833/110141","active_change":"2023-03-29T10:57:08.108369+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4100,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":723585259}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":807798239}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":2794639889}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.111:0","nonce":1113657470}]}]} 2023-03-29T10:57:55.667 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-03-29T10:57:55.667 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-03-29T10:57:55.667 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:57:55.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.848+0000 7fd4803ff700 1 Processor -- start 2023-03-29T10:57:55.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.849+0000 7fd4803ff700 1 -- start start 2023-03-29T10:57:55.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.849+0000 7fd4803ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4780cf970 0x7fd4780cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:55.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.850+0000 7fd4803ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4780cfd40 0x7fd4780d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:55.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.850+0000 7fd4803ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4780cdc10 con 0x7fd478160290 2023-03-29T10:57:55.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.850+0000 7fd4803ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4780b6c60 con 0x7fd4780cfd40 2023-03-29T10:57:55.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.850+0000 7fd4803ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4780cd650 con 0x7fd4780cf970 2023-03-29T10:57:55.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.850+0000 7fd47e19b700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4780cf970 0x7fd4780cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:55.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.850+0000 7fd47e99c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd478160290 0x7fd4781606d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.111:35106/0 (socket says 172.21.15.111:35106) 2023-03-29T10:57:55.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.850+0000 7fd47e99c700 1 -- 172.21.15.111:0/3586162944 learned_addr learned my addr 172.21.15.111:0/3586162944 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:55.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.850+0000 7fd47d99a700 1 --2- 172.21.15.111:0/3586162944 >> v2:172.21.15.119:3300/0 conn(0x7fd4780cfd40 0x7fd4780d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:55.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.851+0000 7fd47d99a700 1 -- 172.21.15.111:0/3586162944 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4780cf970 msgr2=0x7fd4780cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:55.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.851+0000 7fd47d99a700 1 --2- 172.21.15.111:0/3586162944 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4780cf970 0x7fd4780cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.851+0000 7fd47d99a700 1 -- 172.21.15.111:0/3586162944 >> v1:172.21.15.119:6789/0 conn(0x7fd478160290 legacy=0x7fd4781606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:55.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.851+0000 7fd47d99a700 1 -- 172.21.15.111:0/3586162944 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4780cd930 con 0x7fd4780cfd40 2023-03-29T10:57:55.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.851+0000 7fd47d99a700 1 --2- 172.21.15.111:0/3586162944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4780cfd40 0x7fd4780d01b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd468009840 tx=0x7fd468009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e69bfe9dfc66325b server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:55.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.851+0000 7fd47d199700 1 -- 172.21.15.111:0/3586162944 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd468011830 con 0x7fd4780cfd40 2023-03-29T10:57:55.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.851+0000 7fd47d199700 1 -- 172.21.15.111:0/3586162944 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd468011990 con 0x7fd4780cfd40 2023-03-29T10:57:55.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.851+0000 7fd47d199700 1 -- 172.21.15.111:0/3586162944 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd468011d20 con 0x7fd4780cfd40 2023-03-29T10:57:55.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.852+0000 7fd4803ff700 1 -- 172.21.15.111:0/3586162944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4780cfd40 msgr2=0x7fd4780d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:55.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.852+0000 7fd4803ff700 1 --2- 172.21.15.111:0/3586162944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4780cfd40 0x7fd4780d01b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd468009840 tx=0x7fd468009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:55.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.852+0000 7fd4803ff700 1 -- 172.21.15.111:0/3586162944 shutdown_connections 2023-03-29T10:57:55.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.852+0000 7fd4803ff700 1 --2- 172.21.15.111:0/3586162944 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4780cf970 0x7fd4780cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.852+0000 7fd4803ff700 1 --2- 172.21.15.111:0/3586162944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4780cfd40 0x7fd4780d01b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.852+0000 7fd4803ff700 1 -- 172.21.15.111:0/3586162944 >> 172.21.15.111:0/3586162944 conn(0x7fd478071b10 msgr2=0x7fd478071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:55.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.852+0000 7fd4803ff700 1 -- 172.21.15.111:0/3586162944 shutdown_connections 2023-03-29T10:57:55.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.852+0000 7fd4803ff700 1 -- 172.21.15.111:0/3586162944 wait complete. 2023-03-29T10:57:55.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.853+0000 7fd4803ff700 1 Processor -- start 2023-03-29T10:57:55.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.853+0000 7fd4803ff700 1 -- start start 2023-03-29T10:57:55.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.854+0000 7fd4803ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4780cf970 0x7fd47815dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:55.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.854+0000 7fd4803ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4780cfd40 0x7fd4780c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:55.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.854+0000 7fd4803ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd478160290 0x7fd4781f1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:55.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.854+0000 7fd4803ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47815e630 con 0x7fd4780cfd40 2023-03-29T10:57:55.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.854+0000 7fd4803ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47815e910 con 0x7fd4780cf970 2023-03-29T10:57:55.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.854+0000 7fd4803ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47815ebf0 con 0x7fd478160290 2023-03-29T10:57:55.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.855+0000 7fd47d99a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4780cfd40 0x7fd4780c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:55.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.855+0000 7fd47d99a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4780cfd40 0x7fd4780c6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38256/0 (socket says 172.21.15.111:38256) 2023-03-29T10:57:55.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.855+0000 7fd47d99a700 1 -- 172.21.15.111:0/411382417 learned_addr learned my addr 172.21.15.111:0/411382417 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:55.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.855+0000 7fd47e19b700 1 --2- 172.21.15.111:0/411382417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4780cf970 0x7fd47815dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:55.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.855+0000 7fd47e99c700 1 --2- 172.21.15.111:0/411382417 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd478160290 0x7fd4781f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:55.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.855+0000 7fd47d99a700 1 -- 172.21.15.111:0/411382417 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd478160290 msgr2=0x7fd4781f1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:55.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.855+0000 7fd47d99a700 1 --2- 172.21.15.111:0/411382417 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd478160290 0x7fd4781f1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.855+0000 7fd47d99a700 1 -- 172.21.15.111:0/411382417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4780cf970 msgr2=0x7fd47815dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:55.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.855+0000 7fd47d99a700 1 --2- 172.21.15.111:0/411382417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4780cf970 0x7fd47815dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:55.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.855+0000 7fd47d99a700 1 -- 172.21.15.111:0/411382417 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4680094a0 con 0x7fd4780cfd40 2023-03-29T10:57:55.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.855+0000 7fd47e19b700 1 --2- 172.21.15.111:0/411382417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4780cf970 0x7fd47815dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:57:55.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.855+0000 7fd47e99c700 1 --2- 172.21.15.111:0/411382417 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd478160290 0x7fd4781f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:55.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.855+0000 7fd47d99a700 1 --2- 172.21.15.111:0/411382417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4780cfd40 0x7fd4780c6e70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd468009fd0 tx=0x7fd468004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:55.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.856+0000 7fd46effd700 1 -- 172.21.15.111:0/411382417 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4680036a0 con 0x7fd4780cfd40 2023-03-29T10:57:55.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.856+0000 7fd46effd700 1 -- 172.21.15.111:0/411382417 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46801e070 con 0x7fd4780cfd40 2023-03-29T10:57:55.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.856+0000 7fd46effd700 1 -- 172.21.15.111:0/411382417 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd468003a60 con 0x7fd4780cfd40 2023-03-29T10:57:55.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.856+0000 7fd4803ff700 1 -- 172.21.15.111:0/411382417 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4780cf650 con 0x7fd4780cfd40 2023-03-29T10:57:55.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.856+0000 7fd4803ff700 1 -- 172.21.15.111:0/411382417 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4780b68f0 con 0x7fd4780cfd40 2023-03-29T10:57:55.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.857+0000 7fd4803ff700 1 -- 172.21.15.111:0/411382417 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4780d06f0 con 0x7fd4780cfd40 2023-03-29T10:57:55.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.858+0000 7fd46effd700 1 -- 172.21.15.111:0/411382417 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fd468003bc0 con 0x7fd4780cfd40 2023-03-29T10:57:55.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.858+0000 7fd46effd700 1 --2- 172.21.15.111:0/411382417 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd45c069de0 0x7fd45c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:55.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.859+0000 7fd46effd700 1 -- 172.21.15.111:0/411382417 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd4680893d0 con 0x7fd4780cfd40 2023-03-29T10:57:55.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.859+0000 7fd47e19b700 1 --2- 172.21.15.111:0/411382417 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd45c069de0 0x7fd45c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:55.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.859+0000 7fd47e19b700 1 --2- 172.21.15.111:0/411382417 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd45c069de0 0x7fd45c06c290 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd4781f5610 tx=0x7fd474008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:55.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:55.863+0000 7fd46effd700 1 -- 172.21.15.111:0/411382417 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fd468054a20 con 0x7fd4780cfd40 2023-03-29T10:57:56.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.062+0000 7fd4803ff700 1 -- 172.21.15.111:0/411382417 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd47815ed30 con 0x7fd4780cfd40 2023-03-29T10:57:56.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.063+0000 7fd46effd700 1 -- 172.21.15.111:0/411382417 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19796 (secure 0 0 0) 0x7fd47815ed30 con 0x7fd4780cfd40 2023-03-29T10:57:56.064 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:56.064 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":28,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:57:32.331880+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:57:56.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.065+0000 7fd4803ff700 1 -- 172.21.15.111:0/411382417 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd45c069de0 msgr2=0x7fd45c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.065+0000 7fd4803ff700 1 --2- 172.21.15.111:0/411382417 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd45c069de0 0x7fd45c06c290 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd4781f5610 tx=0x7fd474008040 comp rx=0 tx=0).stop 2023-03-29T10:57:56.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.065+0000 7fd4803ff700 1 -- 172.21.15.111:0/411382417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4780cfd40 msgr2=0x7fd4780c6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.065+0000 7fd4803ff700 1 --2- 172.21.15.111:0/411382417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4780cfd40 0x7fd4780c6e70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd468009fd0 tx=0x7fd468004750 comp rx=0 tx=0).stop 2023-03-29T10:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.066+0000 7fd4803ff700 1 -- 172.21.15.111:0/411382417 shutdown_connections 2023-03-29T10:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.066+0000 7fd4803ff700 1 --2- 172.21.15.111:0/411382417 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd45c069de0 0x7fd45c06c290 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.066+0000 7fd4803ff700 1 --2- 172.21.15.111:0/411382417 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd478160290 0x7fd4781f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.066+0000 7fd4803ff700 1 --2- 172.21.15.111:0/411382417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4780cf970 0x7fd47815dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.066+0000 7fd4803ff700 1 --2- 172.21.15.111:0/411382417 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4780cfd40 0x7fd4780c6e70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.066+0000 7fd4803ff700 1 -- 172.21.15.111:0/411382417 >> 172.21.15.111:0/411382417 conn(0x7fd478071b10 msgr2=0x7fd47815c410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.066+0000 7fd4803ff700 1 -- 172.21.15.111:0/411382417 shutdown_connections 2023-03-29T10:57:56.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.066+0000 7fd4803ff700 1 -- 172.21.15.111:0/411382417 wait complete. 2023-03-29T10:57:56.083 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-03-29T10:57:56.083 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:57:56.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.266+0000 7f36269ff700 1 Processor -- start 2023-03-29T10:57:56.266 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.266+0000 7f36269ff700 1 -- start start 2023-03-29T10:57:56.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.267+0000 7f36269ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f36200cf970 0x7f36200cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.267+0000 7f36269ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f36200b70f0 0x7f36200b7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.267+0000 7f36269ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36200cdc10 con 0x7f36200cfd40 2023-03-29T10:57:56.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.267+0000 7f36269ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36200d08c0 con 0x7f36200b70f0 2023-03-29T10:57:56.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.267+0000 7f36269ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36200b6c60 con 0x7f36200cf970 2023-03-29T10:57:56.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.267+0000 7f3617fff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f36200cfd40 0x7f36200d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41648/0 (socket says 172.21.15.111:41648) 2023-03-29T10:57:56.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.267+0000 7f3617fff700 1 -- 172.21.15.111:0/2290230179 learned_addr learned my addr 172.21.15.111:0/2290230179 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:56.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.267+0000 7f361ffff700 1 --2- 172.21.15.111:0/2290230179 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f36200cf970 0x7f36200cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.267+0000 7f3624f9c700 1 --2- 172.21.15.111:0/2290230179 >> v2:172.21.15.119:3300/0 conn(0x7f36200b70f0 0x7f36200b7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.268+0000 7f361f7fe700 1 -- 172.21.15.111:0/2290230179 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133910789 0 0) 0x7f36200cdc10 con 0x7f36200cfd40 2023-03-29T10:57:56.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.268+0000 7f361f7fe700 1 -- 172.21.15.111:0/2290230179 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f360c0035b0 con 0x7f36200cfd40 2023-03-29T10:57:56.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.268+0000 7f3624f9c700 1 -- 172.21.15.111:0/2290230179 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f36200cf970 msgr2=0x7f36200cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:56.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.268+0000 7f3624f9c700 1 --2- 172.21.15.111:0/2290230179 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f36200cf970 0x7f36200cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.268+0000 7f3624f9c700 1 -- 172.21.15.111:0/2290230179 >> v1:172.21.15.111:6789/0 conn(0x7f36200cfd40 legacy=0x7f36200d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.268+0000 7f3624f9c700 1 -- 172.21.15.111:0/2290230179 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36200cd650 con 0x7f36200b70f0 2023-03-29T10:57:56.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.268+0000 7f361ffff700 1 --2- 172.21.15.111:0/2290230179 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f36200cf970 0x7f36200cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:56.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.268+0000 7f3624f9c700 1 --2- 172.21.15.111:0/2290230179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36200b70f0 0x7f36200b7530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3618009d90 tx=0x7f361800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9efe9bcbbf54db9a server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.268+0000 7f361f7fe700 1 -- 172.21.15.111:0/2290230179 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361800e070 con 0x7f36200b70f0 2023-03-29T10:57:56.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.269+0000 7f361f7fe700 1 -- 172.21.15.111:0/2290230179 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361800bb20 con 0x7f36200b70f0 2023-03-29T10:57:56.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.269+0000 7f361f7fe700 1 -- 172.21.15.111:0/2290230179 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618012430 con 0x7f36200b70f0 2023-03-29T10:57:56.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.269+0000 7f36269ff700 1 -- 172.21.15.111:0/2290230179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36200b70f0 msgr2=0x7f36200b7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.269+0000 7f36269ff700 1 --2- 172.21.15.111:0/2290230179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36200b70f0 0x7f36200b7530 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3618009d90 tx=0x7f361800b330 comp rx=0 tx=0).stop 2023-03-29T10:57:56.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.269+0000 7f36269ff700 1 -- 172.21.15.111:0/2290230179 shutdown_connections 2023-03-29T10:57:56.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.269+0000 7f36269ff700 1 --2- 172.21.15.111:0/2290230179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36200b70f0 0x7f36200b7530 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.269+0000 7f36269ff700 1 --2- 172.21.15.111:0/2290230179 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f36200cf970 0x7f36200cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.269+0000 7f36269ff700 1 -- 172.21.15.111:0/2290230179 >> 172.21.15.111:0/2290230179 conn(0x7f3620071b10 msgr2=0x7f3620071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:56.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.269+0000 7f36269ff700 1 -- 172.21.15.111:0/2290230179 shutdown_connections 2023-03-29T10:57:56.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.269+0000 7f36269ff700 1 -- 172.21.15.111:0/2290230179 wait complete. 2023-03-29T10:57:56.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.271+0000 7f36269ff700 1 Processor -- start 2023-03-29T10:57:56.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.271+0000 7f36269ff700 1 -- start start 2023-03-29T10:57:56.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.272+0000 7f36269ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36200b70f0 0x7f362014d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.272+0000 7f36269ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f36200cf970 0x7f36200c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.272+0000 7f36269ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36200cfd40 0x7f36200d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.272+0000 7f36269ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362014d8c0 con 0x7f36200cfd40 2023-03-29T10:57:56.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.272+0000 7f36269ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36200ca1d0 con 0x7f36200b70f0 2023-03-29T10:57:56.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.272+0000 7f36269ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36200ca4b0 con 0x7f36200cf970 2023-03-29T10:57:56.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.272+0000 7f3624f9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36200cfd40 0x7f36200d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.272+0000 7f3624f9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36200cfd40 0x7f36200d20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38268/0 (socket says 172.21.15.111:38268) 2023-03-29T10:57:56.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.272+0000 7f3624f9c700 1 -- 172.21.15.111:0/2179365981 learned_addr learned my addr 172.21.15.111:0/2179365981 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:56.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.272+0000 7f3617fff700 1 --2- 172.21.15.111:0/2179365981 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f36200cf970 0x7f36200c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.272+0000 7f361ffff700 1 --2- 172.21.15.111:0/2179365981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36200b70f0 0x7f362014d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.273+0000 7f3624f9c700 1 -- 172.21.15.111:0/2179365981 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f36200cf970 msgr2=0x7f36200c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.273+0000 7f3624f9c700 1 --2- 172.21.15.111:0/2179365981 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f36200cf970 0x7f36200c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.273+0000 7f3624f9c700 1 -- 172.21.15.111:0/2179365981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36200b70f0 msgr2=0x7f362014d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.273+0000 7f3624f9c700 1 --2- 172.21.15.111:0/2179365981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36200b70f0 0x7f362014d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.273+0000 7f3624f9c700 1 -- 172.21.15.111:0/2179365981 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36100094a0 con 0x7f36200cfd40 2023-03-29T10:57:56.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.273+0000 7f3624f9c700 1 --2- 172.21.15.111:0/2179365981 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36200cfd40 0x7f36200d20c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3618014040 tx=0x7f3618004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.273+0000 7f361d7fa700 1 -- 172.21.15.111:0/2179365981 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36180129f0 con 0x7f36200cfd40 2023-03-29T10:57:56.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.273+0000 7f361d7fa700 1 -- 172.21.15.111:0/2179365981 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3618020070 con 0x7f36200cfd40 2023-03-29T10:57:56.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.273+0000 7f36269ff700 1 -- 172.21.15.111:0/2179365981 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3618009a70 con 0x7f36200cfd40 2023-03-29T10:57:56.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.273+0000 7f361d7fa700 1 -- 172.21.15.111:0/2179365981 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618012db0 con 0x7f36200cfd40 2023-03-29T10:57:56.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.273+0000 7f36269ff700 1 -- 172.21.15.111:0/2179365981 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36200b68f0 con 0x7f36200cfd40 2023-03-29T10:57:56.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.274+0000 7f36269ff700 1 -- 172.21.15.111:0/2179365981 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36200cd930 con 0x7f36200cfd40 2023-03-29T10:57:56.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.275+0000 7f361d7fa700 1 -- 172.21.15.111:0/2179365981 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f3618004040 con 0x7f36200cfd40 2023-03-29T10:57:56.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.276+0000 7f361d7fa700 1 --2- 172.21.15.111:0/2179365981 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3600069de0 0x7f360006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.276+0000 7f361d7fa700 1 -- 172.21.15.111:0/2179365981 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3618022070 con 0x7f36200cfd40 2023-03-29T10:57:56.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.276+0000 7f361ffff700 1 --2- 172.21.15.111:0/2179365981 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3600069de0 0x7f360006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.277+0000 7f361ffff700 1 --2- 172.21.15.111:0/2179365981 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3600069de0 0x7f360006c290 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f362015b6f0 tx=0x7f3610009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.280+0000 7f361d7fa700 1 -- 172.21.15.111:0/2179365981 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f3618055550 con 0x7f36200cfd40 2023-03-29T10:57:56.481 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.482+0000 7f36269ff700 1 -- 172.21.15.111:0/2179365981 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36200cf650 con 0x7f36200cfd40 2023-03-29T10:57:56.482 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.483+0000 7f361d7fa700 1 -- 172.21.15.111:0/2179365981 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19796 (secure 0 0 0) 0x7f361802c020 con 0x7f36200cfd40 2023-03-29T10:57:56.483 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:57:56.483 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":28,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:57:32.331880+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:57:56.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.486+0000 7f36269ff700 1 -- 172.21.15.111:0/2179365981 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3600069de0 msgr2=0x7f360006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.485 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.486+0000 7f36269ff700 1 --2- 172.21.15.111:0/2179365981 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3600069de0 0x7f360006c290 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f362015b6f0 tx=0x7f3610009210 comp rx=0 tx=0).stop 2023-03-29T10:57:56.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.486+0000 7f36269ff700 1 -- 172.21.15.111:0/2179365981 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36200cfd40 msgr2=0x7f36200d20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.486+0000 7f36269ff700 1 --2- 172.21.15.111:0/2179365981 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36200cfd40 0x7f36200d20c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3618014040 tx=0x7f3618004ad0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.486+0000 7f36269ff700 1 -- 172.21.15.111:0/2179365981 shutdown_connections 2023-03-29T10:57:56.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.486+0000 7f36269ff700 1 --2- 172.21.15.111:0/2179365981 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3600069de0 0x7f360006c290 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.486+0000 7f36269ff700 1 --2- 172.21.15.111:0/2179365981 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f36200cfd40 0x7f36200d20c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.486+0000 7f36269ff700 1 --2- 172.21.15.111:0/2179365981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36200b70f0 0x7f362014d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.486+0000 7f36269ff700 1 --2- 172.21.15.111:0/2179365981 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f36200cf970 0x7f36200c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.486+0000 7f36269ff700 1 -- 172.21.15.111:0/2179365981 >> 172.21.15.111:0/2179365981 conn(0x7f3620071b10 msgr2=0x7f3620071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:56.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.486+0000 7f36269ff700 1 -- 172.21.15.111:0/2179365981 shutdown_connections 2023-03-29T10:57:56.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.486+0000 7f36269ff700 1 -- 172.21.15.111:0/2179365981 wait complete. 2023-03-29T10:57:56.503 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-03-29T10:57:56.503 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-03-29T10:57:56.503 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-03-29T10:57:56.504 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-03-29T10:57:56.504 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-03-29T10:57:56.504 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-03-29T10:57:56.505 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-03-29T10:57:56.505 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-03-29T10:57:56.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.834+0000 7f192b5ff700 1 Processor -- start 2023-03-29T10:57:56.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.835+0000 7fb2fcbff700 1 Processor -- start 2023-03-29T10:57:56.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.835+0000 7f192b5ff700 1 -- start start 2023-03-29T10:57:56.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.835+0000 7fb2fcbff700 1 -- start start 2023-03-29T10:57:56.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.837+0000 7f192b5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19240cfbf0 0x7f19241608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.837+0000 7fb2fcbff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2f80ad690 0x7fb2f80adb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.837+0000 7fb2fcbff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f80cd1f0 con 0x7fb2f8160d20 2023-03-29T10:57:56.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.837+0000 7fb2fcbff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f80cc670 con 0x7fb2f80c19c0 2023-03-29T10:57:56.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.837+0000 7fb2fcbff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f81610f0 con 0x7fb2f80ad690 2023-03-29T10:57:56.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.837+0000 7f192939b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f19240b87a0 0x7f19241601e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41650/0 (socket says 172.21.15.111:41650) 2023-03-29T10:57:56.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.837+0000 7f192939b700 1 -- 172.21.15.111:0/1290958502 learned_addr learned my addr 172.21.15.111:0/1290958502 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:56.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.838+0000 7fb2f659c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb2f8160d20 0x7fb2f806cb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41662/0 (socket says 172.21.15.111:41662) 2023-03-29T10:57:56.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.838+0000 7fb2f659c700 1 -- 172.21.15.111:0/285835070 learned_addr learned my addr 172.21.15.111:0/285835070 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:56.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.838+0000 7f1928b9a700 1 --2- 172.21.15.111:0/1290958502 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19240cfbf0 0x7f19241608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.838+0000 7f192b5ff700 1 -- 172.21.15.111:0/1290958502 --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19240d0020 con 0x7f19240b87a0 2023-03-29T10:57:56.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.838+0000 7f192b5ff700 1 -- 172.21.15.111:0/1290958502 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19240cdc60 con 0x7f19240d1100 2023-03-29T10:57:56.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.838+0000 7f192b5ff700 1 -- 172.21.15.111:0/1290958502 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19240d02d0 con 0x7f19240cfbf0 2023-03-29T10:57:56.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.839+0000 7f191bfff700 1 -- 172.21.15.111:0/1290958502 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2344673946 0 0) 0x7f19240cdc60 con 0x7f19240d1100 2023-03-29T10:57:56.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.839+0000 7f191bfff700 1 -- 172.21.15.111:0/1290958502 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19100035b0 con 0x7f19240d1100 2023-03-29T10:57:56.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.839+0000 7f191bfff700 1 -- 172.21.15.111:0/1290958502 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926216251 0 0) 0x7f19240d0020 con 0x7f19240b87a0 2023-03-29T10:57:56.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.839+0000 7f191bfff700 1 -- 172.21.15.111:0/1290958502 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19240cdc60 con 0x7f19240b87a0 2023-03-29T10:57:56.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.839+0000 7fb2f5d9b700 1 --2- 172.21.15.111:0/285835070 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2f80ad690 0x7fb2f80adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.840+0000 7f191bfff700 1 -- 172.21.15.111:0/1290958502 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1929702 0 0) 0x7f19240cdc60 con 0x7f19240b87a0 2023-03-29T10:57:56.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.840+0000 7f191bfff700 1 -- 172.21.15.111:0/1290958502 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19240cfbf0 msgr2=0x7f19241608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:56.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.840+0000 7f191bfff700 1 --2- 172.21.15.111:0/1290958502 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19240cfbf0 0x7f19241608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.840+0000 7f191bfff700 1 -- 172.21.15.111:0/1290958502 >> v1:172.21.15.119:6789/0 conn(0x7f19240d1100 legacy=0x7f19240d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.840+0000 7f191bfff700 1 -- 172.21.15.111:0/1290958502 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19240cd6a0 con 0x7f19240b87a0 2023-03-29T10:57:56.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.840+0000 7f191bfff700 1 -- 172.21.15.111:0/1290958502 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f19200015a0 con 0x7f19240b87a0 2023-03-29T10:57:56.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.840+0000 7f191bfff700 1 -- 172.21.15.111:0/1290958502 >> v1:172.21.15.111:6789/0 conn(0x7f19240b87a0 legacy=0x7f19241601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.840+0000 7f191bfff700 1 --2- 172.21.15.111:0/1290958502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1910003bf0 0x7f1910003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.840+0000 7f191bfff700 1 --2- 172.21.15.111:0/1290958502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1910004850 0x7f1910004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.840+0000 7f191bfff700 1 --2- 172.21.15.111:0/1290958502 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1910005320 0x7f191000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.840+0000 7f191bfff700 1 -- 172.21.15.111:0/1290958502 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c0015b0 con 0x7f1910003bf0 2023-03-29T10:57:56.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.840+0000 7f191bfff700 1 -- 172.21.15.111:0/1290958502 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19240d02d0 con 0x7f1910004850 2023-03-29T10:57:56.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.840+0000 7f191bfff700 1 -- 172.21.15.111:0/1290958502 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191000c1e0 con 0x7f1910005320 2023-03-29T10:57:56.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.840+0000 7fb2f5d9b700 1 -- 172.21.15.111:0/285835070 >> v1:172.21.15.119:6789/0 conn(0x7fb2f80c19c0 legacy=0x7fb2f80c1de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-03-29T10:57:56.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.840+0000 7fb2f5d9b700 1 -- 172.21.15.111:0/285835070 >> v1:172.21.15.111:6789/0 conn(0x7fb2f8160d20 legacy=0x7fb2f806cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.840+0000 7fb2f5d9b700 1 -- 172.21.15.111:0/285835070 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f80cc950 con 0x7fb2f80ad690 2023-03-29T10:57:56.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.841+0000 7fb2f5d9b700 1 --2- 172.21.15.111:0/285835070 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2f80ad690 0x7fb2f80adb10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009850 tx=0x7fb2e8009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43f9fe0993439792 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.840+0000 7f1928b9a700 1 --2- 172.21.15.111:0/1290958502 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19240cfbf0 0x7f19241608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:56.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.841+0000 7f1928b9a700 1 --2- 172.21.15.111:0/1290958502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1910004850 0x7f1910004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.841+0000 7f1929b9c700 1 --2- 172.21.15.111:0/1290958502 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1910005320 0x7f191000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.842+0000 7f192939b700 1 --2- 172.21.15.111:0/1290958502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1910003bf0 0x7f1910003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.842+0000 7f1928b9a700 1 -- 172.21.15.111:0/1290958502 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1910005320 msgr2=0x7f191000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:56.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.842+0000 7f1928b9a700 1 --2- 172.21.15.111:0/1290958502 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1910005320 0x7f191000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.842+0000 7f1928b9a700 1 -- 172.21.15.111:0/1290958502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1910003bf0 msgr2=0x7f1910003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:56.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.842+0000 7f1928b9a700 1 --2- 172.21.15.111:0/1290958502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1910003bf0 0x7f1910003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.842+0000 7f1928b9a700 1 -- 172.21.15.111:0/1290958502 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19240cd6a0 con 0x7f1910004850 2023-03-29T10:57:56.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.842+0000 7f192939b700 1 --2- 172.21.15.111:0/1290958502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1910003bf0 0x7f1910003fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:56.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.842+0000 7fb2f559a700 1 -- 172.21.15.111:0/285835070 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8005e30 con 0x7fb2f80ad690 2023-03-29T10:57:56.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.842+0000 7fb2f559a700 1 -- 172.21.15.111:0/285835070 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e8004030 con 0x7fb2f80ad690 2023-03-29T10:57:56.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.842+0000 7fb2f559a700 1 -- 172.21.15.111:0/285835070 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e80043c0 con 0x7fb2f80ad690 2023-03-29T10:57:56.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.842+0000 7f1928b9a700 1 --2- 172.21.15.111:0/1290958502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1910004850 0x7f1910004cc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f19240ceb90 tx=0x7f1910005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e03f8367529d68ee server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.843+0000 7f191bfff700 1 -- 172.21.15.111:0/1290958502 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191400a3f0 con 0x7f1910004850 2023-03-29T10:57:56.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.843+0000 7f191bfff700 1 -- 172.21.15.111:0/1290958502 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19140115f0 con 0x7f1910004850 2023-03-29T10:57:56.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.843+0000 7f191bfff700 1 -- 172.21.15.111:0/1290958502 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1914011980 con 0x7f1910004850 2023-03-29T10:57:56.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.843+0000 7fb2fcbff700 1 -- 172.21.15.111:0/285835070 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2f80ad690 msgr2=0x7fb2f80adb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.843+0000 7fb2fcbff700 1 --2- 172.21.15.111:0/285835070 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2f80ad690 0x7fb2f80adb10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009850 tx=0x7fb2e8009b60 comp rx=0 tx=0).stop 2023-03-29T10:57:56.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.844+0000 7f192b5ff700 1 -- 172.21.15.111:0/1290958502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1910004850 msgr2=0x7f1910004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.844+0000 7f192b5ff700 1 --2- 172.21.15.111:0/1290958502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1910004850 0x7f1910004cc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f19240ceb90 tx=0x7f1910005880 comp rx=0 tx=0).stop 2023-03-29T10:57:56.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.844+0000 7f1929b9c700 1 -- 172.21.15.111:0/1290958502 reap_dead start 2023-03-29T10:57:56.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.844+0000 7f192b5ff700 1 -- 172.21.15.111:0/1290958502 shutdown_connections 2023-03-29T10:57:56.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.844+0000 7f192b5ff700 1 -- 172.21.15.111:0/1290958502 >> 172.21.15.111:0/1290958502 conn(0x7f1924071b10 msgr2=0x7f1924071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:56.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.844+0000 7f192b5ff700 1 -- 172.21.15.111:0/1290958502 shutdown_connections 2023-03-29T10:57:56.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.845+0000 7f192b5ff700 1 -- 172.21.15.111:0/1290958502 wait complete. 2023-03-29T10:57:56.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.845+0000 7fb2fcbff700 1 -- 172.21.15.111:0/285835070 shutdown_connections 2023-03-29T10:57:56.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.845+0000 7fb2fcbff700 1 --2- 172.21.15.111:0/285835070 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2f80ad690 0x7fb2f80adb10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.845+0000 7fb2fcbff700 1 -- 172.21.15.111:0/285835070 >> 172.21.15.111:0/285835070 conn(0x7fb2f80cf720 msgr2=0x7fb2f80cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:56.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.845+0000 7fb2fcbff700 1 -- 172.21.15.111:0/285835070 shutdown_connections 2023-03-29T10:57:56.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.846+0000 7fb2fcbff700 1 -- 172.21.15.111:0/285835070 wait complete. 2023-03-29T10:57:56.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.848+0000 7fb2fcbff700 1 Processor -- start 2023-03-29T10:57:56.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.847+0000 7f192b5ff700 1 Processor -- start 2023-03-29T10:57:56.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.847+0000 7f09d39ff700 1 Processor -- start 2023-03-29T10:57:56.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.847+0000 7f192b5ff700 1 -- start start 2023-03-29T10:57:56.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.849+0000 7f192b5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1910004850 0x7f1924151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.849+0000 7f192b5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19240c69e0 0x7f19240c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.849+0000 7f192b5ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19240ca4c0 0x7f19241594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.847+0000 7f09d39ff700 1 -- start start 2023-03-29T10:57:56.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.849+0000 7f09d39ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09cc15a140 0x7f09cc0cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.849+0000 7f09d39ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f09cc160ba0 0x7f09cc0cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.849+0000 7f09d39ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09cc15d660 con 0x7f09cc0cfe20 2023-03-29T10:57:56.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.849+0000 7f09d39ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09cc0d02a0 con 0x7f09cc160ba0 2023-03-29T10:57:56.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.849+0000 7f192b5ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19240ccce0 con 0x7f19240ca4c0 2023-03-29T10:57:56.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.849+0000 7f192b5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1924151970 con 0x7f1910004850 2023-03-29T10:57:56.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.849+0000 7f192b5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192415ea40 con 0x7f19240c69e0 2023-03-29T10:57:56.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.849+0000 7f09d39ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc15cd70 con 0x7f09cc15a140 2023-03-29T10:57:56.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7f1928b9a700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19240c69e0 0x7f19240c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7f09d179b700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09cc15a140 0x7f09cc0cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7f09d179b700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09cc15a140 0x7f09cc0cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:50728/0 (socket says 172.21.15.111:50728) 2023-03-29T10:57:56.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7f09d179b700 1 -- 172.21.15.111:0/1374624667 learned_addr learned my addr 172.21.15.111:0/1374624667 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:56.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7f1928b9a700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19240c69e0 0x7f19240c9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:50736/0 (socket says 172.21.15.111:50736) 2023-03-29T10:57:56.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7f1928b9a700 1 -- 172.21.15.111:0/2337583476 learned_addr learned my addr 172.21.15.111:0/2337583476 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:56.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7f1928b9a700 1 -- 172.21.15.111:0/2337583476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1910004850 msgr2=0x7f1924151430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-03-29T10:57:56.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7f1929b9c700 1 --2- 172.21.15.111:0/2337583476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19240ca4c0 0x7f19241594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7f192939b700 1 --2- 172.21.15.111:0/2337583476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1910004850 0x7f1924151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7f1928b9a700 1 --2- 172.21.15.111:0/2337583476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1910004850 0x7f1924151430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7f1928b9a700 1 -- 172.21.15.111:0/2337583476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19240ca4c0 msgr2=0x7f19241594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7f1928b9a700 1 --2- 172.21.15.111:0/2337583476 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19240ca4c0 0x7f19241594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7f1928b9a700 1 -- 172.21.15.111:0/2337583476 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19241608f0 con 0x7f19240c69e0 2023-03-29T10:57:56.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.849+0000 7fb2fcbff700 1 -- start start 2023-03-29T10:57:56.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7fb2fcbff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2f80ad690 0x7fb2f80ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7fb2fcbff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2f80c19c0 0x7fb2f815d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7fb2fcbff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2f8160d20 0x7fb2f80be8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7fb2fcbff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80bd650 con 0x7fb2f80c19c0 2023-03-29T10:57:56.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7fb2fcbff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80ca950 con 0x7fb2f80ad690 2023-03-29T10:57:56.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7fb2fcbff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80bd830 con 0x7fb2f8160d20 2023-03-29T10:57:56.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.851+0000 7fb2f659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2f80ad690 0x7fb2f80ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.851+0000 7fb2f659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2f80ad690 0x7fb2f80ca410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:40870/0 (socket says 172.21.15.111:40870) 2023-03-29T10:57:56.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.851+0000 7fb2f659c700 1 -- 172.21.15.111:0/3545021412 learned_addr learned my addr 172.21.15.111:0/3545021412 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:56.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.851+0000 7fb2f5d9b700 1 --2- 172.21.15.111:0/3545021412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2f80c19c0 0x7fb2f815d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.851+0000 7fb2f6d9d700 1 --2- 172.21.15.111:0/3545021412 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2f8160d20 0x7fb2f80be8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.852+0000 7fb2f659c700 1 -- 172.21.15.111:0/3545021412 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2f8160d20 msgr2=0x7fb2f80be8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.852+0000 7fb2f659c700 1 --2- 172.21.15.111:0/3545021412 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2f8160d20 0x7fb2f80be8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.852+0000 7fb2f659c700 1 -- 172.21.15.111:0/3545021412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2f80c19c0 msgr2=0x7fb2f815d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.852+0000 7fb2f659c700 1 --2- 172.21.15.111:0/3545021412 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2f80c19c0 0x7fb2f815d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.852+0000 7fb2f659c700 1 -- 172.21.15.111:0/3545021412 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e8009430 con 0x7fb2f80ad690 2023-03-29T10:57:56.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.852+0000 7f1928b9a700 1 --2- 172.21.15.111:0/2337583476 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19240c69e0 0x7f19240c9f80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f191400aec0 tx=0x7f19140047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7f09d0f9a700 1 --2- 172.21.15.111:0/1374624667 >> v2:172.21.15.111:3300/0 conn(0x7f09cc160ba0 0x7f09cc0cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7f09d179b700 1 -- 172.21.15.111:0/1374624667 >> v2:172.21.15.111:3300/0 conn(0x7f09cc160ba0 msgr2=0x7f09cc0cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:56.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7f09d179b700 1 --2- 172.21.15.111:0/1374624667 >> v2:172.21.15.111:3300/0 conn(0x7f09cc160ba0 0x7f09cc0cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7f09d179b700 1 -- 172.21.15.111:0/1374624667 >> v1:172.21.15.119:6789/0 conn(0x7f09cc0cfe20 legacy=0x7f09cc0b1b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-03-29T10:57:56.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.850+0000 7f09d179b700 1 -- 172.21.15.111:0/1374624667 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc15d010 con 0x7f09cc15a140 2023-03-29T10:57:56.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.851+0000 7f09d0f9a700 1 --2- 172.21.15.111:0/1374624667 >> v2:172.21.15.111:3300/0 conn(0x7f09cc160ba0 0x7f09cc0cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:56.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.851+0000 7f09d179b700 1 --2- 172.21.15.111:0/1374624667 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09cc15a140 0x7f09cc0cd520 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f09c80098f0 tx=0x7f09c8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12b7fc300f3bcaba server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.852+0000 7f09c3fff700 1 -- 172.21.15.111:0/1374624667 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c8004230 con 0x7f09cc15a140 2023-03-29T10:57:56.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.852+0000 7f09c3fff700 1 -- 172.21.15.111:0/1374624667 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c8004390 con 0x7f09cc15a140 2023-03-29T10:57:56.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.852+0000 7f09c3fff700 1 -- 172.21.15.111:0/1374624667 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c80036a0 con 0x7f09cc15a140 2023-03-29T10:57:56.855 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.852+0000 7f1919ffb700 1 -- 172.21.15.111:0/2337583476 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1914020070 con 0x7f19240c69e0 2023-03-29T10:57:56.855 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.852+0000 7f1919ffb700 1 -- 172.21.15.111:0/2337583476 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191401b560 con 0x7f19240c69e0 2023-03-29T10:57:56.855 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.852+0000 7f1919ffb700 1 -- 172.21.15.111:0/2337583476 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191401b8d0 con 0x7f19240c69e0 2023-03-29T10:57:56.855 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.852+0000 7f192b5ff700 1 -- 172.21.15.111:0/2337583476 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19240cd980 con 0x7f19240c69e0 2023-03-29T10:57:56.855 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.852+0000 7f192b5ff700 1 -- 172.21.15.111:0/2337583476 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1924160a80 con 0x7f19240c69e0 2023-03-29T10:57:56.855 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.853+0000 7fb2f659c700 1 --2- 172.21.15.111:0/3545021412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2f80ad690 0x7fb2f80ca410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0002080 tx=0x7fb2f000a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.855 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.853+0000 7f09d39ff700 1 -- 172.21.15.111:0/1374624667 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09cc15a140 msgr2=0x7f09cc0cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.853+0000 7f09d39ff700 1 --2- 172.21.15.111:0/1374624667 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09cc15a140 0x7f09cc0cd520 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f09c80098f0 tx=0x7f09c8009c00 comp rx=0 tx=0).stop 2023-03-29T10:57:56.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.853+0000 7f6ebebff700 1 Processor -- start 2023-03-29T10:57:56.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.853+0000 7f97dbfff700 1 Processor -- start 2023-03-29T10:57:56.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.853+0000 7f09d39ff700 1 -- 172.21.15.111:0/1374624667 shutdown_connections 2023-03-29T10:57:56.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.853+0000 7f09d39ff700 1 --2- 172.21.15.111:0/1374624667 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09cc15a140 0x7f09cc0cd520 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.853+0000 7f09d39ff700 1 --2- 172.21.15.111:0/1374624667 >> v2:172.21.15.111:3300/0 conn(0x7f09cc160ba0 0x7f09cc0cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.853+0000 7f09d39ff700 1 -- 172.21.15.111:0/1374624667 >> 172.21.15.111:0/1374624667 conn(0x7f09cc071b10 msgr2=0x7f09cc071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:56.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.853+0000 7f6ebebff700 1 -- start start 2023-03-29T10:57:56.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.853+0000 7fb2e6ffd700 1 -- 172.21.15.111:0/3545021412 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f000ad40 con 0x7fb2f80ad690 2023-03-29T10:57:56.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.853+0000 7fb2e6ffd700 1 -- 172.21.15.111:0/3545021412 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f00070d0 con 0x7fb2f80ad690 2023-03-29T10:57:56.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.853+0000 7fb2e6ffd700 1 -- 172.21.15.111:0/3545021412 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f000aea0 con 0x7fb2f80ad690 2023-03-29T10:57:56.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.854+0000 7f09d39ff700 1 -- 172.21.15.111:0/1374624667 shutdown_connections 2023-03-29T10:57:56.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.854+0000 7f09d39ff700 1 -- 172.21.15.111:0/1374624667 wait complete. 2023-03-29T10:57:56.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.855+0000 7fd2499ff700 1 Processor -- start 2023-03-29T10:57:56.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.854+0000 7fb2fcbff700 1 -- 172.21.15.111:0/3545021412 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f80ccf10 con 0x7fb2f80ad690 2023-03-29T10:57:56.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.854+0000 7fb2fcbff700 1 -- 172.21.15.111:0/3545021412 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f80ccc30 con 0x7fb2f80ad690 2023-03-29T10:57:56.858 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.855+0000 7ff77f7ff700 1 Processor -- start 2023-03-29T10:57:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.855+0000 7fb2fcbff700 1 -- 172.21.15.111:0/3545021412 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb2f806c8a0 con 0x7fb2f80ad690 2023-03-29T10:57:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.855+0000 7f6ebebff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6eb80b87a0 0x7f6eb80cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.855+0000 7f6ebebff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6eb81601e0 0x7f6eb8160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.855+0000 7f6ebebff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb8160ff0 con 0x7f6eb8160b70 2023-03-29T10:57:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.856+0000 7f6ebebff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb80d02d0 con 0x7f6eb80b87a0 2023-03-29T10:57:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.856+0000 7f6ebebff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb80d0cb0 con 0x7f6eb81601e0 2023-03-29T10:57:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.856+0000 7f6ebc99b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6eb80b87a0 0x7f6eb80cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.856+0000 7f6ebc99b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6eb80b87a0 0x7f6eb80cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:40882/0 (socket says 172.21.15.111:40882) 2023-03-29T10:57:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.856+0000 7f6ebc99b700 1 -- 172.21.15.111:0/3506162192 learned_addr learned my addr 172.21.15.111:0/3506162192 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.856+0000 7f6eb7fff700 1 --2- 172.21.15.111:0/3506162192 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6eb81601e0 0x7f6eb8160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.855+0000 7fd2499ff700 1 -- start start 2023-03-29T10:57:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.854+0000 7f97dbfff700 1 -- start start 2023-03-29T10:57:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.856+0000 7f97dbfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f97d4160d20 0x7f97d406cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.856+0000 7f97dbfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f97d40ad690 0x7f97d40adb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.856+0000 7f97dbfff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d40cd1f0 con 0x7f97d40c19c0 2023-03-29T10:57:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.856+0000 7f97dbfff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d41610f0 con 0x7f97d40ad690 2023-03-29T10:57:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.856+0000 7f97dbfff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d40cc670 con 0x7f97d4160d20 2023-03-29T10:57:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.857+0000 7fd2499ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd2440cacf0 0x7fd2440add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.857+0000 7fd2499ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd24415cdb0 0x7fd2440c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.857+0000 7fd2499ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd24406a3e0 0x7fd24406a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.857+0000 7fd2499ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd244158d10 con 0x7fd24406a3e0 2023-03-29T10:57:56.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.857+0000 7fd2499ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24406ad80 con 0x7fd24415cdb0 2023-03-29T10:57:56.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.857+0000 7fd2499ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440cd5b0 con 0x7fd2440cacf0 2023-03-29T10:57:56.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.857+0000 7f97da59c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f97d40c19c0 0x7f97d40c1d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.111:35156/0 (socket says 172.21.15.111:35156) 2023-03-29T10:57:56.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.857+0000 7f97da59c700 1 -- 172.21.15.111:0/2785338358 learned_addr learned my addr 172.21.15.111:0/2785338358 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:56.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.857+0000 7f97d9d9b700 1 --2- 172.21.15.111:0/2785338358 >> v2:172.21.15.119:3300/0 conn(0x7f97d4160d20 0x7f97d406cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.857+0000 7f97d959a700 1 --2- 172.21.15.111:0/2785338358 >> v2:172.21.15.111:3300/0 conn(0x7f97d40ad690 0x7f97d40adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.857+0000 7f09d39ff700 1 Processor -- start 2023-03-29T10:57:56.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.857+0000 7f09d39ff700 1 -- start start 2023-03-29T10:57:56.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.858+0000 7fd2427fc700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd24415cdb0 0x7fd2440c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.858+0000 7fd2427fc700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd24415cdb0 0x7fd2440c90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:40894/0 (socket says 172.21.15.111:40894) 2023-03-29T10:57:56.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.858+0000 7fd2427fc700 1 -- 172.21.15.111:0/1122821852 learned_addr learned my addr 172.21.15.111:0/1122821852 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:56.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.858+0000 7f97d9d9b700 1 -- 172.21.15.111:0/2785338358 >> v2:172.21.15.111:3300/0 conn(0x7f97d40ad690 msgr2=0x7f97d40adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:56.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.858+0000 7f97d9d9b700 1 --2- 172.21.15.111:0/2785338358 >> v2:172.21.15.111:3300/0 conn(0x7f97d40ad690 0x7f97d40adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.858+0000 7f97d9d9b700 1 -- 172.21.15.111:0/2785338358 >> v1:172.21.15.119:6789/0 conn(0x7f97d40c19c0 legacy=0x7f97d40c1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.858+0000 7f97d9d9b700 1 -- 172.21.15.111:0/2785338358 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d40cc950 con 0x7f97d4160d20 2023-03-29T10:57:56.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.858+0000 7f192b5ff700 1 -- 172.21.15.111:0/2337583476 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1924074430 con 0x7f19240c69e0 2023-03-29T10:57:56.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7f1919ffb700 1 -- 172.21.15.111:0/2337583476 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f1914024070 con 0x7f19240c69e0 2023-03-29T10:57:56.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.857+0000 7f6eb7fff700 1 -- 172.21.15.111:0/3506162192 >> v2:172.21.15.119:3300/0 conn(0x7f6eb80b87a0 msgr2=0x7f6eb80cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:56.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.857+0000 7f6eb7fff700 1 --2- 172.21.15.111:0/3506162192 >> v2:172.21.15.119:3300/0 conn(0x7f6eb80b87a0 0x7f6eb80cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.857+0000 7f6eb7fff700 1 -- 172.21.15.111:0/3506162192 >> v1:172.21.15.111:6789/0 conn(0x7f6eb8160b70 legacy=0x7f6eb80d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:56.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.857+0000 7f6eb7fff700 1 -- 172.21.15.111:0/3506162192 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb80cdc60 con 0x7f6eb81601e0 2023-03-29T10:57:56.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.858+0000 7f6eb7fff700 1 --2- 172.21.15.111:0/3506162192 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6eb81601e0 0x7f6eb8160630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009840 tx=0x7f6eac009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be5c29982fefe009 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.858+0000 7f6eb77fe700 1 -- 172.21.15.111:0/3506162192 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac004230 con 0x7f6eb81601e0 2023-03-29T10:57:56.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.858+0000 7f6eb77fe700 1 -- 172.21.15.111:0/3506162192 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac004390 con 0x7f6eb81601e0 2023-03-29T10:57:56.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7f6eb77fe700 1 -- 172.21.15.111:0/3506162192 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac0036c0 con 0x7f6eb81601e0 2023-03-29T10:57:56.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.858+0000 7fd2437fe700 1 --2- 172.21.15.111:0/1122821852 >> v2:172.21.15.111:3300/0 conn(0x7fd24406a3e0 0x7fd24406a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7fd2427fc700 1 -- 172.21.15.111:0/1122821852 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd2440cacf0 msgr2=0x7fd2440add70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-03-29T10:57:56.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7fd2427fc700 1 --2- 172.21.15.111:0/1122821852 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd2440cacf0 0x7fd2440add70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7fd2427fc700 1 -- 172.21.15.111:0/1122821852 >> v2:172.21.15.111:3300/0 conn(0x7fd24406a3e0 msgr2=0x7fd24406a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:56.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7fd2427fc700 1 --2- 172.21.15.111:0/1122821852 >> v2:172.21.15.111:3300/0 conn(0x7fd24406a3e0 0x7fd24406a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7fd2427fc700 1 -- 172.21.15.111:0/1122821852 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2440ccff0 con 0x7fd24415cdb0 2023-03-29T10:57:56.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7f97d9d9b700 1 --2- 172.21.15.111:0/2785338358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97d4160d20 0x7f97d406cb90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f97d40705a0 tx=0x7f97c4009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b97e078b1c4d5c49 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.856+0000 7ff77f7ff700 1 -- start start 2023-03-29T10:57:56.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.858+0000 7ff77f7ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff7780cf010 0x7ff778160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.858+0000 7ff77f7ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7780cd520 con 0x7ff7780cff50 2023-03-29T10:57:56.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.858+0000 7ff77f7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7780d2640 con 0x7ff77815c590 2023-03-29T10:57:56.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.858+0000 7ff77f7ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7781610d0 con 0x7ff7780cf010 2023-03-29T10:57:56.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.858+0000 7ff77cd9a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff7780cf010 0x7ff778160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7ff77cd9a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff7780cf010 0x7ff778160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38322/0 (socket says 172.21.15.111:38322) 2023-03-29T10:57:56.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7ff77cd9a700 1 -- 172.21.15.111:0/674729788 learned_addr learned my addr 172.21.15.111:0/674729788 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:56.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7f97d8d99700 1 -- 172.21.15.111:0/2785338358 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4011830 con 0x7f97d4160d20 2023-03-29T10:57:56.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7f97d8d99700 1 -- 172.21.15.111:0/2785338358 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c4011990 con 0x7f97d4160d20 2023-03-29T10:57:56.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7f97d8d99700 1 -- 172.21.15.111:0/2785338358 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4011d20 con 0x7f97d4160d20 2023-03-29T10:57:56.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7fd2427fc700 1 --2- 172.21.15.111:0/1122821852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd24415cdb0 0x7fd2440c90c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd24406b800 tx=0x7fd23800d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d915e2f2737920c3 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7f6ebebff700 1 -- 172.21.15.111:0/3506162192 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6eb81601e0 msgr2=0x7f6eb8160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7f6ebebff700 1 --2- 172.21.15.111:0/3506162192 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6eb81601e0 0x7f6eb8160630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009840 tx=0x7f6eac009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:56.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7fd241ffb700 1 -- 172.21.15.111:0/1122821852 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238010070 con 0x7fd24415cdb0 2023-03-29T10:57:56.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7fd241ffb700 1 -- 172.21.15.111:0/1122821852 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23800da80 con 0x7fd24415cdb0 2023-03-29T10:57:56.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7fd241ffb700 1 -- 172.21.15.111:0/1122821852 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23800de10 con 0x7fd24415cdb0 2023-03-29T10:57:56.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7f09d39ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09cc0cfe20 0x7f09cc0c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7f09d39ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09cc15a140 0x7f09cc0b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7f09d39ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09cc160ba0 0x7f09cc1e6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7f6ebebff700 1 -- 172.21.15.111:0/3506162192 shutdown_connections 2023-03-29T10:57:56.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7f6ebebff700 1 --2- 172.21.15.111:0/3506162192 >> v2:172.21.15.119:3300/0 conn(0x7f6eb80b87a0 0x7f6eb80cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7f6ebebff700 1 --2- 172.21.15.111:0/3506162192 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6eb81601e0 0x7f6eb8160630 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7f6ebebff700 1 -- 172.21.15.111:0/3506162192 >> 172.21.15.111:0/3506162192 conn(0x7f6eb8071b10 msgr2=0x7f6eb8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:56.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7f09d39ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc0b4770 con 0x7f09cc0cfe20 2023-03-29T10:57:56.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7f09d39ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc0b3af0 con 0x7f09cc160ba0 2023-03-29T10:57:56.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.859+0000 7f09d39ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc0b3dd0 con 0x7f09cc15a140 2023-03-29T10:57:56.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7f97dbfff700 1 -- 172.21.15.111:0/2785338358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97d4160d20 msgr2=0x7f97d406cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7f97dbfff700 1 --2- 172.21.15.111:0/2785338358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97d4160d20 0x7f97d406cb90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f97d40705a0 tx=0x7f97c4009b20 comp rx=0 tx=0).stop 2023-03-29T10:57:56.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7ff77cd9a700 1 -- 172.21.15.111:0/674729788 >> v1:172.21.15.119:6789/0 conn(0x7ff77815c590 legacy=0x7ff7780cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:56.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7ff77cd9a700 1 -- 172.21.15.111:0/674729788 >> v1:172.21.15.111:6789/0 conn(0x7ff7780cff50 legacy=0x7ff7780d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7ff77cd9a700 1 -- 172.21.15.111:0/674729788 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7780b23c0 con 0x7ff7780cf010 2023-03-29T10:57:56.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7f09d1f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09cc160ba0 0x7f09cc1e6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7f09d1f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09cc160ba0 0x7f09cc1e6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:40906/0 (socket says 172.21.15.111:40906) 2023-03-29T10:57:56.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7f09d1f9c700 1 -- 172.21.15.111:0/1776614838 learned_addr learned my addr 172.21.15.111:0/1776614838 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:56.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7f09d0f9a700 1 --2- 172.21.15.111:0/1776614838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09cc15a140 0x7f09cc0b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7f1919ffb700 1 --2- 172.21.15.111:0/2337583476 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1908069eb0 0x7f190806c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7f1919ffb700 1 -- 172.21.15.111:0/2337583476 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f19140890d0 con 0x7f19240c69e0 2023-03-29T10:57:56.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7fd2499ff700 1 -- 172.21.15.111:0/1122821852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd24415cdb0 msgr2=0x7fd2440c90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.860+0000 7fd2499ff700 1 --2- 172.21.15.111:0/1122821852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd24415cdb0 0x7fd2440c90c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd24406b800 tx=0x7fd23800d330 comp rx=0 tx=0).stop 2023-03-29T10:57:56.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.861+0000 7f09d179b700 1 --2- 172.21.15.111:0/1776614838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09cc0cfe20 0x7f09cc0c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.861+0000 7f1919ffb700 1 --2- 172.21.15.111:0/2337583476 >> [v2:172.21.15.111:6808/110483,v1:172.21.15.111:6809/110483] conn(0x7f1908073b80 0x7f1908075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.861+0000 7f1919ffb700 1 -- 172.21.15.111:0/2337583476 --> [v2:172.21.15.111:6808/110483,v1:172.21.15.111:6809/110483] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1908076680 con 0x7f1908073b80 2023-03-29T10:57:56.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.861+0000 7f1919ffb700 1 -- 172.21.15.111:0/2337583476 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f191408b6f0 con 0x7f19240c69e0 2023-03-29T10:57:56.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.861+0000 7f97dbfff700 1 -- 172.21.15.111:0/2785338358 shutdown_connections 2023-03-29T10:57:56.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.861+0000 7f97dbfff700 1 --2- 172.21.15.111:0/2785338358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97d4160d20 0x7f97d406cb90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.861+0000 7f97dbfff700 1 --2- 172.21.15.111:0/2785338358 >> v2:172.21.15.111:3300/0 conn(0x7f97d40ad690 0x7f97d40adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.861+0000 7ff77cd9a700 1 --2- 172.21.15.111:0/674729788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7780cf010 0x7ff778160b90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff7740098f0 tx=0x7ff774009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3471c55b636270 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.861+0000 7f97dbfff700 1 -- 172.21.15.111:0/2785338358 >> 172.21.15.111:0/2785338358 conn(0x7f97d40cf720 msgr2=0x7f97d40cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:56.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.861+0000 7f09d0f9a700 1 -- 172.21.15.111:0/1776614838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09cc160ba0 msgr2=0x7f09cc1e6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.861+0000 7f09d0f9a700 1 --2- 172.21.15.111:0/1776614838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09cc160ba0 0x7f09cc1e6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.861+0000 7f09d0f9a700 1 -- 172.21.15.111:0/1776614838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09cc0cfe20 msgr2=0x7f09cc0c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.861+0000 7f09d0f9a700 1 --2- 172.21.15.111:0/1776614838 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09cc0cfe20 0x7f09cc0c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.861+0000 7f09d0f9a700 1 -- 172.21.15.111:0/1776614838 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c8009550 con 0x7f09cc15a140 2023-03-29T10:57:56.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.861+0000 7f1929b9c700 1 --2- 172.21.15.111:0/2337583476 >> [v2:172.21.15.111:6808/110483,v1:172.21.15.111:6809/110483] conn(0x7f1908073b80 0x7f1908075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.861+0000 7f6ebebff700 1 -- 172.21.15.111:0/3506162192 shutdown_connections 2023-03-29T10:57:56.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.862+0000 7fd2499ff700 1 -- 172.21.15.111:0/1122821852 shutdown_connections 2023-03-29T10:57:56.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.862+0000 7fd2499ff700 1 --2- 172.21.15.111:0/1122821852 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd2440cacf0 0x7fd2440add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.862+0000 7fd2499ff700 1 --2- 172.21.15.111:0/1122821852 >> v2:172.21.15.111:3300/0 conn(0x7fd24406a3e0 0x7fd24406a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.862+0000 7fd2499ff700 1 --2- 172.21.15.111:0/1122821852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd24415cdb0 0x7fd2440c90c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.862+0000 7fd2499ff700 1 -- 172.21.15.111:0/1122821852 >> 172.21.15.111:0/1122821852 conn(0x7fd2440cf6e0 msgr2=0x7fd2440cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:56.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.862+0000 7f09d0f9a700 1 --2- 172.21.15.111:0/1776614838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09cc15a140 0x7f09cc0b3180 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00c7c0 tx=0x7f09bc00cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.862+0000 7f1929b9c700 1 --2- 172.21.15.111:0/2337583476 >> [v2:172.21.15.111:6808/110483,v1:172.21.15.111:6809/110483] conn(0x7f1908073b80 0x7f1908075fb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.861+0000 7f97dbfff700 1 -- 172.21.15.111:0/2785338358 shutdown_connections 2023-03-29T10:57:56.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.861+0000 7f97dbfff700 1 -- 172.21.15.111:0/2785338358 wait complete. 2023-03-29T10:57:56.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.862+0000 7ff76ffff700 1 -- 172.21.15.111:0/674729788 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774004230 con 0x7ff7780cf010 2023-03-29T10:57:56.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.862+0000 7ff76ffff700 1 -- 172.21.15.111:0/674729788 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff774004390 con 0x7ff7780cf010 2023-03-29T10:57:56.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.862+0000 7ff76ffff700 1 -- 172.21.15.111:0/674729788 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7740036a0 con 0x7ff7780cf010 2023-03-29T10:57:56.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.861+0000 7f6ebebff700 1 -- 172.21.15.111:0/3506162192 wait complete. 2023-03-29T10:57:56.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.862+0000 7f192939b700 1 --2- 172.21.15.111:0/2337583476 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1908069eb0 0x7f190806c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.863+0000 7f09c1ffb700 1 -- 172.21.15.111:0/1776614838 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc011800 con 0x7f09cc15a140 2023-03-29T10:57:56.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.863+0000 7f09c1ffb700 1 -- 172.21.15.111:0/1776614838 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09bc011960 con 0x7f09cc15a140 2023-03-29T10:57:56.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.863+0000 7f09c1ffb700 1 -- 172.21.15.111:0/1776614838 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc011cf0 con 0x7f09cc15a140 2023-03-29T10:57:56.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.863+0000 7ff77f7ff700 1 -- 172.21.15.111:0/674729788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7780cf010 msgr2=0x7ff778160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.863+0000 7ff77f7ff700 1 --2- 172.21.15.111:0/674729788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7780cf010 0x7ff778160b90 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff7740098f0 tx=0x7ff774009c00 comp rx=0 tx=0).stop 2023-03-29T10:57:56.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.863+0000 7ff77f7ff700 1 -- 172.21.15.111:0/674729788 shutdown_connections 2023-03-29T10:57:56.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.863+0000 7ff77f7ff700 1 --2- 172.21.15.111:0/674729788 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7780cf010 0x7ff778160b90 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.863+0000 7ff77f7ff700 1 -- 172.21.15.111:0/674729788 >> 172.21.15.111:0/674729788 conn(0x7ff778071b10 msgr2=0x7ff778071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:56.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.863+0000 7f192939b700 1 --2- 172.21.15.111:0/2337583476 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1908069eb0 0x7f190806c360 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1920001eb0 tx=0x7f1920009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.864+0000 7fd2499ff700 1 -- 172.21.15.111:0/1122821852 shutdown_connections 2023-03-29T10:57:56.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.863+0000 7f09d39ff700 1 -- 172.21.15.111:0/1776614838 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09cc15daf0 con 0x7f09cc15a140 2023-03-29T10:57:56.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.863+0000 7f09d39ff700 1 -- 172.21.15.111:0/1776614838 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09cc15d2f0 con 0x7f09cc15a140 2023-03-29T10:57:56.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.864+0000 7f6cb47ff700 1 Processor -- start 2023-03-29T10:57:56.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.864+0000 7f6ebebff700 1 Processor -- start 2023-03-29T10:57:56.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.864+0000 7f97dbfff700 1 Processor -- start 2023-03-29T10:57:56.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.864+0000 7fd2499ff700 1 -- 172.21.15.111:0/1122821852 wait complete. 2023-03-29T10:57:56.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.864+0000 7f09d39ff700 1 -- 172.21.15.111:0/1776614838 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f09ac000fb0 con 0x7f09cc15a140 2023-03-29T10:57:56.876 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.865+0000 7ff77f7ff700 1 -- 172.21.15.111:0/674729788 shutdown_connections 2023-03-29T10:57:56.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.864+0000 7f97dbfff700 1 -- start start 2023-03-29T10:57:56.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.864+0000 7f6ebebff700 1 -- start start 2023-03-29T10:57:56.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.865+0000 7f6cb47ff700 1 -- start start 2023-03-29T10:57:56.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.863+0000 7fb2e6ffd700 1 -- 172.21.15.111:0/3545021412 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fb2f0007480 con 0x7fb2f80ad690 2023-03-29T10:57:56.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.865+0000 7fb2e6ffd700 1 --2- 172.21.15.111:0/3545021412 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb2d8069ee0 0x7fb2d806c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.865+0000 7fb2e6ffd700 1 -- 172.21.15.111:0/3545021412 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb2f0088aa0 con 0x7fb2f80ad690 2023-03-29T10:57:56.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.865+0000 7fb2e6ffd700 1 --2- 172.21.15.111:0/3545021412 >> [v2:172.21.15.111:6816/110484,v1:172.21.15.111:6817/110484] conn(0x7fb2d8073bb0 0x7fb2d8075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.866+0000 7fb2e6ffd700 1 -- 172.21.15.111:0/3545021412 --> [v2:172.21.15.111:6816/110484,v1:172.21.15.111:6817/110484] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2d80766b0 con 0x7fb2d8073bb0 2023-03-29T10:57:56.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.866+0000 7fb2e6ffd700 1 -- 172.21.15.111:0/3545021412 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2f0088e50 con 0x7fb2f80ad690 2023-03-29T10:57:56.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.866+0000 7f1919ffb700 1 -- 172.21.15.111:0/2337583476 <== osd.1 v2:172.21.15.111:6808/110483 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1908076680 con 0x7f1908073b80 2023-03-29T10:57:56.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.866+0000 7f6ebebff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eb80b87a0 0x7f6eb8154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.866+0000 7f6ebebff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6eb81601e0 0x7f6eb80b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.866+0000 7f6ebebff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6eb8160b70 0x7f6eb80b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.866+0000 7f6ebebff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb815cd40 con 0x7f6eb8160b70 2023-03-29T10:57:56.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.866+0000 7f6ebebff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb815cfe0 con 0x7f6eb80b87a0 2023-03-29T10:57:56.878 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.866+0000 7f6ebebff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb815d2c0 con 0x7f6eb81601e0 2023-03-29T10:57:56.879 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.866+0000 7fd2499ff700 1 Processor -- start 2023-03-29T10:57:56.879 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.866+0000 7f97dbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97d40ad690 0x7f97d41f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.879 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.866+0000 7f97dbfff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97d40c19c0 0x7f97d40bff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.879 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.866+0000 7f97dbfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97d4160d20 0x7f97d40c0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.879 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.866+0000 7f97dbfff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d41f64f0 con 0x7f97d4160d20 2023-03-29T10:57:56.879 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.866+0000 7f97dbfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d40c0af0 con 0x7f97d40ad690 2023-03-29T10:57:56.879 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.866+0000 7f97dbfff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d40c0dd0 con 0x7f97d40c19c0 2023-03-29T10:57:56.879 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.866+0000 7ff77f7ff700 1 -- 172.21.15.111:0/674729788 wait complete. 2023-03-29T10:57:56.880 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.866+0000 7fb2f6d9d700 1 --2- 172.21.15.111:0/3545021412 >> [v2:172.21.15.111:6816/110484,v1:172.21.15.111:6817/110484] conn(0x7fb2d8073bb0 0x7fb2d8075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.880 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.867+0000 7f6cb47ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6cac0b87a0 0x7f6cac0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.880 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.866+0000 7fd2499ff700 1 -- start start 2023-03-29T10:57:56.880 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.867+0000 7f6cb47ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6cac1601e0 0x7f6cac160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.880 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.867+0000 7f6cb47ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cac160ff0 con 0x7f6cac160b70 2023-03-29T10:57:56.880 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.867+0000 7f6cb47ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cac0d02d0 con 0x7f6cac0b87a0 2023-03-29T10:57:56.880 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.867+0000 7f6cb47ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac0d0cb0 con 0x7f6cac1601e0 2023-03-29T10:57:56.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7fb2f5d9b700 1 --2- 172.21.15.111:0/3545021412 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb2d8069ee0 0x7fb2d806c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.870+0000 7fb2f5d9b700 1 --2- 172.21.15.111:0/3545021412 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb2d8069ee0 0x7fb2d806c390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8000c00 tx=0x7fb2e8010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7fd2499ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd24406a3e0 0x7fd2441513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7fd2499ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2440cacf0 0x7fd2440c4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7fd2499ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd24415cdb0 0x7fd2440b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7fd2499ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440c40c0 con 0x7fd2440cacf0 2023-03-29T10:57:56.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7fd2499ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd244151910 con 0x7fd24415cdb0 2023-03-29T10:57:56.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7fd2499ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440c34b0 con 0x7fd24406a3e0 2023-03-29T10:57:56.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.869+0000 7fd242ffd700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd24406a3e0 0x7fd2441513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.869+0000 7fd2437fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd24415cdb0 0x7fd2440b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.881 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.869+0000 7fd242ffd700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd24406a3e0 0x7fd2441513d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:50800/0 (socket says 172.21.15.111:50800) 2023-03-29T10:57:56.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.869+0000 7fd242ffd700 1 -- 172.21.15.111:0/4158441818 learned_addr learned my addr 172.21.15.111:0/4158441818 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:56.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.869+0000 7fd2427fc700 1 --2- 172.21.15.111:0/4158441818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2440cacf0 0x7fd2440c4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.869+0000 7fd2437fe700 1 -- 172.21.15.111:0/4158441818 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd24406a3e0 msgr2=0x7fd2441513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.869+0000 7fd2437fe700 1 --2- 172.21.15.111:0/4158441818 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd24406a3e0 0x7fd2441513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.869+0000 7fd2437fe700 1 -- 172.21.15.111:0/4158441818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2440cacf0 msgr2=0x7fd2440c4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.869+0000 7fd2437fe700 1 --2- 172.21.15.111:0/4158441818 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd2440cacf0 0x7fd2440c4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.869+0000 7fd2437fe700 1 -- 172.21.15.111:0/4158441818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2380085d0 con 0x7fd24415cdb0 2023-03-29T10:57:56.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.870+0000 7fd2437fe700 1 --2- 172.21.15.111:0/4158441818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd24415cdb0 0x7fd2440b3460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd23c007cf0 tx=0x7fd23c00e0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.867+0000 7f97d9d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97d40ad690 0x7f97d41f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.867+0000 7f97d9d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97d40ad690 0x7f97d41f5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:40926/0 (socket says 172.21.15.111:40926) 2023-03-29T10:57:56.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.867+0000 7f97d9d9b700 1 -- 172.21.15.111:0/1948979204 learned_addr learned my addr 172.21.15.111:0/1948979204 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:56.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.867+0000 7f97d959a700 1 --2- 172.21.15.111:0/1948979204 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97d40c19c0 0x7f97d40bff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.867+0000 7f97da59c700 1 --2- 172.21.15.111:0/1948979204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97d4160d20 0x7f97d40c0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.882 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7f97d9d9b700 1 -- 172.21.15.111:0/1948979204 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97d40c19c0 msgr2=0x7f97d40bff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7f97d9d9b700 1 --2- 172.21.15.111:0/1948979204 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f97d40c19c0 0x7f97d40bff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7f97d9d9b700 1 -- 172.21.15.111:0/1948979204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97d4160d20 msgr2=0x7f97d40c0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7f97d9d9b700 1 --2- 172.21.15.111:0/1948979204 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f97d4160d20 0x7f97d40c0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7f97d9d9b700 1 -- 172.21.15.111:0/1948979204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c40094a0 con 0x7f97d40ad690 2023-03-29T10:57:56.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7ff77f7ff700 1 Processor -- start 2023-03-29T10:57:56.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.869+0000 7ff77f7ff700 1 -- start start 2023-03-29T10:57:56.883 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.869+0000 7f09c1ffb700 1 -- 172.21.15.111:0/1776614838 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f09bc0233e0 con 0x7f09cc15a140 2023-03-29T10:57:56.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.867+0000 7f6cb259b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6cac0b87a0 0x7f6cac0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7f6cb259b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6cac0b87a0 0x7f6cac0cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:40934/0 (socket says 172.21.15.111:40934) 2023-03-29T10:57:56.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7f6cb259b700 1 -- 172.21.15.111:0/3822446816 learned_addr learned my addr 172.21.15.111:0/3822446816 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:56.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7f6cb1d9a700 1 --2- 172.21.15.111:0/3822446816 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6cac1601e0 0x7f6cac160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.869+0000 7f6cb259b700 1 -- 172.21.15.111:0/3822446816 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6cac1601e0 msgr2=0x7f6cac160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:56.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.869+0000 7f6cb259b700 1 --2- 172.21.15.111:0/3822446816 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6cac1601e0 0x7f6cac160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.869+0000 7f6cb259b700 1 -- 172.21.15.111:0/3822446816 >> v1:172.21.15.111:6789/0 conn(0x7f6cac160b70 legacy=0x7f6cac0d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:56.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.869+0000 7f6cb259b700 1 -- 172.21.15.111:0/3822446816 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac0cdc60 con 0x7f6cac0b87a0 2023-03-29T10:57:56.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.870+0000 7f6cb259b700 1 --2- 172.21.15.111:0/3822446816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cac0b87a0 0x7f6cac0cfbf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80098f0 tx=0x7f6ca8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58aabb9b7f5c2eff server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.870+0000 7ff77f7ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7780cf010 0x7ff778158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.870+0000 7ff77f7ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7780cff50 0x7ff7780c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.870+0000 7ff77f7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff77815c590 0x7ff7780c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.884 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.870+0000 7ff77f7ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7781586a0 con 0x7ff7780cf010 2023-03-29T10:57:56.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.870+0000 7ff77f7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77806fc10 con 0x7ff77815c590 2023-03-29T10:57:56.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.870+0000 7ff77f7ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77806fef0 con 0x7ff7780cff50 2023-03-29T10:57:56.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.870+0000 7f6cb1599700 1 -- 172.21.15.111:0/3822446816 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8004230 con 0x7f6cac0b87a0 2023-03-29T10:57:56.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.870+0000 7f6cb1599700 1 -- 172.21.15.111:0/3822446816 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca8004390 con 0x7f6cac0b87a0 2023-03-29T10:57:56.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.870+0000 7f6cb1599700 1 -- 172.21.15.111:0/3822446816 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca80036a0 con 0x7f6cac0b87a0 2023-03-29T10:57:56.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.870+0000 7fb2f6d9d700 1 --2- 172.21.15.111:0/3545021412 >> [v2:172.21.15.111:6816/110484,v1:172.21.15.111:6817/110484] conn(0x7fb2d8073bb0 0x7fb2d8075fe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.885 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.867+0000 7f6ebc99b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eb80b87a0 0x7f6eb8154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.867+0000 7f6eb7fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6eb81601e0 0x7f6eb80b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.867+0000 7f6eb7fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6eb81601e0 0x7f6eb80b3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:50786/0 (socket says 172.21.15.111:50786) 2023-03-29T10:57:56.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.867+0000 7f6eb7fff700 1 -- 172.21.15.111:0/2341607651 learned_addr learned my addr 172.21.15.111:0/2341607651 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:56.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7f09c1ffb700 1 --2- 172.21.15.111:0/1776614838 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f09b0069eb0 0x7f09b006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7f6ebc99b700 1 -- 172.21.15.111:0/2341607651 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6eb81601e0 msgr2=0x7f6eb80b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7f6ebc99b700 1 --2- 172.21.15.111:0/2341607651 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6eb81601e0 0x7f6eb80b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.886 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7f6ebc99b700 1 -- 172.21.15.111:0/2341607651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6eb8160b70 msgr2=0x7f6eb80b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7f6ebc99b700 1 --2- 172.21.15.111:0/2341607651 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6eb8160b70 0x7f6eb80b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.868+0000 7f6ebc99b700 1 -- 172.21.15.111:0/2341607651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac0094a0 con 0x7f6eb80b87a0 2023-03-29T10:57:56.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.870+0000 7f6ebc99b700 1 --2- 172.21.15.111:0/2341607651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eb80b87a0 0x7f6eb8154890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000ebc0 tx=0x7f6eb000ef80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.870+0000 7f6eb57fa700 1 -- 172.21.15.111:0/2341607651 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb000c980 con 0x7f6eb80b87a0 2023-03-29T10:57:56.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7f6eb57fa700 1 -- 172.21.15.111:0/2341607651 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb000f040 con 0x7f6eb80b87a0 2023-03-29T10:57:56.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7f6eb57fa700 1 -- 172.21.15.111:0/2341607651 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb000cce0 con 0x7f6eb80b87a0 2023-03-29T10:57:56.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.870+0000 7fd22bfff700 1 -- 172.21.15.111:0/4158441818 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c018070 con 0x7fd24415cdb0 2023-03-29T10:57:56.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.870+0000 7fd22bfff700 1 -- 172.21.15.111:0/4158441818 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23c003a20 con 0x7fd24415cdb0 2023-03-29T10:57:56.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.870+0000 7fd22bfff700 1 -- 172.21.15.111:0/4158441818 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c003db0 con 0x7fd24415cdb0 2023-03-29T10:57:56.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7ff77dd9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff77815c590 0x7ff7780c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7ff77dd9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff77815c590 0x7ff7780c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:40950/0 (socket says 172.21.15.111:40950) 2023-03-29T10:57:56.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7ff77dd9c700 1 -- 172.21.15.111:0/2315631303 learned_addr learned my addr 172.21.15.111:0/2315631303 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:56.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7ff77cd9a700 1 --2- 172.21.15.111:0/2315631303 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7780cff50 0x7ff7780c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7ff77dd9c700 1 -- 172.21.15.111:0/2315631303 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7780cff50 msgr2=0x7ff7780c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7ff77dd9c700 1 --2- 172.21.15.111:0/2315631303 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff7780cff50 0x7ff7780c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7ff77dd9c700 1 -- 172.21.15.111:0/2315631303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7780cf010 msgr2=0x7ff778158160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-03-29T10:57:56.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7ff77dd9c700 1 --2- 172.21.15.111:0/2315631303 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff7780cf010 0x7ff778158160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7ff77dd9c700 1 -- 172.21.15.111:0/2315631303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff774009550 con 0x7ff77815c590 2023-03-29T10:57:56.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7f09c1ffb700 1 -- 172.21.15.111:0/1776614838 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f09bc088710 con 0x7f09cc15a140 2023-03-29T10:57:56.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7f09c1ffb700 1 --2- 172.21.15.111:0/1776614838 >> [v2:172.21.15.119:6808/110645,v1:172.21.15.119:6809/110645] conn(0x7f09b0073b80 0x7f09b0075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7f09c1ffb700 1 -- 172.21.15.111:0/1776614838 --> [v2:172.21.15.119:6808/110645,v1:172.21.15.119:6809/110645] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09b0076680 con 0x7f09b0073b80 2023-03-29T10:57:56.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7f09c1ffb700 1 -- 172.21.15.111:0/1776614838 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09bc088ac0 con 0x7f09cc15a140 2023-03-29T10:57:56.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.872+0000 7f09d1f9c700 1 --2- 172.21.15.111:0/1776614838 >> [v2:172.21.15.119:6808/110645,v1:172.21.15.119:6809/110645] conn(0x7f09b0073b80 0x7f09b0075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.872+0000 7f09d179b700 1 --2- 172.21.15.111:0/1776614838 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f09b0069eb0 0x7f09b006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7f6cb47ff700 1 -- 172.21.15.111:0/3822446816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cac0b87a0 msgr2=0x7f6cac0cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7f6cb47ff700 1 --2- 172.21.15.111:0/3822446816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cac0b87a0 0x7f6cac0cfbf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6ca80098f0 tx=0x7f6ca8009c00 comp rx=0 tx=0).stop 2023-03-29T10:57:56.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.872+0000 7f6cb47ff700 1 -- 172.21.15.111:0/3822446816 shutdown_connections 2023-03-29T10:57:56.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.872+0000 7f6cb47ff700 1 --2- 172.21.15.111:0/3822446816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cac0b87a0 0x7f6cac0cfbf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.872+0000 7f6cb47ff700 1 --2- 172.21.15.111:0/3822446816 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6cac1601e0 0x7f6cac160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.872+0000 7f6cb47ff700 1 -- 172.21.15.111:0/3822446816 >> 172.21.15.111:0/3822446816 conn(0x7f6cac071b10 msgr2=0x7f6cac071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:56.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.872+0000 7ff77dd9c700 1 --2- 172.21.15.111:0/2315631303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff77815c590 0x7ff7780c6cf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff77000cba0 tx=0x7ff77000cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7f6ebebff700 1 -- 172.21.15.111:0/2341607651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb80cd980 con 0x7f6eb80b87a0 2023-03-29T10:57:56.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7f6ebebff700 1 -- 172.21.15.111:0/2341607651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb80cd6a0 con 0x7f6eb80b87a0 2023-03-29T10:57:56.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7fd2499ff700 1 -- 172.21.15.111:0/4158441818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24406cc40 con 0x7fd24415cdb0 2023-03-29T10:57:56.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.871+0000 7fd2499ff700 1 -- 172.21.15.111:0/4158441818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2440cd2d0 con 0x7fd24415cdb0 2023-03-29T10:57:56.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.872+0000 7f97d9d9b700 1 --2- 172.21.15.111:0/1948979204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97d40ad690 0x7f97d41f5fb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f97c4000c00 tx=0x7f97c4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.872+0000 7fd2499ff700 1 -- 172.21.15.111:0/4158441818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd24406cf20 con 0x7fd24415cdb0 2023-03-29T10:57:56.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.872+0000 7f6cb47ff700 1 -- 172.21.15.111:0/3822446816 shutdown_connections 2023-03-29T10:57:56.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.873+0000 7f6ebebff700 1 -- 172.21.15.111:0/2341607651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6eb815d150 con 0x7f6eb80b87a0 2023-03-29T10:57:56.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.873+0000 7ff76dffb700 1 -- 172.21.15.111:0/2315631303 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770003c00 con 0x7ff77815c590 2023-03-29T10:57:56.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.873+0000 7ff76dffb700 1 -- 172.21.15.111:0/2315631303 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff770003d60 con 0x7ff77815c590 2023-03-29T10:57:56.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.873+0000 7ff76dffb700 1 -- 172.21.15.111:0/2315631303 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77000abc0 con 0x7ff77815c590 2023-03-29T10:57:56.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.873+0000 7ff77f7ff700 1 -- 172.21.15.111:0/2315631303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7780cd770 con 0x7ff77815c590 2023-03-29T10:57:56.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.873+0000 7ff77f7ff700 1 -- 172.21.15.111:0/2315631303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7780cf560 con 0x7ff77815c590 2023-03-29T10:57:56.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.874+0000 7f09d1f9c700 1 --2- 172.21.15.111:0/1776614838 >> [v2:172.21.15.119:6808/110645,v1:172.21.15.119:6809/110645] conn(0x7f09b0073b80 0x7f09b0075fb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.874+0000 7fb2e6ffd700 1 -- 172.21.15.111:0/3545021412 <== osd.0 v2:172.21.15.111:6816/110484 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb2d80766b0 con 0x7fb2d8073bb0 2023-03-29T10:57:56.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.874+0000 7f09d179b700 1 --2- 172.21.15.111:0/1776614838 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f09b0069eb0 0x7f09b006c360 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f09c8000c00 tx=0x7f09c8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.874+0000 7f97ca7fc700 1 -- 172.21.15.111:0/1948979204 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4004590 con 0x7f97d40ad690 2023-03-29T10:57:56.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.874+0000 7f97ca7fc700 1 -- 172.21.15.111:0/1948979204 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c401e070 con 0x7f97d40ad690 2023-03-29T10:57:56.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.874+0000 7f97ca7fc700 1 -- 172.21.15.111:0/1948979204 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c40038e0 con 0x7f97d40ad690 2023-03-29T10:57:56.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.875+0000 7f97dbfff700 1 -- 172.21.15.111:0/1948979204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d40ccf10 con 0x7f97d40ad690 2023-03-29T10:57:56.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.875+0000 7ff77f7ff700 1 -- 172.21.15.111:0/2315631303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff77815c960 con 0x7ff77815c590 2023-03-29T10:57:56.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.875+0000 7f6eb57fa700 1 -- 172.21.15.111:0/2341607651 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f6eb0039930 con 0x7f6eb80b87a0 2023-03-29T10:57:56.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.876+0000 7f6eb57fa700 1 --2- 172.21.15.111:0/2341607651 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6ea4069eb0 0x7f6ea406c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.877+0000 7f6eb57fa700 1 -- 172.21.15.111:0/2341607651 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6eb0088800 con 0x7f6eb80b87a0 2023-03-29T10:57:56.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.877+0000 7f6eb57fa700 1 --2- 172.21.15.111:0/2341607651 >> [v2:172.21.15.111:6824/110486,v1:172.21.15.111:6825/110486] conn(0x7f6ea4073b80 0x7f6ea4075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.877+0000 7f6eb57fa700 1 -- 172.21.15.111:0/2341607651 --> [v2:172.21.15.111:6824/110486,v1:172.21.15.111:6825/110486] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ea4076680 con 0x7f6ea4073b80 2023-03-29T10:57:56.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.877+0000 7f6eb57fa700 1 -- 172.21.15.111:0/2341607651 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6eb0088bb0 con 0x7f6eb80b87a0 2023-03-29T10:57:56.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.875+0000 7f97dbfff700 1 -- 172.21.15.111:0/1948979204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d40ccc30 con 0x7f97d40ad690 2023-03-29T10:57:56.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.876+0000 7f97dbfff700 1 -- 172.21.15.111:0/1948979204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f97d40c2ec0 con 0x7f97d40ad690 2023-03-29T10:57:56.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.872+0000 7f6cb47ff700 1 -- 172.21.15.111:0/3822446816 wait complete. 2023-03-29T10:57:56.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.875+0000 7f6cb47ff700 1 Processor -- start 2023-03-29T10:57:56.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.878+0000 7f6eb7fff700 1 --2- 172.21.15.111:0/2341607651 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6ea4069eb0 0x7f6ea406c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.875+0000 7f09c1ffb700 1 -- 172.21.15.111:0/1776614838 <== osd.6 v2:172.21.15.119:6808/110645 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f09b0076680 con 0x7f09b0073b80 2023-03-29T10:57:56.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.875+0000 7f6cb47ff700 1 -- start start 2023-03-29T10:57:56.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.877+0000 7f6cb47ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6cac0b87a0 0x7f6cac0c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.877+0000 7f6cb47ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cac1601e0 0x7f6cac15b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.894 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.877+0000 7f6cb47ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cac160b70 0x7f6cac15f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.877+0000 7f6cb47ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac15eb50 con 0x7f6cac1601e0 2023-03-29T10:57:56.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.877+0000 7f6cb47ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac15bc50 con 0x7f6cac160b70 2023-03-29T10:57:56.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.877+0000 7f6cb47ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cac15dd90 con 0x7f6cac0b87a0 2023-03-29T10:57:56.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.878+0000 7f6cb1d9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cac1601e0 0x7f6cac15b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.878+0000 7f6cb2d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cac160b70 0x7f6cac15f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.878+0000 7f6cb2d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cac160b70 0x7f6cac15f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:40956/0 (socket says 172.21.15.111:40956) 2023-03-29T10:57:56.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.878+0000 7f6cb2d9c700 1 -- 172.21.15.111:0/2001919289 learned_addr learned my addr 172.21.15.111:0/2001919289 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:56.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.879+0000 7f6ebd19c700 1 --2- 172.21.15.111:0/2341607651 >> [v2:172.21.15.111:6824/110486,v1:172.21.15.111:6825/110486] conn(0x7f6ea4073b80 0x7f6ea4075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.879+0000 7f6eb7fff700 1 --2- 172.21.15.111:0/2341607651 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6ea4069eb0 0x7f6ea406c360 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6eb81e72c0 tx=0x7f6eac010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.879+0000 7f6ebd19c700 1 --2- 172.21.15.111:0/2341607651 >> [v2:172.21.15.111:6824/110486,v1:172.21.15.111:6825/110486] conn(0x7f6ea4073b80 0x7f6ea4075fb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.879+0000 7f6cb2d9c700 1 -- 172.21.15.111:0/2001919289 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6cac0b87a0 msgr2=0x7f6cac0c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.879+0000 7f6cb2d9c700 1 --2- 172.21.15.111:0/2001919289 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6cac0b87a0 0x7f6cac0c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.879+0000 7f6cb2d9c700 1 -- 172.21.15.111:0/2001919289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cac1601e0 msgr2=0x7f6cac15b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.879+0000 7f6cb2d9c700 1 --2- 172.21.15.111:0/2001919289 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cac1601e0 0x7f6cac15b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.879+0000 7f6cb2d9c700 1 -- 172.21.15.111:0/2001919289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca8009550 con 0x7f6cac160b70 2023-03-29T10:57:56.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.880+0000 7f6cb2d9c700 1 --2- 172.21.15.111:0/2001919289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cac160b70 0x7f6cac15f140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00c4c0 tx=0x7f6c9c001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.878+0000 7f97ca7fc700 1 -- 172.21.15.111:0/1948979204 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f97c4003a40 con 0x7f97d40ad690 2023-03-29T10:57:56.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.879+0000 7f97ca7fc700 1 --2- 172.21.15.111:0/1948979204 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f97b8069eb0 0x7f97b806c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.880+0000 7f97ca7fc700 1 -- 172.21.15.111:0/1948979204 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f97c40891e0 con 0x7f97d40ad690 2023-03-29T10:57:56.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.880+0000 7f97ca7fc700 1 --2- 172.21.15.111:0/1948979204 >> [v2:172.21.15.119:6800/110644,v1:172.21.15.119:6801/110644] conn(0x7f97b8073b80 0x7f97b8075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.880+0000 7f97ca7fc700 1 -- 172.21.15.111:0/1948979204 --> [v2:172.21.15.119:6800/110644,v1:172.21.15.119:6801/110644] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f97b8076680 con 0x7f97b8073b80 2023-03-29T10:57:56.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.880+0000 7f97ca7fc700 1 -- 172.21.15.111:0/1948979204 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97c4089590 con 0x7f97d40ad690 2023-03-29T10:57:56.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.880+0000 7f97d959a700 1 --2- 172.21.15.111:0/1948979204 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f97b8069eb0 0x7f97b806c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.881+0000 7f6eb57fa700 1 -- 172.21.15.111:0/2341607651 <== osd.2 v2:172.21.15.111:6824/110486 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6ea4076680 con 0x7f6ea4073b80 2023-03-29T10:57:56.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.880+0000 7f6ca2ffd700 1 -- 172.21.15.111:0/2001919289 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c0054e0 con 0x7f6cac160b70 2023-03-29T10:57:56.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.880+0000 7f6ca2ffd700 1 -- 172.21.15.111:0/2001919289 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9c003680 con 0x7f6cac160b70 2023-03-29T10:57:56.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.880+0000 7f6ca2ffd700 1 -- 172.21.15.111:0/2001919289 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c003a30 con 0x7f6cac160b70 2023-03-29T10:57:56.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.881+0000 7f6cb47ff700 1 -- 172.21.15.111:0/2001919289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cac0cd980 con 0x7f6cac160b70 2023-03-29T10:57:56.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.881+0000 7f6cb47ff700 1 -- 172.21.15.111:0/2001919289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cac0cd6a0 con 0x7f6cac160b70 2023-03-29T10:57:56.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.881+0000 7f97da59c700 1 --2- 172.21.15.111:0/1948979204 >> [v2:172.21.15.119:6800/110644,v1:172.21.15.119:6801/110644] conn(0x7f97b8073b80 0x7f97b8075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.882+0000 7f97d959a700 1 --2- 172.21.15.111:0/1948979204 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f97b8069eb0 0x7f97b806c360 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f97d4155cf0 tx=0x7f97d0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.880+0000 7ff76dffb700 1 -- 172.21.15.111:0/2315631303 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7ff77000ad20 con 0x7ff77815c590 2023-03-29T10:57:56.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.882+0000 7fd22bfff700 1 -- 172.21.15.111:0/4158441818 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fd23c00ea20 con 0x7fd24415cdb0 2023-03-29T10:57:56.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.882+0000 7ff76dffb700 1 --2- 172.21.15.111:0/2315631303 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff75c069de0 0x7ff75c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.882+0000 7ff76dffb700 1 -- 172.21.15.111:0/2315631303 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff770088610 con 0x7ff77815c590 2023-03-29T10:57:56.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.883+0000 7fd22bfff700 1 --2- 172.21.15.111:0/4158441818 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd230069eb0 0x7fd23006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.883+0000 7fd22bfff700 1 -- 172.21.15.111:0/4158441818 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd23c089020 con 0x7fd24415cdb0 2023-03-29T10:57:56.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.884+0000 7fd22bfff700 1 --2- 172.21.15.111:0/4158441818 >> [v2:172.21.15.111:6800/110485,v1:172.21.15.111:6801/110485] conn(0x7fd230073b80 0x7fd230075fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.884+0000 7fd22bfff700 1 -- 172.21.15.111:0/4158441818 --> [v2:172.21.15.111:6800/110485,v1:172.21.15.111:6801/110485] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd230076680 con 0x7fd230073b80 2023-03-29T10:57:56.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.884+0000 7fd22bfff700 1 -- 172.21.15.111:0/4158441818 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd23c0893d0 con 0x7fd24415cdb0 2023-03-29T10:57:56.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.884+0000 7fd242ffd700 1 --2- 172.21.15.111:0/4158441818 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd230069eb0 0x7fd23006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.883+0000 7ff76dffb700 1 --2- 172.21.15.111:0/2315631303 >> [v2:172.21.15.119:6816/110646,v1:172.21.15.119:6817/110646] conn(0x7ff75c073ab0 0x7ff75c075ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.883+0000 7ff76dffb700 1 -- 172.21.15.111:0/2315631303 --> [v2:172.21.15.119:6816/110646,v1:172.21.15.119:6817/110646] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff75c0765b0 con 0x7ff75c073ab0 2023-03-29T10:57:56.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.883+0000 7ff76dffb700 1 -- 172.21.15.111:0/2315631303 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7700889c0 con 0x7ff77815c590 2023-03-29T10:57:56.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.884+0000 7ff77cd9a700 1 --2- 172.21.15.111:0/2315631303 >> [v2:172.21.15.119:6816/110646,v1:172.21.15.119:6817/110646] conn(0x7ff75c073ab0 0x7ff75c075ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.885+0000 7ff77d59b700 1 --2- 172.21.15.111:0/2315631303 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff75c069de0 0x7ff75c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.885+0000 7fd242ffd700 1 --2- 172.21.15.111:0/4158441818 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd230069eb0 0x7fd23006c360 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd234009ad0 tx=0x7fd234009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.885+0000 7f97da59c700 1 --2- 172.21.15.111:0/1948979204 >> [v2:172.21.15.119:6800/110644,v1:172.21.15.119:6801/110644] conn(0x7f97b8073b80 0x7f97b8075fb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.886+0000 7ff77d59b700 1 --2- 172.21.15.111:0/2315631303 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff75c069de0 0x7ff75c06c290 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff768009690 tx=0x7ff768006be0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.886+0000 7ff77cd9a700 1 --2- 172.21.15.111:0/2315631303 >> [v2:172.21.15.119:6816/110646,v1:172.21.15.119:6817/110646] conn(0x7ff75c073ab0 0x7ff75c075ee0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.886+0000 7fd2427fc700 1 --2- 172.21.15.111:0/4158441818 >> [v2:172.21.15.111:6800/110485,v1:172.21.15.111:6801/110485] conn(0x7fd230073b80 0x7fd230075fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.887+0000 7f97ca7fc700 1 -- 172.21.15.111:0/1948979204 <== osd.4 v2:172.21.15.119:6800/110644 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f97b8076680 con 0x7f97b8073b80 2023-03-29T10:57:56.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.887+0000 7f6ca2ffd700 1 -- 172.21.15.111:0/2001919289 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f6c9c003b90 con 0x7f6cac160b70 2023-03-29T10:57:56.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.887+0000 7fd2427fc700 1 --2- 172.21.15.111:0/4158441818 >> [v2:172.21.15.111:6800/110485,v1:172.21.15.111:6801/110485] conn(0x7fd230073b80 0x7fd230075fb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.888+0000 7f6ca2ffd700 1 --2- 172.21.15.111:0/2001919289 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6c90069eb0 0x7f6c9006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.888+0000 7f6ca2ffd700 1 -- 172.21.15.111:0/2001919289 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6c9c0877f0 con 0x7f6cac160b70 2023-03-29T10:57:56.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.888+0000 7f6cb47ff700 1 --2- 172.21.15.111:0/2001919289 >> [v2:172.21.15.119:6824/110655,v1:172.21.15.119:6825/110655] conn(0x7f6cac0b1e30 0x7f6cac15e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:56.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.888+0000 7f6cb47ff700 1 -- 172.21.15.111:0/2001919289 --> [v2:172.21.15.119:6824/110655,v1:172.21.15.119:6825/110655] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6cac074430 con 0x7f6cac0b1e30 2023-03-29T10:57:56.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.888+0000 7ff76dffb700 1 -- 172.21.15.111:0/2315631303 <== osd.5 v2:172.21.15.119:6816/110646 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff75c0765b0 con 0x7ff75c073ab0 2023-03-29T10:57:56.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.889+0000 7f6cb1d9a700 1 --2- 172.21.15.111:0/2001919289 >> [v2:172.21.15.119:6824/110655,v1:172.21.15.119:6825/110655] conn(0x7f6cac0b1e30 0x7f6cac15e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.889+0000 7f6cb259b700 1 --2- 172.21.15.111:0/2001919289 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6c90069eb0 0x7f6c9006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:56.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.890+0000 7f6cb259b700 1 --2- 172.21.15.111:0/2001919289 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6c90069eb0 0x7f6c9006c360 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009c40 tx=0x7f6ca8010000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.892+0000 7f6cb1d9a700 1 --2- 172.21.15.111:0/2001919289 >> [v2:172.21.15.119:6824/110655,v1:172.21.15.119:6825/110655] conn(0x7f6cac0b1e30 0x7f6cac15e280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:56.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.893+0000 7f6ca2ffd700 1 -- 172.21.15.111:0/2001919289 <== osd.7 v2:172.21.15.119:6824/110655 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6cac074430 con 0x7f6cac0b1e30 2023-03-29T10:57:56.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.890+0000 7fd22bfff700 1 -- 172.21.15.111:0/4158441818 <== osd.3 v2:172.21.15.111:6800/110485 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd230076680 con 0x7fd230073b80 2023-03-29T10:57:56.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.895+0000 7f192b5ff700 1 -- 172.21.15.111:0/2337583476 --> [v2:172.21.15.111:6808/110483,v1:172.21.15.111:6809/110483] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f19240b52f0 con 0x7f1908073b80 2023-03-29T10:57:56.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.896+0000 7f1919ffb700 1 -- 172.21.15.111:0/2337583476 <== osd.1 v2:172.21.15.111:6808/110483 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f19240b52f0 con 0x7f1908073b80 2023-03-29T10:57:56.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.897+0000 7f192b5ff700 1 -- 172.21.15.111:0/2337583476 >> [v2:172.21.15.111:6808/110483,v1:172.21.15.111:6809/110483] conn(0x7f1908073b80 msgr2=0x7f1908075fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.897+0000 7f192b5ff700 1 --2- 172.21.15.111:0/2337583476 >> [v2:172.21.15.111:6808/110483,v1:172.21.15.111:6809/110483] conn(0x7f1908073b80 0x7f1908075fb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.898+0000 7f192b5ff700 1 -- 172.21.15.111:0/2337583476 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1908069eb0 msgr2=0x7f190806c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.898+0000 7f192b5ff700 1 --2- 172.21.15.111:0/2337583476 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1908069eb0 0x7f190806c360 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1920001eb0 tx=0x7f1920009040 comp rx=0 tx=0).stop 2023-03-29T10:57:56.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.898+0000 7f192b5ff700 1 -- 172.21.15.111:0/2337583476 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19240c69e0 msgr2=0x7f19240c9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.898+0000 7f192b5ff700 1 --2- 172.21.15.111:0/2337583476 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19240c69e0 0x7f19240c9f80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f191400aec0 tx=0x7f19140047c0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.898+0000 7f1929b9c700 1 -- 172.21.15.111:0/2337583476 reap_dead start 2023-03-29T10:57:56.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.899+0000 7f192b5ff700 1 -- 172.21.15.111:0/2337583476 shutdown_connections 2023-03-29T10:57:56.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.899+0000 7f192b5ff700 1 -- 172.21.15.111:0/2337583476 >> 172.21.15.111:0/2337583476 conn(0x7f1924071b10 msgr2=0x7f19240b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:56.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.899+0000 7f192b5ff700 1 -- 172.21.15.111:0/2337583476 shutdown_connections 2023-03-29T10:57:56.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.899+0000 7f192b5ff700 1 -- 172.21.15.111:0/2337583476 wait complete. 2023-03-29T10:57:56.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.911+0000 7fb2fcbff700 1 -- 172.21.15.111:0/3545021412 --> [v2:172.21.15.111:6816/110484,v1:172.21.15.111:6817/110484] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb2f8074430 con 0x7fb2d8073bb0 2023-03-29T10:57:56.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.913+0000 7fb2e6ffd700 1 -- 172.21.15.111:0/3545021412 <== osd.0 v2:172.21.15.111:6816/110484 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb2f8074430 con 0x7fb2d8073bb0 2023-03-29T10:57:56.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.913+0000 7f09d39ff700 1 -- 172.21.15.111:0/1776614838 --> [v2:172.21.15.119:6808/110645,v1:172.21.15.119:6809/110645] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f09ac002d30 con 0x7f09b0073b80 2023-03-29T10:57:56.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.913+0000 7fb2fcbff700 1 -- 172.21.15.111:0/3545021412 >> [v2:172.21.15.111:6816/110484,v1:172.21.15.111:6817/110484] conn(0x7fb2d8073bb0 msgr2=0x7fb2d8075fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.913+0000 7fb2fcbff700 1 --2- 172.21.15.111:0/3545021412 >> [v2:172.21.15.111:6816/110484,v1:172.21.15.111:6817/110484] conn(0x7fb2d8073bb0 0x7fb2d8075fe0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.914+0000 7fb2fcbff700 1 -- 172.21.15.111:0/3545021412 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb2d8069ee0 msgr2=0x7fb2d806c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.914+0000 7fb2fcbff700 1 --2- 172.21.15.111:0/3545021412 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb2d8069ee0 0x7fb2d806c390 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8000c00 tx=0x7fb2e8010000 comp rx=0 tx=0).stop 2023-03-29T10:57:56.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.914+0000 7fb2fcbff700 1 -- 172.21.15.111:0/3545021412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2f80ad690 msgr2=0x7fb2f80ca410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.914+0000 7fb2fcbff700 1 --2- 172.21.15.111:0/3545021412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2f80ad690 0x7fb2f80ca410 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0002080 tx=0x7fb2f000a340 comp rx=0 tx=0).stop 2023-03-29T10:57:56.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.914+0000 7fb2f6d9d700 1 -- 172.21.15.111:0/3545021412 reap_dead start 2023-03-29T10:57:56.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.914+0000 7fb2fcbff700 1 -- 172.21.15.111:0/3545021412 shutdown_connections 2023-03-29T10:57:56.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.914+0000 7fb2fcbff700 1 -- 172.21.15.111:0/3545021412 >> 172.21.15.111:0/3545021412 conn(0x7fb2f80cf720 msgr2=0x7fb2f80b1950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:56.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.915+0000 7fb2fcbff700 1 -- 172.21.15.111:0/3545021412 shutdown_connections 2023-03-29T10:57:56.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.915+0000 7f09c1ffb700 1 -- 172.21.15.111:0/1776614838 <== osd.6 v2:172.21.15.119:6808/110645 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f09ac002d30 con 0x7f09b0073b80 2023-03-29T10:57:56.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.915+0000 7fb2fcbff700 1 -- 172.21.15.111:0/3545021412 wait complete. 2023-03-29T10:57:56.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.915+0000 7f6ebebff700 1 -- 172.21.15.111:0/2341607651 --> [v2:172.21.15.111:6824/110486,v1:172.21.15.111:6825/110486] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6eb80d2110 con 0x7f6ea4073b80 2023-03-29T10:57:56.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.915+0000 7f09d39ff700 1 -- 172.21.15.111:0/1776614838 >> [v2:172.21.15.119:6808/110645,v1:172.21.15.119:6809/110645] conn(0x7f09b0073b80 msgr2=0x7f09b0075fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.915+0000 7f09d39ff700 1 --2- 172.21.15.111:0/1776614838 >> [v2:172.21.15.119:6808/110645,v1:172.21.15.119:6809/110645] conn(0x7f09b0073b80 0x7f09b0075fb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.916+0000 7f09d39ff700 1 -- 172.21.15.111:0/1776614838 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f09b0069eb0 msgr2=0x7f09b006c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.916+0000 7f09d39ff700 1 --2- 172.21.15.111:0/1776614838 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f09b0069eb0 0x7f09b006c360 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f09c8000c00 tx=0x7f09c8008030 comp rx=0 tx=0).stop 2023-03-29T10:57:56.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.916+0000 7f09d39ff700 1 -- 172.21.15.111:0/1776614838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09cc15a140 msgr2=0x7f09cc0b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.916+0000 7f09d39ff700 1 --2- 172.21.15.111:0/1776614838 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09cc15a140 0x7f09cc0b3180 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f09bc00c7c0 tx=0x7f09bc00cad0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.916+0000 7f09d1f9c700 1 -- 172.21.15.111:0/1776614838 reap_dead start 2023-03-29T10:57:56.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.917+0000 7f6eb57fa700 1 -- 172.21.15.111:0/2341607651 <== osd.2 v2:172.21.15.111:6824/110486 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6eb80d2110 con 0x7f6ea4073b80 2023-03-29T10:57:56.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.917+0000 7f09d39ff700 1 -- 172.21.15.111:0/1776614838 shutdown_connections 2023-03-29T10:57:56.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.917+0000 7f09d39ff700 1 -- 172.21.15.111:0/1776614838 >> 172.21.15.111:0/1776614838 conn(0x7f09cc071b10 msgr2=0x7f09cc071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:56.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.917+0000 7f09d39ff700 1 -- 172.21.15.111:0/1776614838 shutdown_connections 2023-03-29T10:57:56.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.917+0000 7f09d39ff700 1 -- 172.21.15.111:0/1776614838 wait complete. 2023-03-29T10:57:56.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.919+0000 7f97dbfff700 1 -- 172.21.15.111:0/1948979204 --> [v2:172.21.15.119:6800/110644,v1:172.21.15.119:6801/110644] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f97d406d0d0 con 0x7f97b8073b80 2023-03-29T10:57:56.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.920+0000 7f6ebebff700 1 -- 172.21.15.111:0/2341607651 >> [v2:172.21.15.111:6824/110486,v1:172.21.15.111:6825/110486] conn(0x7f6ea4073b80 msgr2=0x7f6ea4075fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.920+0000 7f6ebebff700 1 --2- 172.21.15.111:0/2341607651 >> [v2:172.21.15.111:6824/110486,v1:172.21.15.111:6825/110486] conn(0x7f6ea4073b80 0x7f6ea4075fb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.920+0000 7f97ca7fc700 1 -- 172.21.15.111:0/1948979204 <== osd.4 v2:172.21.15.119:6800/110644 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f97d406d0d0 con 0x7f97b8073b80 2023-03-29T10:57:56.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.920+0000 7f6ebebff700 1 -- 172.21.15.111:0/2341607651 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6ea4069eb0 msgr2=0x7f6ea406c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.920+0000 7f6ebebff700 1 --2- 172.21.15.111:0/2341607651 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6ea4069eb0 0x7f6ea406c360 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6eb81e72c0 tx=0x7f6eac010000 comp rx=0 tx=0).stop 2023-03-29T10:57:56.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.920+0000 7f6ebebff700 1 -- 172.21.15.111:0/2341607651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eb80b87a0 msgr2=0x7f6eb8154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.920+0000 7f6ebebff700 1 --2- 172.21.15.111:0/2341607651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eb80b87a0 0x7f6eb8154890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000ebc0 tx=0x7f6eb000ef80 comp rx=0 tx=0).stop 2023-03-29T10:57:56.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.920+0000 7f6ebd19c700 1 -- 172.21.15.111:0/2341607651 reap_dead start 2023-03-29T10:57:56.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.921+0000 7f6ebebff700 1 -- 172.21.15.111:0/2341607651 shutdown_connections 2023-03-29T10:57:56.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.921+0000 7f6ebebff700 1 -- 172.21.15.111:0/2341607651 >> 172.21.15.111:0/2341607651 conn(0x7f6eb8071b10 msgr2=0x7f6eb80cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:56.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.922+0000 7fd2499ff700 1 -- 172.21.15.111:0/4158441818 --> [v2:172.21.15.111:6800/110485,v1:172.21.15.111:6801/110485] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd244074430 con 0x7fd230073b80 2023-03-29T10:57:56.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.923+0000 7f6ebebff700 1 -- 172.21.15.111:0/2341607651 shutdown_connections 2023-03-29T10:57:56.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.923+0000 7f6ebebff700 1 -- 172.21.15.111:0/2341607651 wait complete. 2023-03-29T10:57:56.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.923+0000 7fd22bfff700 1 -- 172.21.15.111:0/4158441818 <== osd.3 v2:172.21.15.111:6800/110485 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd244074430 con 0x7fd230073b80 2023-03-29T10:57:56.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.923+0000 7ff77f7ff700 1 -- 172.21.15.111:0/2315631303 --> [v2:172.21.15.119:6816/110646,v1:172.21.15.119:6817/110646] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff778074430 con 0x7ff75c073ab0 2023-03-29T10:57:56.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.923+0000 7f97dbfff700 1 -- 172.21.15.111:0/1948979204 >> [v2:172.21.15.119:6800/110644,v1:172.21.15.119:6801/110644] conn(0x7f97b8073b80 msgr2=0x7f97b8075fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.923+0000 7f97dbfff700 1 --2- 172.21.15.111:0/1948979204 >> [v2:172.21.15.119:6800/110644,v1:172.21.15.119:6801/110644] conn(0x7f97b8073b80 0x7f97b8075fb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.923+0000 7fd2499ff700 1 -- 172.21.15.111:0/4158441818 >> [v2:172.21.15.111:6800/110485,v1:172.21.15.111:6801/110485] conn(0x7fd230073b80 msgr2=0x7fd230075fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.923+0000 7fd2499ff700 1 --2- 172.21.15.111:0/4158441818 >> [v2:172.21.15.111:6800/110485,v1:172.21.15.111:6801/110485] conn(0x7fd230073b80 0x7fd230075fb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.923+0000 7f97dbfff700 1 -- 172.21.15.111:0/1948979204 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f97b8069eb0 msgr2=0x7f97b806c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.923+0000 7f97dbfff700 1 --2- 172.21.15.111:0/1948979204 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f97b8069eb0 0x7f97b806c360 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f97d4155cf0 tx=0x7f97d0008040 comp rx=0 tx=0).stop 2023-03-29T10:57:56.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.923+0000 7f97dbfff700 1 -- 172.21.15.111:0/1948979204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97d40ad690 msgr2=0x7f97d41f5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.923+0000 7f97dbfff700 1 --2- 172.21.15.111:0/1948979204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97d40ad690 0x7f97d41f5fb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f97c4000c00 tx=0x7f97c4004750 comp rx=0 tx=0).stop 2023-03-29T10:57:56.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.924+0000 7f97da59c700 1 -- 172.21.15.111:0/1948979204 reap_dead start 2023-03-29T10:57:56.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.924+0000 7f97dbfff700 1 -- 172.21.15.111:0/1948979204 shutdown_connections 2023-03-29T10:57:56.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.924+0000 7f97dbfff700 1 -- 172.21.15.111:0/1948979204 >> 172.21.15.111:0/1948979204 conn(0x7f97d40cf720 msgr2=0x7f97d40b4040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:56.925 INFO:teuthology.orchestra.run.smithi111.stdout:55834574860 2023-03-29T10:57:56.925 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-03-29T10:57:56.926 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.924+0000 7f97dbfff700 1 -- 172.21.15.111:0/1948979204 shutdown_connections 2023-03-29T10:57:56.926 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.924+0000 7f97dbfff700 1 -- 172.21.15.111:0/1948979204 wait complete. 2023-03-29T10:57:56.926 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.924+0000 7ff76dffb700 1 -- 172.21.15.111:0/2315631303 <== osd.5 v2:172.21.15.119:6816/110646 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff778074430 con 0x7ff75c073ab0 2023-03-29T10:57:56.926 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.924+0000 7fd2499ff700 1 -- 172.21.15.111:0/4158441818 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd230069eb0 msgr2=0x7fd23006c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.924+0000 7fd2499ff700 1 --2- 172.21.15.111:0/4158441818 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd230069eb0 0x7fd23006c360 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd234009ad0 tx=0x7fd234009210 comp rx=0 tx=0).stop 2023-03-29T10:57:56.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.924+0000 7fd2499ff700 1 -- 172.21.15.111:0/4158441818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd24415cdb0 msgr2=0x7fd2440b3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.924+0000 7fd2499ff700 1 --2- 172.21.15.111:0/4158441818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd24415cdb0 0x7fd2440b3460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd23c007cf0 tx=0x7fd23c00e0e0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.924+0000 7fd2437fe700 1 -- 172.21.15.111:0/4158441818 reap_dead start 2023-03-29T10:57:56.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.925+0000 7fd2499ff700 1 -- 172.21.15.111:0/4158441818 shutdown_connections 2023-03-29T10:57:56.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.925+0000 7fd2499ff700 1 -- 172.21.15.111:0/4158441818 >> 172.21.15.111:0/4158441818 conn(0x7fd2440cf6e0 msgr2=0x7fd24415b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:56.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.925+0000 7ff77f7ff700 1 -- 172.21.15.111:0/2315631303 >> [v2:172.21.15.119:6816/110646,v1:172.21.15.119:6817/110646] conn(0x7ff75c073ab0 msgr2=0x7ff75c075ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.925+0000 7ff77f7ff700 1 --2- 172.21.15.111:0/2315631303 >> [v2:172.21.15.119:6816/110646,v1:172.21.15.119:6817/110646] conn(0x7ff75c073ab0 0x7ff75c075ee0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.926+0000 7ff77f7ff700 1 -- 172.21.15.111:0/2315631303 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff75c069de0 msgr2=0x7ff75c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.926+0000 7ff77f7ff700 1 --2- 172.21.15.111:0/2315631303 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff75c069de0 0x7ff75c06c290 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff768009690 tx=0x7ff768006be0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.926+0000 7ff77f7ff700 1 -- 172.21.15.111:0/2315631303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff77815c590 msgr2=0x7ff7780c6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.926+0000 7ff77f7ff700 1 --2- 172.21.15.111:0/2315631303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff77815c590 0x7ff7780c6cf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff77000cba0 tx=0x7ff77000cf60 comp rx=0 tx=0).stop 2023-03-29T10:57:56.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.926+0000 7fd2499ff700 1 -- 172.21.15.111:0/4158441818 shutdown_connections 2023-03-29T10:57:56.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.926+0000 7ff77dd9c700 1 -- 172.21.15.111:0/2315631303 reap_dead start 2023-03-29T10:57:56.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.927+0000 7fd2499ff700 1 -- 172.21.15.111:0/4158441818 wait complete. 2023-03-29T10:57:56.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.928+0000 7f6cb47ff700 1 -- 172.21.15.111:0/2001919289 --> [v2:172.21.15.119:6824/110655,v1:172.21.15.119:6825/110655] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6cac153750 con 0x7f6cac0b1e30 2023-03-29T10:57:56.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.928+0000 7ff77f7ff700 1 -- 172.21.15.111:0/2315631303 shutdown_connections 2023-03-29T10:57:56.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.928+0000 7ff77f7ff700 1 -- 172.21.15.111:0/2315631303 >> 172.21.15.111:0/2315631303 conn(0x7ff778071b10 msgr2=0x7ff778072ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:56.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.928+0000 7ff77f7ff700 1 -- 172.21.15.111:0/2315631303 shutdown_connections 2023-03-29T10:57:56.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.928+0000 7ff77f7ff700 1 -- 172.21.15.111:0/2315631303 wait complete. 2023-03-29T10:57:56.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.929+0000 7f6ca2ffd700 1 -- 172.21.15.111:0/2001919289 <== osd.7 v2:172.21.15.119:6824/110655 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6cac153750 con 0x7f6cac0b1e30 2023-03-29T10:57:56.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.929+0000 7f6cb47ff700 1 -- 172.21.15.111:0/2001919289 >> [v2:172.21.15.119:6824/110655,v1:172.21.15.119:6825/110655] conn(0x7f6cac0b1e30 msgr2=0x7f6cac15e280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.929+0000 7f6cb47ff700 1 --2- 172.21.15.111:0/2001919289 >> [v2:172.21.15.119:6824/110655,v1:172.21.15.119:6825/110655] conn(0x7f6cac0b1e30 0x7f6cac15e280 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:56.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.932+0000 7f6cb47ff700 1 -- 172.21.15.111:0/2001919289 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6c90069eb0 msgr2=0x7f6c9006c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.932+0000 7f6cb47ff700 1 --2- 172.21.15.111:0/2001919289 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6c90069eb0 0x7f6c9006c360 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8009c40 tx=0x7f6ca8010000 comp rx=0 tx=0).stop 2023-03-29T10:57:56.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.932+0000 7f6cb47ff700 1 -- 172.21.15.111:0/2001919289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cac160b70 msgr2=0x7f6cac15f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:56.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.932+0000 7f6cb47ff700 1 --2- 172.21.15.111:0/2001919289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cac160b70 0x7f6cac15f140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00c4c0 tx=0x7f6c9c001e70 comp rx=0 tx=0).stop 2023-03-29T10:57:56.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.937+0000 7f6cb2d9c700 1 -- 172.21.15.111:0/2001919289 reap_dead start 2023-03-29T10:57:56.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.937+0000 7f6cb47ff700 1 -- 172.21.15.111:0/2001919289 shutdown_connections 2023-03-29T10:57:56.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.937+0000 7f6cb47ff700 1 -- 172.21.15.111:0/2001919289 >> 172.21.15.111:0/2001919289 conn(0x7f6cac071b10 msgr2=0x7f6cac0cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:56.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.937+0000 7f6cb47ff700 1 -- 172.21.15.111:0/2001919289 shutdown_connections 2023-03-29T10:57:56.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:56.937+0000 7f6cb47ff700 1 -- 172.21.15.111:0/2001919289 wait complete. 2023-03-29T10:57:56.941 INFO:teuthology.orchestra.run.smithi111.stdout:55834574860 2023-03-29T10:57:56.941 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-03-29T10:57:56.949 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2023-03-29T10:57:56.950 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-03-29T10:57:56.956 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2023-03-29T10:57:56.957 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-03-29T10:57:56.961 INFO:teuthology.orchestra.run.smithi111.stdout:55834574860 2023-03-29T10:57:56.961 INFO:teuthology.orchestra.run.smithi111.stdout:55834574860 2023-03-29T10:57:56.962 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-03-29T10:57:56.963 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2023-03-29T10:57:56.963 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-03-29T10:57:56.964 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-03-29T10:57:56.965 INFO:teuthology.orchestra.run.smithi111.stdout:55834574860 2023-03-29T10:57:56.966 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-03-29T10:57:57.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.224+0000 7f41b0dff700 1 Processor -- start 2023-03-29T10:57:57.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.224+0000 7f41b0dff700 1 -- start start 2023-03-29T10:57:57.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.226+0000 7f41b0dff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f41ac0ad690 0x7f41ac0adb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.226+0000 7f41b0dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f41ac0c19c0 0x7f41ac0c1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.226+0000 7f41b0dff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41ac0cd1f0 con 0x7f41ac160d20 2023-03-29T10:57:57.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.226+0000 7f41b0dff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41ac1610f0 con 0x7f41ac0ad690 2023-03-29T10:57:57.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.226+0000 7f41b0dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac0c2320 con 0x7f41ac0c19c0 2023-03-29T10:57:57.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.227+0000 7f41aa59c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f41ac160d20 0x7f41ac06cb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41676/0 (socket says 172.21.15.111:41676) 2023-03-29T10:57:57.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.227+0000 7f41aa59c700 1 -- 172.21.15.111:0/2863496565 learned_addr learned my addr 172.21.15.111:0/2863496565 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:57.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.228+0000 7f41aad9d700 1 --2- 172.21.15.111:0/2863496565 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f41ac0c19c0 0x7f41ac0c1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.228+0000 7f41a9d9b700 1 --2- 172.21.15.111:0/2863496565 >> v2:172.21.15.111:3300/0 conn(0x7f41ac0ad690 0x7f41ac0adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.228+0000 7f41a959a700 1 -- 172.21.15.111:0/2863496565 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459968004 0 0) 0x7f41ac0cd1f0 con 0x7f41ac160d20 2023-03-29T10:57:57.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.228+0000 7f41a959a700 1 -- 172.21.15.111:0/2863496565 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41940035b0 con 0x7f41ac160d20 2023-03-29T10:57:57.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.229+0000 7f41a959a700 1 -- 172.21.15.111:0/2863496565 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 97081737 0 0) 0x7f41940035b0 con 0x7f41ac160d20 2023-03-29T10:57:57.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.229+0000 7f41a959a700 1 -- 172.21.15.111:0/2863496565 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f41ac0c19c0 msgr2=0x7f41ac0c1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:57.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.229+0000 7f41a959a700 1 --2- 172.21.15.111:0/2863496565 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f41ac0c19c0 0x7f41ac0c1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.229+0000 7f41a959a700 1 -- 172.21.15.111:0/2863496565 >> v2:172.21.15.111:3300/0 conn(0x7f41ac0ad690 msgr2=0x7f41ac0adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:57.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.229+0000 7f41a959a700 1 --2- 172.21.15.111:0/2863496565 >> v2:172.21.15.111:3300/0 conn(0x7f41ac0ad690 0x7f41ac0adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.229+0000 7f41a959a700 1 -- 172.21.15.111:0/2863496565 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac0cc670 con 0x7f41ac160d20 2023-03-29T10:57:57.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.229+0000 7f41a959a700 1 -- 172.21.15.111:0/2863496565 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f419c001670 con 0x7f41ac160d20 2023-03-29T10:57:57.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.229+0000 7f41a959a700 1 -- 172.21.15.111:0/2863496565 >> v1:172.21.15.111:6789/0 conn(0x7f41ac160d20 legacy=0x7f41ac06cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.229+0000 7f41a959a700 1 --2- 172.21.15.111:0/2863496565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4194003a00 0x7f4194003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.229+0000 7f41a959a700 1 --2- 172.21.15.111:0/2863496565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4194004850 0x7f4194004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.229+0000 7f41a959a700 1 --2- 172.21.15.111:0/2863496565 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4194005200 0x7f419400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.229+0000 7f41a959a700 1 -- 172.21.15.111:0/2863496565 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac1610f0 con 0x7f4194004850 2023-03-29T10:57:57.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.229+0000 7f41a959a700 1 -- 172.21.15.111:0/2863496565 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac0c2320 con 0x7f4194003a00 2023-03-29T10:57:57.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.229+0000 7f41a959a700 1 -- 172.21.15.111:0/2863496565 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419400c0b0 con 0x7f4194005200 2023-03-29T10:57:57.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.229+0000 7f41a9d9b700 1 --2- 172.21.15.111:0/2863496565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4194004850 0x7f4194004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.230+0000 7f41aad9d700 1 --2- 172.21.15.111:0/2863496565 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4194005200 0x7f419400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.231+0000 7f41aad9d700 1 -- 172.21.15.111:0/2863496565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4194003a00 msgr2=0x7f4194003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:57.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.231+0000 7f41aad9d700 1 --2- 172.21.15.111:0/2863496565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4194003a00 0x7f4194003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.231+0000 7f41aad9d700 1 -- 172.21.15.111:0/2863496565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4194004850 msgr2=0x7f4194004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:57.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.231+0000 7f41aad9d700 1 --2- 172.21.15.111:0/2863496565 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4194004850 0x7f4194004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.231+0000 7f41aad9d700 1 -- 172.21.15.111:0/2863496565 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac0cc670 con 0x7f4194005200 2023-03-29T10:57:57.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.232+0000 7f41aad9d700 1 --2- 172.21.15.111:0/2863496565 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4194005200 0x7f419400ba10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f41ac0c1d90 tx=0x7f4194005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d41d81ae69f0af0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.233+0000 7f41a959a700 1 -- 172.21.15.111:0/2863496565 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a400eb50 con 0x7f4194005200 2023-03-29T10:57:57.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.233+0000 7f41a959a700 1 -- 172.21.15.111:0/2863496565 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a400f040 con 0x7f4194005200 2023-03-29T10:57:57.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.233+0000 7f41a959a700 1 -- 172.21.15.111:0/2863496565 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a400eed0 con 0x7f4194005200 2023-03-29T10:57:57.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.234+0000 7f41b0dff700 1 -- 172.21.15.111:0/2863496565 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4194005200 msgr2=0x7f419400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.234+0000 7f41b0dff700 1 --2- 172.21.15.111:0/2863496565 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4194005200 0x7f419400ba10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f41ac0c1d90 tx=0x7f4194005880 comp rx=0 tx=0).stop 2023-03-29T10:57:57.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.234+0000 7f41aad9d700 1 -- 172.21.15.111:0/2863496565 reap_dead start 2023-03-29T10:57:57.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.234+0000 7f41b0dff700 1 -- 172.21.15.111:0/2863496565 shutdown_connections 2023-03-29T10:57:57.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.234+0000 7f41b0dff700 1 -- 172.21.15.111:0/2863496565 >> 172.21.15.111:0/2863496565 conn(0x7f41ac0cf720 msgr2=0x7f41ac0cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:57.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.234+0000 7f41b0dff700 1 -- 172.21.15.111:0/2863496565 shutdown_connections 2023-03-29T10:57:57.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.234+0000 7f41b0dff700 1 -- 172.21.15.111:0/2863496565 wait complete. 2023-03-29T10:57:57.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.237+0000 7f41b0dff700 1 Processor -- start 2023-03-29T10:57:57.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.237+0000 7f41b0dff700 1 -- start start 2023-03-29T10:57:57.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.239+0000 7f41b0dff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4194005200 0x7f41ac151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.239+0000 7f41b0dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f41ac1622c0 0x7f41ac162730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.239+0000 7f41b0dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41ac0c2540 0x7f41ac0c29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.239+0000 7f41b0dff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac162c70 con 0x7f4194005200 2023-03-29T10:57:57.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.239+0000 7f41b0dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac0c32b0 con 0x7f41ac0c2540 2023-03-29T10:57:57.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.239+0000 7f41b0dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac150d10 con 0x7f41ac1622c0 2023-03-29T10:57:57.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.242+0000 7f41a9d9b700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f41ac1622c0 0x7f41ac162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.242+0000 7f41aad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41ac0c2540 0x7f41ac0c29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.243+0000 7f41a9d9b700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f41ac1622c0 0x7f41ac162730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:50828/0 (socket says 172.21.15.111:50828) 2023-03-29T10:57:57.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.243+0000 7f41a9d9b700 1 -- 172.21.15.111:0/669926111 learned_addr learned my addr 172.21.15.111:0/669926111 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:57.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.243+0000 7f41aa59c700 1 --2- 172.21.15.111:0/669926111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4194005200 0x7f41ac151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.244 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.244+0000 7f41aa59c700 1 -- 172.21.15.111:0/669926111 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f41ac1622c0 msgr2=0x7f41ac162730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.244 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.244+0000 7f41aa59c700 1 --2- 172.21.15.111:0/669926111 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f41ac1622c0 0x7f41ac162730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.244 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.244+0000 7f41aa59c700 1 -- 172.21.15.111:0/669926111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41ac0c2540 msgr2=0x7f41ac0c29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.244 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.244+0000 7f41aa59c700 1 --2- 172.21.15.111:0/669926111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41ac0c2540 0x7f41ac0c29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.244 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.245+0000 7f41aa59c700 1 -- 172.21.15.111:0/669926111 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac0ccc30 con 0x7f4194005200 2023-03-29T10:57:57.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.245+0000 7f41a9d9b700 1 --2- 172.21.15.111:0/669926111 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f41ac1622c0 0x7f41ac162730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:57:57.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.245+0000 7f41aa59c700 1 --2- 172.21.15.111:0/669926111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4194005200 0x7f41ac151ad0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f419c00caf0 tx=0x7f419c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.247 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.247+0000 7f419affd700 1 -- 172.21.15.111:0/669926111 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c004a80 con 0x7f4194005200 2023-03-29T10:57:57.247 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.247+0000 7f419affd700 1 -- 172.21.15.111:0/669926111 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419c004be0 con 0x7f4194005200 2023-03-29T10:57:57.247 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.247+0000 7f419affd700 1 -- 172.21.15.111:0/669926111 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c0070d0 con 0x7f4194005200 2023-03-29T10:57:57.247 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.247+0000 7f41b0dff700 1 -- 172.21.15.111:0/669926111 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ac0cc950 con 0x7f4194005200 2023-03-29T10:57:57.247 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.247+0000 7f41b0dff700 1 -- 172.21.15.111:0/669926111 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ac0ccf10 con 0x7f4194005200 2023-03-29T10:57:57.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.250+0000 7fc5d31ff700 1 Processor -- start 2023-03-29T10:57:57.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.251+0000 7fc5d31ff700 1 -- start start 2023-03-29T10:57:57.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.253+0000 7fc5d31ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc5cc0cacf0 0x7fc5cc0add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.253+0000 7fc5d31ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5cc06a3e0 0x7fc5cc06a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.253+0000 7fc5d31ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5cc158d10 con 0x7fc5cc15cdb0 2023-03-29T10:57:57.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.253+0000 7fc5d31ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5cc0c97d0 con 0x7fc5cc0cacf0 2023-03-29T10:57:57.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.253+0000 7fc5d31ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0cd5b0 con 0x7fc5cc06a3e0 2023-03-29T10:57:57.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.254+0000 7fc5d0f9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc5cc0cacf0 0x7fc5cc0add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.254+0000 7fc5d0f9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc5cc0cacf0 0x7fc5cc0add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:40982/0 (socket says 172.21.15.111:40982) 2023-03-29T10:57:57.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.252+0000 7f419affd700 1 -- 172.21.15.111:0/669926111 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f419c007230 con 0x7f4194005200 2023-03-29T10:57:57.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.254+0000 7fc5d0f9b700 1 -- 172.21.15.111:0/615834835 learned_addr learned my addr 172.21.15.111:0/615834835 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:57.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.254+0000 7fc5d179c700 1 --2- 172.21.15.111:0/615834835 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5cc06a3e0 0x7fc5cc06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.253+0000 7f419affd700 1 --2- 172.21.15.111:0/669926111 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f418c069eb0 0x7f418c06c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.253+0000 7f419affd700 1 -- 172.21.15.111:0/669926111 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f419c087a80 con 0x7f4194005200 2023-03-29T10:57:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.254+0000 7f41b0dff700 1 -- 172.21.15.111:0/669926111 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41900052e0 con 0x7f4194005200 2023-03-29T10:57:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.255+0000 7fc5d179c700 1 -- 172.21.15.111:0/615834835 >> v2:172.21.15.119:3300/0 conn(0x7fc5cc0cacf0 msgr2=0x7fc5cc0add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.255+0000 7fc5d179c700 1 --2- 172.21.15.111:0/615834835 >> v2:172.21.15.119:3300/0 conn(0x7fc5cc0cacf0 0x7fc5cc0add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.255+0000 7fc5d179c700 1 -- 172.21.15.111:0/615834835 >> v1:172.21.15.119:6789/0 conn(0x7fc5cc15cdb0 legacy=0x7fc5cc0c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:57.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.255+0000 7fc5d179c700 1 -- 172.21.15.111:0/615834835 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc06ad80 con 0x7fc5cc06a3e0 2023-03-29T10:57:57.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.255+0000 7fc5d179c700 1 --2- 172.21.15.111:0/615834835 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5cc06a3e0 0x7fc5cc06a840 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009840 tx=0x7fc5bc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb8744694d1f4514 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.256+0000 7fc5cb7fe700 1 -- 172.21.15.111:0/615834835 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc004230 con 0x7fc5cc06a3e0 2023-03-29T10:57:57.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.256+0000 7fc5cb7fe700 1 -- 172.21.15.111:0/615834835 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5bc004390 con 0x7fc5cc06a3e0 2023-03-29T10:57:57.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.256+0000 7fc5cb7fe700 1 -- 172.21.15.111:0/615834835 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc0036a0 con 0x7fc5cc06a3e0 2023-03-29T10:57:57.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.257+0000 7fc5d31ff700 1 -- 172.21.15.111:0/615834835 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5cc06a3e0 msgr2=0x7fc5cc06a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.257+0000 7fc5d31ff700 1 --2- 172.21.15.111:0/615834835 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5cc06a3e0 0x7fc5cc06a840 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009840 tx=0x7fc5bc009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:57.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.257+0000 7fc5d31ff700 1 -- 172.21.15.111:0/615834835 shutdown_connections 2023-03-29T10:57:57.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.257+0000 7fc5d31ff700 1 --2- 172.21.15.111:0/615834835 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5cc06a3e0 0x7fc5cc06a840 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.257+0000 7fc5d31ff700 1 --2- 172.21.15.111:0/615834835 >> v2:172.21.15.119:3300/0 conn(0x7fc5cc0cacf0 0x7fc5cc0add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.257+0000 7fc5d31ff700 1 -- 172.21.15.111:0/615834835 >> 172.21.15.111:0/615834835 conn(0x7fc5cc0cf6e0 msgr2=0x7fc5cc0cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:57.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.255+0000 7f41a9d9b700 1 --2- 172.21.15.111:0/669926111 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f418c069eb0 0x7f418c06c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.256+0000 7f41a9d9b700 1 --2- 172.21.15.111:0/669926111 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f418c069eb0 0x7f418c06c360 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f41ac163f20 tx=0x7f41a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.261+0000 7fc5d31ff700 1 -- 172.21.15.111:0/615834835 shutdown_connections 2023-03-29T10:57:57.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.261+0000 7f419affd700 1 -- 172.21.15.111:0/669926111 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f419c018020 con 0x7f4194005200 2023-03-29T10:57:57.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.261+0000 7fc5d31ff700 1 -- 172.21.15.111:0/615834835 wait complete. 2023-03-29T10:57:57.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.264+0000 7fc5d31ff700 1 Processor -- start 2023-03-29T10:57:57.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.264+0000 7fc5d31ff700 1 -- start start 2023-03-29T10:57:57.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.266+0000 7fc5d31ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5cc06a3e0 0x7fc5cc0b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.266+0000 7fc5d31ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5cc0cacf0 0x7fc5cc0b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.266+0000 7fc5d31ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5cc15cdb0 0x7fc5cc150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.266+0000 7fc5d31ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0b0640 con 0x7fc5cc0cacf0 2023-03-29T10:57:57.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.266+0000 7fc5d31ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0b18c0 con 0x7fc5cc15cdb0 2023-03-29T10:57:57.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.266+0000 7fc5d31ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5cc0b39d0 con 0x7fc5cc06a3e0 2023-03-29T10:57:57.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.267+0000 7fc5d0f9b700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5cc06a3e0 0x7fc5cc0b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.267+0000 7fc5d179c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5cc15cdb0 0x7fc5cc150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.267+0000 7fc5d179c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5cc15cdb0 0x7fc5cc150310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:40994/0 (socket says 172.21.15.111:40994) 2023-03-29T10:57:57.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.267+0000 7fc5d179c700 1 -- 172.21.15.111:0/351102429 learned_addr learned my addr 172.21.15.111:0/351102429 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:57.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.267+0000 7f299a9ff700 1 Processor -- start 2023-03-29T10:57:57.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.268+0000 7fc5d179c700 1 -- 172.21.15.111:0/351102429 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5cc06a3e0 msgr2=0x7fc5cc0b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.268+0000 7fc5d179c700 1 --2- 172.21.15.111:0/351102429 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5cc06a3e0 0x7fc5cc0b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.268+0000 7fc5d179c700 1 -- 172.21.15.111:0/351102429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5cc0cacf0 msgr2=0x7fc5cc0b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.268+0000 7fc5d179c700 1 --2- 172.21.15.111:0/351102429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5cc0cacf0 0x7fc5cc0b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.268+0000 7fc5d179c700 1 -- 172.21.15.111:0/351102429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc0094a0 con 0x7fc5cc15cdb0 2023-03-29T10:57:57.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.269+0000 7fc5d179c700 1 --2- 172.21.15.111:0/351102429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5cc15cdb0 0x7fc5cc150310 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc06b8d0 tx=0x7fc5bc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.269+0000 7fc5c97fa700 1 -- 172.21.15.111:0/351102429 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc003e90 con 0x7fc5cc15cdb0 2023-03-29T10:57:57.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.269+0000 7fc5c97fa700 1 -- 172.21.15.111:0/351102429 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5bc0045c0 con 0x7fc5cc15cdb0 2023-03-29T10:57:57.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.269+0000 7fc5c97fa700 1 -- 172.21.15.111:0/351102429 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc011850 con 0x7fc5cc15cdb0 2023-03-29T10:57:57.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.269+0000 7fc5d31ff700 1 -- 172.21.15.111:0/351102429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5cc0cd2d0 con 0x7fc5cc15cdb0 2023-03-29T10:57:57.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.269+0000 7fc5d31ff700 1 -- 172.21.15.111:0/351102429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5cc0ccff0 con 0x7fc5cc15cdb0 2023-03-29T10:57:57.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.267+0000 7f299a9ff700 1 -- start start 2023-03-29T10:57:57.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.269+0000 7f299a9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2994160d20 0x7f299406cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.269+0000 7f299a9ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f29940ad690 0x7f29940adb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.269+0000 7f299a9ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29940c19c0 0x7f29940c1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.269+0000 7f299a9ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29941610f0 con 0x7f29940ad690 2023-03-29T10:57:57.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.269+0000 7f299a9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29940cd1f0 con 0x7f2994160d20 2023-03-29T10:57:57.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.269+0000 7f299a9ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29940c2320 con 0x7f29940c19c0 2023-03-29T10:57:57.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.272+0000 7f2993fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2994160d20 0x7f299406cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.272+0000 7f2998f9c700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29940c19c0 0x7f29940c1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.272+0000 7f2998f9c700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29940c19c0 0x7f29940c1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:50838/0 (socket says 172.21.15.111:50838) 2023-03-29T10:57:57.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.272+0000 7f2998f9c700 1 -- 172.21.15.111:0/3122462195 learned_addr learned my addr 172.21.15.111:0/3122462195 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:57.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.273+0000 7fc5d31ff700 1 -- 172.21.15.111:0/351102429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ac0052e0 con 0x7fc5cc15cdb0 2023-03-29T10:57:57.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.273+0000 7f29937fe700 1 --2- 172.21.15.111:0/3122462195 >> v2:172.21.15.111:3300/0 conn(0x7f29940ad690 0x7f29940adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.273+0000 7f2998f9c700 1 -- 172.21.15.111:0/3122462195 >> v2:172.21.15.119:3300/0 conn(0x7f2994160d20 msgr2=0x7f299406cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:57.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.273+0000 7f2998f9c700 1 --2- 172.21.15.111:0/3122462195 >> v2:172.21.15.119:3300/0 conn(0x7f2994160d20 0x7f299406cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.273+0000 7f2998f9c700 1 -- 172.21.15.111:0/3122462195 >> v2:172.21.15.111:3300/0 conn(0x7f29940ad690 msgr2=0x7f29940adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:57.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.273+0000 7f2998f9c700 1 --2- 172.21.15.111:0/3122462195 >> v2:172.21.15.111:3300/0 conn(0x7f29940ad690 0x7f29940adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.273+0000 7f2998f9c700 1 -- 172.21.15.111:0/3122462195 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29940cc670 con 0x7f29940c19c0 2023-03-29T10:57:57.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.273+0000 7f29937fe700 1 --2- 172.21.15.111:0/3122462195 >> v2:172.21.15.111:3300/0 conn(0x7f29940ad690 0x7f29940adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:57.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.274+0000 7f2998f9c700 1 --2- 172.21.15.111:0/3122462195 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29940c19c0 0x7f29940c1de0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f29940705a0 tx=0x7f2988009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd7570d31cb209b server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.274+0000 7f2992ffd700 1 -- 172.21.15.111:0/3122462195 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2988011830 con 0x7f29940c19c0 2023-03-29T10:57:57.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.274+0000 7f2992ffd700 1 -- 172.21.15.111:0/3122462195 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2988011990 con 0x7f29940c19c0 2023-03-29T10:57:57.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.275+0000 7f2992ffd700 1 -- 172.21.15.111:0/3122462195 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2988011d00 con 0x7f29940c19c0 2023-03-29T10:57:57.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.275+0000 7f299a9ff700 1 -- 172.21.15.111:0/3122462195 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29940c19c0 msgr2=0x7f29940c1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.275+0000 7f299a9ff700 1 --2- 172.21.15.111:0/3122462195 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29940c19c0 0x7f29940c1de0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f29940705a0 tx=0x7f2988009bd0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.276+0000 7f299a9ff700 1 -- 172.21.15.111:0/3122462195 shutdown_connections 2023-03-29T10:57:57.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.276+0000 7f299a9ff700 1 --2- 172.21.15.111:0/3122462195 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29940c19c0 0x7f29940c1de0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.276+0000 7f299a9ff700 1 --2- 172.21.15.111:0/3122462195 >> v2:172.21.15.119:3300/0 conn(0x7f2994160d20 0x7f299406cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.276+0000 7f299a9ff700 1 --2- 172.21.15.111:0/3122462195 >> v2:172.21.15.111:3300/0 conn(0x7f29940ad690 0x7f29940adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.276+0000 7f299a9ff700 1 -- 172.21.15.111:0/3122462195 >> 172.21.15.111:0/3122462195 conn(0x7f29940cf720 msgr2=0x7f29940cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:57.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.277+0000 7fc5c97fa700 1 -- 172.21.15.111:0/351102429 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fc5bc018050 con 0x7fc5cc15cdb0 2023-03-29T10:57:57.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.278+0000 7fc5c97fa700 1 --2- 172.21.15.111:0/351102429 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc5b8069eb0 0x7fc5b806c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.278+0000 7fc5c97fa700 1 -- 172.21.15.111:0/351102429 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc5bc088f70 con 0x7fc5cc15cdb0 2023-03-29T10:57:57.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.277+0000 7f299a9ff700 1 -- 172.21.15.111:0/3122462195 shutdown_connections 2023-03-29T10:57:57.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.279+0000 7fc5d0f9b700 1 --2- 172.21.15.111:0/351102429 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc5b8069eb0 0x7fc5b806c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.277+0000 7f299a9ff700 1 -- 172.21.15.111:0/3122462195 wait complete. 2023-03-29T10:57:57.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.281+0000 7f299a9ff700 1 Processor -- start 2023-03-29T10:57:57.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.282+0000 7f299a9ff700 1 -- start start 2023-03-29T10:57:57.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.283+0000 7f299a9ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29940ad690 0x7f29940c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.283+0000 7f299a9ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29940c19c0 0x7f2994150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.283+0000 7f299a9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2994160d20 0x7f29942027d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.283+0000 7f299a9ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29941515a0 con 0x7f29940ad690 2023-03-29T10:57:57.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.283+0000 7f299a9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2994150ab0 con 0x7f2994160d20 2023-03-29T10:57:57.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.283+0000 7f299a9ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2994150d90 con 0x7f29940c19c0 2023-03-29T10:57:57.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.284+0000 7f2993fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29940ad690 0x7f29940c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.285+0000 7f2993fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29940ad690 0x7f29940c3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38422/0 (socket says 172.21.15.111:38422) 2023-03-29T10:57:57.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.285+0000 7f2993fff700 1 -- 172.21.15.111:0/1873388254 learned_addr learned my addr 172.21.15.111:0/1873388254 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:57.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.286+0000 7f2993fff700 1 -- 172.21.15.111:0/1873388254 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29940c19c0 msgr2=0x7f2994150060 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-03-29T10:57:57.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.286+0000 7f2993fff700 1 --2- 172.21.15.111:0/1873388254 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29940c19c0 0x7f2994150060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.286+0000 7f2993fff700 1 -- 172.21.15.111:0/1873388254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2994160d20 msgr2=0x7f29942027d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-03-29T10:57:57.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.286+0000 7f2993fff700 1 --2- 172.21.15.111:0/1873388254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2994160d20 0x7f29942027d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.286+0000 7f2993fff700 1 -- 172.21.15.111:0/1873388254 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2984009430 con 0x7f29940ad690 2023-03-29T10:57:57.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.286+0000 7f2993fff700 1 --2- 172.21.15.111:0/1873388254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29940ad690 0x7f29940c3cb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f298c00ccd0 tx=0x7f298c00a4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.287+0000 7f2990ff9700 1 -- 172.21.15.111:0/1873388254 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c00ad30 con 0x7f29940ad690 2023-03-29T10:57:57.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.287+0000 7f2990ff9700 1 -- 172.21.15.111:0/1873388254 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298c00ae90 con 0x7f29940ad690 2023-03-29T10:57:57.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.287+0000 7f2990ff9700 1 -- 172.21.15.111:0/1873388254 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c003c40 con 0x7f29940ad690 2023-03-29T10:57:57.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.287+0000 7f299a9ff700 1 -- 172.21.15.111:0/1873388254 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2988009550 con 0x7f29940ad690 2023-03-29T10:57:57.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.287+0000 7f299a9ff700 1 -- 172.21.15.111:0/1873388254 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29940ccc30 con 0x7f29940ad690 2023-03-29T10:57:57.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.281+0000 7fc5c97fa700 1 -- 172.21.15.111:0/351102429 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fc5bc0545a0 con 0x7fc5cc15cdb0 2023-03-29T10:57:57.295 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.293+0000 7fc5d0f9b700 1 --2- 172.21.15.111:0/351102429 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc5b8069eb0 0x7fc5b806c360 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4007250 tx=0x7fc5c4007660 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.289+0000 7f299a9ff700 1 -- 172.21.15.111:0/1873388254 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29940cc950 con 0x7f29940ad690 2023-03-29T10:57:57.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.305+0000 7f2990ff9700 1 -- 172.21.15.111:0/1873388254 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f298c004020 con 0x7f29940ad690 2023-03-29T10:57:57.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.306+0000 7f2990ff9700 1 --2- 172.21.15.111:0/1873388254 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2980069eb0 0x7f298006c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.306+0000 7f2990ff9700 1 -- 172.21.15.111:0/1873388254 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f298c088910 con 0x7f29940ad690 2023-03-29T10:57:57.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.307+0000 7f2990ff9700 1 -- 172.21.15.111:0/1873388254 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f298c0b4200 con 0x7f29940ad690 2023-03-29T10:57:57.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.308+0000 7f29937fe700 1 --2- 172.21.15.111:0/1873388254 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2980069eb0 0x7f298006c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.309+0000 7f29937fe700 1 --2- 172.21.15.111:0/1873388254 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2980069eb0 0x7f298006c360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f29940adf60 tx=0x7f29840091c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.337+0000 7f762d3ff700 1 Processor -- start 2023-03-29T10:57:57.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.338+0000 7f762d3ff700 1 -- start start 2023-03-29T10:57:57.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.340+0000 7f762d3ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f76280cacf0 0x7f76280add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.340+0000 7f762d3ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7628158d10 con 0x7f762815cdb0 2023-03-29T10:57:57.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.340+0000 7f762d3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76280cd5b0 con 0x7f762806a3e0 2023-03-29T10:57:57.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.340+0000 7f762d3ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76280c97d0 con 0x7f76280cacf0 2023-03-29T10:57:57.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.341+0000 7f76277fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f762806a3e0 0x7f762806a7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.111:35188/0 (socket says 172.21.15.111:35188) 2023-03-29T10:57:57.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.341+0000 7f76277fe700 1 -- 172.21.15.111:0/3454644052 learned_addr learned my addr 172.21.15.111:0/3454644052 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:57.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.342+0000 7f7626ffd700 1 --2- 172.21.15.111:0/3454644052 >> v2:172.21.15.119:3300/0 conn(0x7f76280cacf0 0x7f76280add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.343+0000 7f7625ffb700 1 -- 172.21.15.111:0/3454644052 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 612292201 0 0) 0x7f76280cd5b0 con 0x7f762806a3e0 2023-03-29T10:57:57.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.343+0000 7f7625ffb700 1 -- 172.21.15.111:0/3454644052 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76100035b0 con 0x7f762806a3e0 2023-03-29T10:57:57.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.343+0000 7f7626ffd700 1 -- 172.21.15.111:0/3454644052 >> v1:172.21.15.119:6789/0 conn(0x7f762806a3e0 legacy=0x7f762806a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.343+0000 7f7626ffd700 1 -- 172.21.15.111:0/3454644052 >> v1:172.21.15.111:6789/0 conn(0x7f762815cdb0 legacy=0x7f76280c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.343+0000 7f7626ffd700 1 -- 172.21.15.111:0/3454644052 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76280ccff0 con 0x7f76280cacf0 2023-03-29T10:57:57.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.344+0000 7f7626ffd700 1 --2- 172.21.15.111:0/3454644052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76280cacf0 0x7f76280add70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7618009840 tx=0x7f7618009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74e1a2873f6611ce server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.344+0000 7f7625ffb700 1 -- 172.21.15.111:0/3454644052 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7618004230 con 0x7f76280cacf0 2023-03-29T10:57:57.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.344+0000 7f7625ffb700 1 -- 172.21.15.111:0/3454644052 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7618004390 con 0x7f76280cacf0 2023-03-29T10:57:57.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.344+0000 7f7625ffb700 1 -- 172.21.15.111:0/3454644052 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76180036c0 con 0x7f76280cacf0 2023-03-29T10:57:57.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.345+0000 7f762d3ff700 1 -- 172.21.15.111:0/3454644052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76280cacf0 msgr2=0x7f76280add70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.345+0000 7f762d3ff700 1 --2- 172.21.15.111:0/3454644052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76280cacf0 0x7f76280add70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7618009840 tx=0x7f7618009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:57.346 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.346+0000 7f762d3ff700 1 -- 172.21.15.111:0/3454644052 shutdown_connections 2023-03-29T10:57:57.346 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.346+0000 7f762d3ff700 1 --2- 172.21.15.111:0/3454644052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76280cacf0 0x7f76280add70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.346 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.346+0000 7f762d3ff700 1 -- 172.21.15.111:0/3454644052 >> 172.21.15.111:0/3454644052 conn(0x7f76280cf6e0 msgr2=0x7f76280cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:57.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.350+0000 7fe5387ff700 1 Processor -- start 2023-03-29T10:57:57.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.350+0000 7fe5387ff700 1 -- start start 2023-03-29T10:57:57.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.352+0000 7fe5387ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe530160fd0 0x7fe53006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.353+0000 7fe5387ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe53006c6c0 0x7fe530070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.353+0000 7fe5387ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe530070af0 con 0x7fe5300ad930 2023-03-29T10:57:57.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.353+0000 7fe5387ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe530161530 con 0x7fe53006c6c0 2023-03-29T10:57:57.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.353+0000 7fe5387ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe530161810 con 0x7fe530160fd0 2023-03-29T10:57:57.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.347+0000 7f762d3ff700 1 -- 172.21.15.111:0/3454644052 shutdown_connections 2023-03-29T10:57:57.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.347+0000 7f762d3ff700 1 -- 172.21.15.111:0/3454644052 wait complete. 2023-03-29T10:57:57.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.349+0000 7f762d3ff700 1 Processor -- start 2023-03-29T10:57:57.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.349+0000 7f762d3ff700 1 -- start start 2023-03-29T10:57:57.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.355+0000 7fe536d9c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe53006c6c0 0x7fe530070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.355+0000 7fe536d9c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe53006c6c0 0x7fe530070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38444/0 (socket says 172.21.15.111:38444) 2023-03-29T10:57:57.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.355+0000 7fe536d9c700 1 -- 172.21.15.111:0/816153851 learned_addr learned my addr 172.21.15.111:0/816153851 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:57.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.356+0000 7fe536d9c700 1 -- 172.21.15.111:0/816153851 >> v2:172.21.15.119:3300/0 conn(0x7fe530160fd0 msgr2=0x7fe53006a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:57.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.356+0000 7fe536d9c700 1 --2- 172.21.15.111:0/816153851 >> v2:172.21.15.119:3300/0 conn(0x7fe530160fd0 0x7fe53006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.356+0000 7fe536d9c700 1 -- 172.21.15.111:0/816153851 >> v1:172.21.15.111:6789/0 conn(0x7fe5300ad930 legacy=0x7fe5300add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.356+0000 7fe536d9c700 1 -- 172.21.15.111:0/816153851 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe530161af0 con 0x7fe53006c6c0 2023-03-29T10:57:57.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.351+0000 7f762d3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f762806a3e0 0x7f76280af070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.351+0000 7f762d3ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76280cacf0 0x7f76280c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.351+0000 7f762d3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f762815cdb0 0x7f76280aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.351+0000 7f762d3ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7628152910 con 0x7f76280cacf0 2023-03-29T10:57:57.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.351+0000 7f762d3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76280af5b0 con 0x7f762815cdb0 2023-03-29T10:57:57.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.351+0000 7f762d3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76280b0c30 con 0x7f762806a3e0 2023-03-29T10:57:57.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.351+0000 7f7626ffd700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f762806a3e0 0x7f76280af070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.351+0000 7f7626ffd700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f762806a3e0 0x7f76280af070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:50854/0 (socket says 172.21.15.111:50854) 2023-03-29T10:57:57.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.351+0000 7f7626ffd700 1 -- 172.21.15.111:0/3535907615 learned_addr learned my addr 172.21.15.111:0/3535907615 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:57.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.352+0000 7f76277fe700 1 --2- 172.21.15.111:0/3535907615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f762815cdb0 0x7f76280aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.352+0000 7f76267fc700 1 --2- 172.21.15.111:0/3535907615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76280cacf0 0x7f76280c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.353+0000 7f7626ffd700 1 -- 172.21.15.111:0/3535907615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f762815cdb0 msgr2=0x7f76280aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.353+0000 7f7626ffd700 1 --2- 172.21.15.111:0/3535907615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f762815cdb0 0x7f76280aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.353+0000 7f7626ffd700 1 -- 172.21.15.111:0/3535907615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76280cacf0 msgr2=0x7f76280c47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.353+0000 7f7626ffd700 1 --2- 172.21.15.111:0/3535907615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76280cacf0 0x7f76280c47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.353+0000 7f7626ffd700 1 -- 172.21.15.111:0/3535907615 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76180094a0 con 0x7f762806a3e0 2023-03-29T10:57:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.353+0000 7f7626ffd700 1 --2- 172.21.15.111:0/3535907615 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f762806a3e0 0x7f76280af070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f76180090b0 tx=0x7f76180047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.354+0000 7f760f7fe700 1 -- 172.21.15.111:0/3535907615 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761801d040 con 0x7f762806a3e0 2023-03-29T10:57:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.354+0000 7f760f7fe700 1 -- 172.21.15.111:0/3535907615 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f761800c840 con 0x7f762806a3e0 2023-03-29T10:57:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.354+0000 7f760f7fe700 1 -- 172.21.15.111:0/3535907615 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f761800cbf0 con 0x7f762806a3e0 2023-03-29T10:57:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.355+0000 7f762d3ff700 1 -- 172.21.15.111:0/3535907615 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f762806cc40 con 0x7f762806a3e0 2023-03-29T10:57:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.355+0000 7f762d3ff700 1 -- 172.21.15.111:0/3535907615 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76280cd2d0 con 0x7f762806a3e0 2023-03-29T10:57:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.357+0000 7fe536d9c700 1 --2- 172.21.15.111:0/816153851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe53006c6c0 0x7fe530070ff0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe5300708e0 tx=0x7fe52800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0b7b6ae77cde2 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.357+0000 7fe535599700 1 -- 172.21.15.111:0/816153851 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52800e070 con 0x7fe53006c6c0 2023-03-29T10:57:57.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.357+0000 7fe535599700 1 -- 172.21.15.111:0/816153851 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52800b9c0 con 0x7fe53006c6c0 2023-03-29T10:57:57.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.357+0000 7fe535599700 1 -- 172.21.15.111:0/816153851 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52800bd70 con 0x7fe53006c6c0 2023-03-29T10:57:57.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.358+0000 7f0ca31ff700 1 Processor -- start 2023-03-29T10:57:57.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.358+0000 7fe5387ff700 1 -- 172.21.15.111:0/816153851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe53006c6c0 msgr2=0x7fe530070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.358+0000 7fe5387ff700 1 --2- 172.21.15.111:0/816153851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe53006c6c0 0x7fe530070ff0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe5300708e0 tx=0x7fe52800b330 comp rx=0 tx=0).stop 2023-03-29T10:57:57.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.359+0000 7fe5387ff700 1 -- 172.21.15.111:0/816153851 shutdown_connections 2023-03-29T10:57:57.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.359+0000 7fe5387ff700 1 --2- 172.21.15.111:0/816153851 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe53006c6c0 0x7fe530070ff0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.359+0000 7fe5387ff700 1 --2- 172.21.15.111:0/816153851 >> v2:172.21.15.119:3300/0 conn(0x7fe530160fd0 0x7fe53006a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.359+0000 7fe5387ff700 1 -- 172.21.15.111:0/816153851 >> 172.21.15.111:0/816153851 conn(0x7fe5300cf6e0 msgr2=0x7fe5300cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:57.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.359+0000 7f760d7fa700 1 -- 172.21.15.111:0/3535907615 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76000052e0 con 0x7f762806a3e0 2023-03-29T10:57:57.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.358+0000 7f0ca31ff700 1 -- start start 2023-03-29T10:57:57.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.360+0000 7f0ca31ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0c9c0cfbf0 0x7f0c9c1608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.360+0000 7f0ca31ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c9c0d0020 con 0x7f0c9c0d1100 2023-03-29T10:57:57.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.360+0000 7f0ca31ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c9c0cdc60 con 0x7f0c9c0b87a0 2023-03-29T10:57:57.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.360+0000 7f0ca31ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c9c0d02d0 con 0x7f0c9c0cfbf0 2023-03-29T10:57:57.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.361+0000 7f0c9bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0c9c0cfbf0 0x7f0c9c1608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.361+0000 7f0c9bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0c9c0cfbf0 0x7f0c9c1608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:41050/0 (socket says 172.21.15.111:41050) 2023-03-29T10:57:57.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.361+0000 7f0c9bfff700 1 -- 172.21.15.111:0/3630721033 learned_addr learned my addr 172.21.15.111:0/3630721033 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:57.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.362+0000 7f0c9bfff700 1 -- 172.21.15.111:0/3630721033 >> v1:172.21.15.119:6789/0 conn(0x7f0c9c0b87a0 legacy=0x7f0c9c1601e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-03-29T10:57:57.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.362+0000 7f0c9bfff700 1 -- 172.21.15.111:0/3630721033 >> v1:172.21.15.111:6789/0 conn(0x7f0c9c0d1100 legacy=0x7f0c9c0d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:57.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.362+0000 7f0c9bfff700 1 -- 172.21.15.111:0/3630721033 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9c0cd6a0 con 0x7f0c9c0cfbf0 2023-03-29T10:57:57.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.362+0000 7f0c9bfff700 1 --2- 172.21.15.111:0/3630721033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c9c0cfbf0 0x7f0c9c1608f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0c900098f0 tx=0x7f0c90009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0375bf5e4128b1e server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.363+0000 7f0c9b7fe700 1 -- 172.21.15.111:0/3630721033 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90004230 con 0x7f0c9c0cfbf0 2023-03-29T10:57:57.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.363+0000 7f0c9b7fe700 1 -- 172.21.15.111:0/3630721033 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c90004390 con 0x7f0c9c0cfbf0 2023-03-29T10:57:57.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.363+0000 7f0c9b7fe700 1 -- 172.21.15.111:0/3630721033 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c900036a0 con 0x7f0c9c0cfbf0 2023-03-29T10:57:57.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.361+0000 7f760f7fe700 1 -- 172.21.15.111:0/3535907615 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f761800cd50 con 0x7f762806a3e0 2023-03-29T10:57:57.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.363+0000 7f760f7fe700 1 --2- 172.21.15.111:0/3535907615 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f76140adec0 0x7f76140b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.363+0000 7f760f7fe700 1 -- 172.21.15.111:0/3535907615 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f761808ae50 con 0x7f762806a3e0 2023-03-29T10:57:57.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.363+0000 7f76267fc700 1 --2- 172.21.15.111:0/3535907615 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f76140adec0 0x7f76140b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.363+0000 7f2b01bff700 1 Processor -- start 2023-03-29T10:57:57.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.364+0000 7f2b01bff700 1 -- start start 2023-03-29T10:57:57.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.364+0000 7f0ca31ff700 1 -- 172.21.15.111:0/3630721033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c9c0cfbf0 msgr2=0x7f0c9c1608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.364+0000 7f0ca31ff700 1 --2- 172.21.15.111:0/3630721033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c9c0cfbf0 0x7f0c9c1608f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0c900098f0 tx=0x7f0c90009c00 comp rx=0 tx=0).stop 2023-03-29T10:57:57.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.364+0000 7f0ca31ff700 1 -- 172.21.15.111:0/3630721033 shutdown_connections 2023-03-29T10:57:57.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.364+0000 7f0ca31ff700 1 --2- 172.21.15.111:0/3630721033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c9c0cfbf0 0x7f0c9c1608f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.364+0000 7f0ca31ff700 1 -- 172.21.15.111:0/3630721033 >> 172.21.15.111:0/3630721033 conn(0x7f0c9c071b10 msgr2=0x7f0c9c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:57.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.364+0000 7f0ca31ff700 1 -- 172.21.15.111:0/3630721033 shutdown_connections 2023-03-29T10:57:57.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.365+0000 7fe5387ff700 1 -- 172.21.15.111:0/816153851 shutdown_connections 2023-03-29T10:57:57.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.365+0000 7f76267fc700 1 --2- 172.21.15.111:0/3535907615 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f76140adec0 0x7f76140b0370 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f76280c9270 tx=0x7f761c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.365+0000 7f2b01bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2afc06a3e0 0x7f2afc06a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.365+0000 7f2b01bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2afc0cc6b0 0x7f2afc070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.365+0000 7f2afb7fe700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2afc06a3e0 0x7f2afc06a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.365+0000 7f2afb7fe700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2afc06a3e0 0x7f2afc06a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:50858/0 (socket says 172.21.15.111:50858) 2023-03-29T10:57:57.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.366+0000 7f2b01bff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2afc0ccb20 0x7f2afc071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.366+0000 7f2afb7fe700 1 -- 172.21.15.111:0/460427081 learned_addr learned my addr 172.21.15.111:0/460427081 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:57.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.366+0000 7f2b01bff700 1 -- 172.21.15.111:0/460427081 --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2afc06acf0 con 0x7f2afc0ccb20 2023-03-29T10:57:57.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.366+0000 7f2b01bff700 1 -- 172.21.15.111:0/460427081 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2afc06e5e0 con 0x7f2afc0cc6b0 2023-03-29T10:57:57.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.366+0000 7f2b01bff700 1 -- 172.21.15.111:0/460427081 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc0cd0e0 con 0x7f2afc06a3e0 2023-03-29T10:57:57.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.366+0000 7f2afb7fe700 1 -- 172.21.15.111:0/460427081 >> v2:172.21.15.119:3300/0 conn(0x7f2afc0cc6b0 msgr2=0x7f2afc070af0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-03-29T10:57:57.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.366+0000 7f2afb7fe700 1 --2- 172.21.15.111:0/460427081 >> v2:172.21.15.119:3300/0 conn(0x7f2afc0cc6b0 0x7f2afc070af0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.366+0000 7f2afb7fe700 1 -- 172.21.15.111:0/460427081 >> v2:172.21.15.111:3300/0 conn(0x7f2afc0ccb20 msgr2=0x7f2afc071030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-29T10:57:57.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.366+0000 7f2afb7fe700 1 --2- 172.21.15.111:0/460427081 >> v2:172.21.15.111:3300/0 conn(0x7f2afc0ccb20 0x7f2afc071030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.366+0000 7f2afb7fe700 1 -- 172.21.15.111:0/460427081 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc160240 con 0x7f2afc06a3e0 2023-03-29T10:57:57.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.366+0000 7f2afb7fe700 1 --2- 172.21.15.111:0/460427081 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2afc06a3e0 0x7f2afc06a7b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2afc06b800 tx=0x7f2aec014ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eb0e8a766325ea5 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.365+0000 7f0ca31ff700 1 -- 172.21.15.111:0/3630721033 wait complete. 2023-03-29T10:57:57.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.367+0000 7f0ca31ff700 1 Processor -- start 2023-03-29T10:57:57.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.366+0000 7f760f7fe700 1 -- 172.21.15.111:0/3535907615 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f7618056560 con 0x7f762806a3e0 2023-03-29T10:57:57.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.368+0000 7f2afaffd700 1 -- 172.21.15.111:0/460427081 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec013af0 con 0x7f2afc06a3e0 2023-03-29T10:57:57.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.368+0000 7f2afaffd700 1 -- 172.21.15.111:0/460427081 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec013c50 con 0x7f2afc06a3e0 2023-03-29T10:57:57.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.368+0000 7f2afaffd700 1 -- 172.21.15.111:0/460427081 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec013fe0 con 0x7f2afc06a3e0 2023-03-29T10:57:57.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.369+0000 7f2b01bff700 1 -- 172.21.15.111:0/460427081 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2afc06a3e0 msgr2=0x7f2afc06a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.369+0000 7f2b01bff700 1 --2- 172.21.15.111:0/460427081 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2afc06a3e0 0x7f2afc06a7b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2afc06b800 tx=0x7f2aec014ce0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.367+0000 7f0ca31ff700 1 -- start start 2023-03-29T10:57:57.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.369+0000 7f0ca31ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c9c0b87a0 0x7f0c9c0ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.369+0000 7f0ca31ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c9c0cfbf0 0x7f0c9c0c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.369+0000 7f0ca31ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c9c0d1100 0x7f0c9c0c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.369+0000 7f0ca31ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c0cd060 con 0x7f0c9c0b87a0 2023-03-29T10:57:57.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.369+0000 7f0ca31ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c15ccf0 con 0x7f0c9c0d1100 2023-03-29T10:57:57.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.369+0000 7f0ca31ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c15cfd0 con 0x7f0c9c0cfbf0 2023-03-29T10:57:57.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.370+0000 7f0c9bfff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c9c0cfbf0 0x7f0c9c0c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.370+0000 7f0c9bfff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c9c0cfbf0 0x7f0c9c0c7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:50870/0 (socket says 172.21.15.111:50870) 2023-03-29T10:57:57.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.370+0000 7f0c9bfff700 1 -- 172.21.15.111:0/1971019728 learned_addr learned my addr 172.21.15.111:0/1971019728 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:57.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.370+0000 7f0ca0f9b700 1 --2- 172.21.15.111:0/1971019728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c9c0b87a0 0x7f0c9c0ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.371+0000 7f0c9bfff700 1 -- 172.21.15.111:0/1971019728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c9c0d1100 msgr2=0x7f0c9c0c7f70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-03-29T10:57:57.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.371+0000 7f0c9bfff700 1 --2- 172.21.15.111:0/1971019728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c9c0d1100 0x7f0c9c0c7f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.371+0000 7f0c9bfff700 1 -- 172.21.15.111:0/1971019728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c9c0b87a0 msgr2=0x7f0c9c0ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.371+0000 7f0c9bfff700 1 --2- 172.21.15.111:0/1971019728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c9c0b87a0 0x7f0c9c0ccb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.371+0000 7f0c9bfff700 1 -- 172.21.15.111:0/1971019728 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c90009550 con 0x7f0c9c0cfbf0 2023-03-29T10:57:57.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.371+0000 7f0c9bfff700 1 --2- 172.21.15.111:0/1971019728 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c9c0cfbf0 0x7f0c9c0c7a30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c072760 tx=0x7f0c90004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.368+0000 7fe5387ff700 1 -- 172.21.15.111:0/816153851 wait complete. 2023-03-29T10:57:57.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.371+0000 7fe5387ff700 1 Processor -- start 2023-03-29T10:57:57.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.372+0000 7f2b01bff700 1 -- 172.21.15.111:0/460427081 shutdown_connections 2023-03-29T10:57:57.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.372+0000 7f2b01bff700 1 --2- 172.21.15.111:0/460427081 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2afc06a3e0 0x7f2afc06a7b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.372+0000 7f2b01bff700 1 --2- 172.21.15.111:0/460427081 >> v2:172.21.15.111:3300/0 conn(0x7f2afc0ccb20 0x7f2afc071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.372+0000 7f2b01bff700 1 --2- 172.21.15.111:0/460427081 >> v2:172.21.15.119:3300/0 conn(0x7f2afc0cc6b0 0x7f2afc070af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.372+0000 7f2b01bff700 1 -- 172.21.15.111:0/460427081 >> 172.21.15.111:0/460427081 conn(0x7f2afc0cf6e0 msgr2=0x7f2afc0cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:57.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.372+0000 7f0c997fa700 1 -- 172.21.15.111:0/1971019728 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90011830 con 0x7f0c9c0cfbf0 2023-03-29T10:57:57.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.372+0000 7f0c997fa700 1 -- 172.21.15.111:0/1971019728 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c90011990 con 0x7f0c9c0cfbf0 2023-03-29T10:57:57.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.372+0000 7f0c997fa700 1 -- 172.21.15.111:0/1971019728 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c90011d00 con 0x7f0c9c0cfbf0 2023-03-29T10:57:57.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.372+0000 7f0ca31ff700 1 -- 172.21.15.111:0/1971019728 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9c160e30 con 0x7f0c9c0cfbf0 2023-03-29T10:57:57.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.372+0000 7f0ca31ff700 1 -- 172.21.15.111:0/1971019728 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9c0cd980 con 0x7f0c9c0cfbf0 2023-03-29T10:57:57.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.373+0000 7f2b01bff700 1 -- 172.21.15.111:0/460427081 shutdown_connections 2023-03-29T10:57:57.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.373+0000 7f2b01bff700 1 -- 172.21.15.111:0/460427081 wait complete. 2023-03-29T10:57:57.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.371+0000 7fe5387ff700 1 -- start start 2023-03-29T10:57:57.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.373+0000 7fe5387ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe53006c6c0 0x7fe53014f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.373+0000 7fe5387ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe5300ad930 0x7fe5301501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.373+0000 7fe5387ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5300c4770 0x7fe5300c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.373+0000 7fe5387ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5300c1670 con 0x7fe5300c4770 2023-03-29T10:57:57.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.373+0000 7fe5387ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe530147b20 con 0x7fe53006c6c0 2023-03-29T10:57:57.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.373+0000 7fe5387ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53014f1a0 con 0x7fe5300ad930 2023-03-29T10:57:57.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.374+0000 7fe536d9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5300c4770 0x7fe5300c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.374+0000 7f7801fff700 1 Processor -- start 2023-03-29T10:57:57.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.374+0000 7fe536d9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5300c4770 0x7fe5300c4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38454/0 (socket says 172.21.15.111:38454) 2023-03-29T10:57:57.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.374+0000 7fe536d9c700 1 -- 172.21.15.111:0/1648291488 learned_addr learned my addr 172.21.15.111:0/1648291488 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:57.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.374+0000 7f7801fff700 1 -- start start 2023-03-29T10:57:57.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.375+0000 7fe536d9c700 1 -- 172.21.15.111:0/1648291488 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe5300ad930 msgr2=0x7fe5301501b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-03-29T10:57:57.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.375+0000 7fe536d9c700 1 --2- 172.21.15.111:0/1648291488 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe5300ad930 0x7fe5301501b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.376+0000 7fe536d9c700 1 -- 172.21.15.111:0/1648291488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe53006c6c0 msgr2=0x7fe53014f630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-03-29T10:57:57.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.376+0000 7fe536d9c700 1 --2- 172.21.15.111:0/1648291488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe53006c6c0 0x7fe53014f630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.376+0000 7fe536d9c700 1 -- 172.21.15.111:0/1648291488 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe528009b30 con 0x7fe5300c4770 2023-03-29T10:57:57.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.376+0000 7f2b01bff700 1 Processor -- start 2023-03-29T10:57:57.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.376+0000 7fe536d9c700 1 --2- 172.21.15.111:0/1648291488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5300c4770 0x7fe5300c4bf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe528003f50 tx=0x7fe528004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.376+0000 7f2b01bff700 1 -- start start 2023-03-29T10:57:57.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.379+0000 7f7801fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f77fc158aa0 0x7f77fc0add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.379+0000 7f7801fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77fc0cbc20 0x7f77fc15cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.379+0000 7f7801fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f77fc0cacf0 0x7f77fc0c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.379+0000 7f7801fff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77fc15d2f0 con 0x7f77fc158aa0 2023-03-29T10:57:57.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.379+0000 7f7801fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77fc0ccff0 con 0x7f77fc0cacf0 2023-03-29T10:57:57.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.379+0000 7f7801fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc0cd2d0 con 0x7f77fc0cbc20 2023-03-29T10:57:57.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.377+0000 7f0ca31ff700 1 -- 172.21.15.111:0/1971019728 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c9c160fc0 con 0x7f0c9c0cfbf0 2023-03-29T10:57:57.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.378+0000 7fe526ffd700 1 -- 172.21.15.111:0/1648291488 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe528004390 con 0x7fe5300c4770 2023-03-29T10:57:57.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.378+0000 7fe526ffd700 1 -- 172.21.15.111:0/1648291488 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5280044f0 con 0x7fe5300c4770 2023-03-29T10:57:57.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.378+0000 7fe526ffd700 1 -- 172.21.15.111:0/1648291488 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5280048a0 con 0x7fe5300c4770 2023-03-29T10:57:57.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.380+0000 7fe5387ff700 1 -- 172.21.15.111:0/1648291488 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53006ace0 con 0x7fe5300c4770 2023-03-29T10:57:57.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.380+0000 7fe5387ff700 1 -- 172.21.15.111:0/1648291488 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe530161dd0 con 0x7fe5300c4770 2023-03-29T10:57:57.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.378+0000 7f2b01bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2afc06a3e0 0x7f2afc1f99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.378+0000 7f2b01bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2afc0cc6b0 0x7f2afc0b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.378+0000 7f2b01bff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2afc0ccb20 0x7f2afc1ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.378+0000 7f2b01bff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc0b18c0 con 0x7f2afc0ccb20 2023-03-29T10:57:57.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.378+0000 7f2b01bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc200520 con 0x7f2afc0cc6b0 2023-03-29T10:57:57.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.378+0000 7f2b01bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc200800 con 0x7f2afc06a3e0 2023-03-29T10:57:57.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.379+0000 7f2af2ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2afc0cc6b0 0x7f2afc0b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.380+0000 7f2afb7fe700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2afc06a3e0 0x7f2afc1f99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.380+0000 7f2afb7fe700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2afc06a3e0 0x7f2afc1f99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:50880/0 (socket says 172.21.15.111:50880) 2023-03-29T10:57:57.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.380+0000 7f2afb7fe700 1 -- 172.21.15.111:0/2808584674 learned_addr learned my addr 172.21.15.111:0/2808584674 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:57.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.380+0000 7f77fbfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f77fc0cacf0 0x7f77fc0c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.380+0000 7f77fbfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f77fc0cacf0 0x7f77fc0c90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:41092/0 (socket says 172.21.15.111:41092) 2023-03-29T10:57:57.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.380+0000 7f77fbfff700 1 -- 172.21.15.111:0/3714837730 learned_addr learned my addr 172.21.15.111:0/3714837730 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:57.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.380+0000 7f0c997fa700 1 -- 172.21.15.111:0/1971019728 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f0c90022070 con 0x7f0c9c0cfbf0 2023-03-29T10:57:57.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.380+0000 7f2afbfff700 1 --2- 172.21.15.111:0/2808584674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2afc0ccb20 0x7f2afc1ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.381+0000 7f2afb7fe700 1 -- 172.21.15.111:0/2808584674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2afc0cc6b0 msgr2=0x7f2afc0b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.381+0000 7f2afb7fe700 1 --2- 172.21.15.111:0/2808584674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2afc0cc6b0 0x7f2afc0b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.381+0000 7f2afb7fe700 1 -- 172.21.15.111:0/2808584674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2afc0ccb20 msgr2=0x7f2afc1ff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.381+0000 7f2afb7fe700 1 --2- 172.21.15.111:0/2808584674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2afc0ccb20 0x7f2afc1ff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.381+0000 7f2afb7fe700 1 -- 172.21.15.111:0/2808584674 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec014660 con 0x7f2afc06a3e0 2023-03-29T10:57:57.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.381+0000 7f77fbfff700 1 -- 172.21.15.111:0/3714837730 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77fc0cbc20 msgr2=0x7f77fc15cdb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-29T10:57:57.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.381+0000 7f77fbfff700 1 --2- 172.21.15.111:0/3714837730 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77fc0cbc20 0x7f77fc15cdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.381+0000 7f77fbfff700 1 -- 172.21.15.111:0/3714837730 >> v2:172.21.15.111:3300/0 conn(0x7f77fc158aa0 msgr2=0x7f77fc0add70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-29T10:57:57.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.381+0000 7f77fbfff700 1 --2- 172.21.15.111:0/3714837730 >> v2:172.21.15.111:3300/0 conn(0x7f77fc158aa0 0x7f77fc0add70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.381+0000 7f77fbfff700 1 -- 172.21.15.111:0/3714837730 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77fc06cc40 con 0x7f77fc0cacf0 2023-03-29T10:57:57.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.381+0000 7f77fbfff700 1 --2- 172.21.15.111:0/3714837730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77fc0cacf0 0x7f77fc0c90c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0709f0 tx=0x7f77f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a00e65bda23c12a5 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.381+0000 7f0c997fa700 1 --2- 172.21.15.111:0/1971019728 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0c88069eb0 0x7f0c8806c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.382+0000 7f0c997fa700 1 -- 172.21.15.111:0/1971019728 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0c9008a260 con 0x7f0c9c0cfbf0 2023-03-29T10:57:57.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.382+0000 7f77fa7fc700 1 -- 172.21.15.111:0/3714837730 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f400e070 con 0x7f77fc0cacf0 2023-03-29T10:57:57.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.382+0000 7f77fa7fc700 1 -- 172.21.15.111:0/3714837730 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f400b9c0 con 0x7f77fc0cacf0 2023-03-29T10:57:57.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.382+0000 7f77fa7fc700 1 -- 172.21.15.111:0/3714837730 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f400bd70 con 0x7f77fc0cacf0 2023-03-29T10:57:57.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.382+0000 7f2afb7fe700 1 --2- 172.21.15.111:0/2808584674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2afc06a3e0 0x7f2afc1f99d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2aec016010 tx=0x7f2aec004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.383+0000 7f0ca0f9b700 1 --2- 172.21.15.111:0/1971019728 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0c88069eb0 0x7f0c8806c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.383+0000 7f7801fff700 1 -- 172.21.15.111:0/3714837730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77fc0cacf0 msgr2=0x7f77fc0c90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.383+0000 7f7801fff700 1 --2- 172.21.15.111:0/3714837730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77fc0cacf0 0x7f77fc0c90c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0709f0 tx=0x7f77f400b330 comp rx=0 tx=0).stop 2023-03-29T10:57:57.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.383+0000 7f7801fff700 1 -- 172.21.15.111:0/3714837730 shutdown_connections 2023-03-29T10:57:57.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.383+0000 7f7801fff700 1 --2- 172.21.15.111:0/3714837730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77fc0cacf0 0x7f77fc0c90c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.383+0000 7f7801fff700 1 --2- 172.21.15.111:0/3714837730 >> v2:172.21.15.111:3300/0 conn(0x7f77fc158aa0 0x7f77fc0add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.383+0000 7f7801fff700 1 --2- 172.21.15.111:0/3714837730 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77fc0cbc20 0x7f77fc15cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.383+0000 7f7801fff700 1 -- 172.21.15.111:0/3714837730 >> 172.21.15.111:0/3714837730 conn(0x7f77fc0cf6e0 msgr2=0x7f77fc0cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:57.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.383+0000 7f2af8ff9700 1 -- 172.21.15.111:0/2808584674 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec004450 con 0x7f2afc06a3e0 2023-03-29T10:57:57.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.384+0000 7f2af8ff9700 1 -- 172.21.15.111:0/2808584674 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec0045b0 con 0x7f2afc06a3e0 2023-03-29T10:57:57.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.384+0000 7f2af8ff9700 1 -- 172.21.15.111:0/2808584674 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec004940 con 0x7f2afc06a3e0 2023-03-29T10:57:57.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.384+0000 7f2b01bff700 1 -- 172.21.15.111:0/2808584674 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2afc160800 con 0x7f2afc06a3e0 2023-03-29T10:57:57.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.384+0000 7f0c997fa700 1 -- 172.21.15.111:0/1971019728 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f0c90055890 con 0x7f0c9c0cfbf0 2023-03-29T10:57:57.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.384+0000 7f2b01bff700 1 -- 172.21.15.111:0/2808584674 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2afc160520 con 0x7f2afc06a3e0 2023-03-29T10:57:57.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.384+0000 7f7801fff700 1 -- 172.21.15.111:0/3714837730 shutdown_connections 2023-03-29T10:57:57.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.385+0000 7f7801fff700 1 -- 172.21.15.111:0/3714837730 wait complete. 2023-03-29T10:57:57.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.387+0000 7f7801fff700 1 Processor -- start 2023-03-29T10:57:57.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.388+0000 7f7801fff700 1 -- start start 2023-03-29T10:57:57.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.385+0000 7fe5387ff700 1 -- 172.21.15.111:0/1648291488 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5300cd150 con 0x7fe5300c4770 2023-03-29T10:57:57.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.386+0000 7fe526ffd700 1 -- 172.21.15.111:0/1648291488 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fe52801c440 con 0x7fe5300c4770 2023-03-29T10:57:57.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.388+0000 7fe526ffd700 1 --2- 172.21.15.111:0/1648291488 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe514069eb0 0x7fe51406c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.388+0000 7fe526ffd700 1 -- 172.21.15.111:0/1648291488 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe528089970 con 0x7fe5300c4770 2023-03-29T10:57:57.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.389+0000 7f2af8ff9700 1 -- 172.21.15.111:0/2808584674 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f2aec009030 con 0x7f2afc06a3e0 2023-03-29T10:57:57.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.390+0000 7f2af1ffb700 1 -- 172.21.15.111:0/2808584674 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2afc160ae0 con 0x7f2afc06a3e0 2023-03-29T10:57:57.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.390+0000 7f2af8ff9700 1 --2- 172.21.15.111:0/2808584674 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2adc069eb0 0x7f2adc06c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.391+0000 7f2af8ff9700 1 -- 172.21.15.111:0/2808584674 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2aec089d00 con 0x7f2afc06a3e0 2023-03-29T10:57:57.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.389+0000 7f7801fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77fc0cacf0 0x7f77fc0c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.389+0000 7f7801fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77fc158aa0 0x7f77fc0c3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.389+0000 7f7801fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77fc0b3e50 0x7f77fc14bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.389+0000 7f7801fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc0c5720 con 0x7f77fc0cacf0 2023-03-29T10:57:57.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.389+0000 7f7801fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc0c3c60 con 0x7f77fc158aa0 2023-03-29T10:57:57.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.389+0000 7f7801fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77fc0c3eb0 con 0x7f77fc0b3e50 2023-03-29T10:57:57.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.391+0000 7f77fbfff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77fc0b3e50 0x7f77fc14bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.391+0000 7f77fbfff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77fc0b3e50 0x7f77fc14bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:50886/0 (socket says 172.21.15.111:50886) 2023-03-29T10:57:57.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.391+0000 7f77fbfff700 1 -- 172.21.15.111:0/1170026446 learned_addr learned my addr 172.21.15.111:0/1170026446 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:57.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.392+0000 7f77faffd700 1 --2- 172.21.15.111:0/1170026446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77fc158aa0 0x7f77fc0c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.392+0000 7f77fb7fe700 1 --2- 172.21.15.111:0/1170026446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77fc0cacf0 0x7f77fc0c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.392+0000 7f77fbfff700 1 -- 172.21.15.111:0/1170026446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77fc158aa0 msgr2=0x7f77fc0c3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.392+0000 7f77fbfff700 1 --2- 172.21.15.111:0/1170026446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77fc158aa0 0x7f77fc0c3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.392+0000 7f77fbfff700 1 -- 172.21.15.111:0/1170026446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77fc0cacf0 msgr2=0x7f77fc0c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.392+0000 7f77fbfff700 1 --2- 172.21.15.111:0/1170026446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77fc0cacf0 0x7f77fc0c51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.392+0000 7f77fbfff700 1 -- 172.21.15.111:0/1170026446 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f4009b30 con 0x7f77fc0b3e50 2023-03-29T10:57:57.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.393+0000 7f2af2ffd700 1 --2- 172.21.15.111:0/2808584674 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2adc069eb0 0x7f2adc06c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.392+0000 7f77fbfff700 1 --2- 172.21.15.111:0/1170026446 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77fc0b3e50 0x7f77fc14bb60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f77f400b730 tx=0x7f77f4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.393+0000 7fe53659b700 1 --2- 172.21.15.111:0/1648291488 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe514069eb0 0x7fe51406c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.393+0000 7f0ca0f9b700 1 --2- 172.21.15.111:0/1971019728 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0c88069eb0 0x7f0c8806c360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009910 tx=0x7f0c8c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.394+0000 7fe53659b700 1 --2- 172.21.15.111:0/1648291488 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe514069eb0 0x7fe51406c360 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe52c000f60 tx=0x7fe52c007a40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.394+0000 7f2af2ffd700 1 --2- 172.21.15.111:0/2808584674 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2adc069eb0 0x7f2adc06c360 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2afc15c8b0 tx=0x7f2ae4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.397+0000 7fe526ffd700 1 -- 172.21.15.111:0/1648291488 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fe528054f40 con 0x7fe5300c4770 2023-03-29T10:57:57.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.396+0000 7f2af8ff9700 1 -- 172.21.15.111:0/2808584674 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f2aec055330 con 0x7f2afc06a3e0 2023-03-29T10:57:57.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.396+0000 7f77e3fff700 1 -- 172.21.15.111:0/1170026446 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f4004590 con 0x7f77fc0b3e50 2023-03-29T10:57:57.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.396+0000 7f77e3fff700 1 -- 172.21.15.111:0/1170026446 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f40046f0 con 0x7f77fc0b3e50 2023-03-29T10:57:57.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.396+0000 7f77e3fff700 1 -- 172.21.15.111:0/1170026446 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f401c440 con 0x7f77fc0b3e50 2023-03-29T10:57:57.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.396+0000 7f7801fff700 1 -- 172.21.15.111:0/1170026446 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77fc0cb190 con 0x7f77fc0b3e50 2023-03-29T10:57:57.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.396+0000 7f7801fff700 1 -- 172.21.15.111:0/1170026446 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77fc06cf20 con 0x7f77fc0b3e50 2023-03-29T10:57:57.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.406+0000 7f7801fff700 1 -- 172.21.15.111:0/1170026446 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77dc0052e0 con 0x7f77fc0b3e50 2023-03-29T10:57:57.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.407+0000 7f77e3fff700 1 -- 172.21.15.111:0/1170026446 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f77f401c5a0 con 0x7f77fc0b3e50 2023-03-29T10:57:57.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.409+0000 7f77e3fff700 1 --2- 172.21.15.111:0/1170026446 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f77e8069eb0 0x7f77e806c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:57.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.409+0000 7f77e3fff700 1 -- 172.21.15.111:0/1170026446 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f77f4089c10 con 0x7f77fc0b3e50 2023-03-29T10:57:57.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.410+0000 7f77fb7fe700 1 --2- 172.21.15.111:0/1170026446 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f77e8069eb0 0x7f77e806c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:57.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.411+0000 7f77fb7fe700 1 --2- 172.21.15.111:0/1170026446 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f77e8069eb0 0x7f77e806c360 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009910 tx=0x7f77ec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:57.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.415+0000 7f77e3fff700 1 -- 172.21.15.111:0/1170026446 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f77f40551b0 con 0x7f77fc0b3e50 2023-03-29T10:57:57.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.707+0000 7f41b0dff700 1 -- 172.21.15.111:0/669926111 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4190005590 con 0x7f4194005200 2023-03-29T10:57:57.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.708+0000 7fc5d31ff700 1 -- 172.21.15.111:0/351102429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc5ac005860 con 0x7fc5cc15cdb0 2023-03-29T10:57:57.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.708+0000 7f419affd700 1 -- 172.21.15.111:0/669926111 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f419c052f90 con 0x7f4194005200 2023-03-29T10:57:57.708 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2023-03-29T10:57:57.712 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2023-03-29T10:57:57.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.709+0000 7fc5c97fa700 1 -- 172.21.15.111:0/351102429 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc5bc016020 con 0x7fc5cc15cdb0 2023-03-29T10:57:57.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.713+0000 7f41b0dff700 1 -- 172.21.15.111:0/669926111 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f418c069eb0 msgr2=0x7f418c06c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.713+0000 7f41b0dff700 1 --2- 172.21.15.111:0/669926111 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f418c069eb0 0x7f418c06c360 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f41ac163f20 tx=0x7f41a0008040 comp rx=0 tx=0).stop 2023-03-29T10:57:57.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.713+0000 7f41b0dff700 1 -- 172.21.15.111:0/669926111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4194005200 msgr2=0x7f41ac151ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.713+0000 7f41b0dff700 1 --2- 172.21.15.111:0/669926111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4194005200 0x7f41ac151ad0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f419c00caf0 tx=0x7f419c00ceb0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.713+0000 7f41b0dff700 1 -- 172.21.15.111:0/669926111 shutdown_connections 2023-03-29T10:57:57.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.713+0000 7f41b0dff700 1 --2- 172.21.15.111:0/669926111 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f418c069eb0 0x7f418c06c360 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.713+0000 7f41b0dff700 1 --2- 172.21.15.111:0/669926111 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4194005200 0x7f41ac151ad0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.713+0000 7f41b0dff700 1 --2- 172.21.15.111:0/669926111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41ac0c2540 0x7f41ac0c29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.713+0000 7f41b0dff700 1 --2- 172.21.15.111:0/669926111 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f41ac1622c0 0x7f41ac162730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.713+0000 7f41b0dff700 1 -- 172.21.15.111:0/669926111 >> 172.21.15.111:0/669926111 conn(0x7f41ac0cf720 msgr2=0x7f41ac163230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:57.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.714+0000 7fc5d31ff700 1 -- 172.21.15.111:0/351102429 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc5b8069eb0 msgr2=0x7fc5b806c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.714+0000 7fc5d31ff700 1 --2- 172.21.15.111:0/351102429 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc5b8069eb0 0x7fc5b806c360 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4007250 tx=0x7fc5c4007660 comp rx=0 tx=0).stop 2023-03-29T10:57:57.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.714+0000 7fc5d31ff700 1 -- 172.21.15.111:0/351102429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5cc15cdb0 msgr2=0x7fc5cc150310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.714+0000 7fc5d31ff700 1 --2- 172.21.15.111:0/351102429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5cc15cdb0 0x7fc5cc150310 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc06b8d0 tx=0x7fc5bc004750 comp rx=0 tx=0).stop 2023-03-29T10:57:57.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.714+0000 7f41b0dff700 1 -- 172.21.15.111:0/669926111 shutdown_connections 2023-03-29T10:57:57.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.714+0000 7f41b0dff700 1 -- 172.21.15.111:0/669926111 wait complete. 2023-03-29T10:57:57.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.715+0000 7fc5d31ff700 1 -- 172.21.15.111:0/351102429 shutdown_connections 2023-03-29T10:57:57.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.715+0000 7fc5d31ff700 1 --2- 172.21.15.111:0/351102429 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc5b8069eb0 0x7fc5b806c360 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4007250 tx=0x7fc5c4007660 comp rx=0 tx=0).stop 2023-03-29T10:57:57.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.715+0000 7fc5d31ff700 1 --2- 172.21.15.111:0/351102429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5cc15cdb0 0x7fc5cc150310 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.715+0000 7fc5d31ff700 1 --2- 172.21.15.111:0/351102429 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc5cc06a3e0 0x7fc5cc0b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.715+0000 7fc5d31ff700 1 --2- 172.21.15.111:0/351102429 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc5cc0cacf0 0x7fc5cc0b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.715+0000 7fc5d31ff700 1 -- 172.21.15.111:0/351102429 >> 172.21.15.111:0/351102429 conn(0x7fc5cc0cf6e0 msgr2=0x7fc5cc06d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:57.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.716+0000 7fc5d31ff700 1 -- 172.21.15.111:0/351102429 shutdown_connections 2023-03-29T10:57:57.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.716+0000 7fc5d31ff700 1 -- 172.21.15.111:0/351102429 wait complete. 2023-03-29T10:57:57.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.745+0000 7f299a9ff700 1 -- 172.21.15.111:0/1873388254 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f29940ccf10 con 0x7f29940ad690 2023-03-29T10:57:57.746 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.1 2023-03-29T10:57:57.747 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.0 2023-03-29T10:57:57.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.748+0000 7f2990ff9700 1 -- 172.21.15.111:0/1873388254 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f298c01d090 con 0x7f29940ad690 2023-03-29T10:57:57.747 INFO:teuthology.orchestra.run.smithi111.stdout:55834574857 2023-03-29T10:57:57.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.750+0000 7f299a9ff700 1 -- 172.21.15.111:0/1873388254 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2980069eb0 msgr2=0x7f298006c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.750+0000 7f299a9ff700 1 --2- 172.21.15.111:0/1873388254 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2980069eb0 0x7f298006c360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f29940adf60 tx=0x7f29840091c0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.750+0000 7f299a9ff700 1 -- 172.21.15.111:0/1873388254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29940ad690 msgr2=0x7f29940c3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.750+0000 7f299a9ff700 1 --2- 172.21.15.111:0/1873388254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29940ad690 0x7f29940c3cb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f298c00ccd0 tx=0x7f298c00a4d0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.751+0000 7f299a9ff700 1 -- 172.21.15.111:0/1873388254 shutdown_connections 2023-03-29T10:57:57.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.751+0000 7f299a9ff700 1 --2- 172.21.15.111:0/1873388254 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2980069eb0 0x7f298006c360 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.751+0000 7f299a9ff700 1 --2- 172.21.15.111:0/1873388254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f29940ad690 0x7f29940c3cb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.751+0000 7f299a9ff700 1 --2- 172.21.15.111:0/1873388254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2994160d20 0x7f29942027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.751+0000 7f299a9ff700 1 --2- 172.21.15.111:0/1873388254 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f29940c19c0 0x7f2994150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.751+0000 7f299a9ff700 1 -- 172.21.15.111:0/1873388254 >> 172.21.15.111:0/1873388254 conn(0x7f29940cf720 msgr2=0x7f2994163940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:57.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.751+0000 7f299a9ff700 1 -- 172.21.15.111:0/1873388254 shutdown_connections 2023-03-29T10:57:57.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.751+0000 7f299a9ff700 1 -- 172.21.15.111:0/1873388254 wait complete. 2023-03-29T10:57:57.769 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.6 2023-03-29T10:57:57.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.786+0000 7fe5387ff700 1 -- 172.21.15.111:0/1648291488 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe5300cc010 con 0x7fe5300c4770 2023-03-29T10:57:57.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.787+0000 7fe526ffd700 1 -- 172.21.15.111:0/1648291488 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe528058560 con 0x7fe5300c4770 2023-03-29T10:57:57.786 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2023-03-29T10:57:57.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.789+0000 7fe5387ff700 1 -- 172.21.15.111:0/1648291488 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe514069eb0 msgr2=0x7fe51406c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.789+0000 7fe5387ff700 1 --2- 172.21.15.111:0/1648291488 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe514069eb0 0x7fe51406c360 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe52c000f60 tx=0x7fe52c007a40 comp rx=0 tx=0).stop 2023-03-29T10:57:57.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.789+0000 7fe5387ff700 1 -- 172.21.15.111:0/1648291488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5300c4770 msgr2=0x7fe5300c4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.789+0000 7fe5387ff700 1 --2- 172.21.15.111:0/1648291488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5300c4770 0x7fe5300c4bf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe528003f50 tx=0x7fe528004ad0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.789+0000 7fe5387ff700 1 -- 172.21.15.111:0/1648291488 shutdown_connections 2023-03-29T10:57:57.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.789+0000 7fe5387ff700 1 --2- 172.21.15.111:0/1648291488 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe514069eb0 0x7fe51406c360 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.789+0000 7fe5387ff700 1 --2- 172.21.15.111:0/1648291488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe5300c4770 0x7fe5300c4bf0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.789+0000 7fe5387ff700 1 --2- 172.21.15.111:0/1648291488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe53006c6c0 0x7fe53014f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.789+0000 7fe5387ff700 1 --2- 172.21.15.111:0/1648291488 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe5300ad930 0x7fe5301501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.789+0000 7fe5387ff700 1 -- 172.21.15.111:0/1648291488 >> 172.21.15.111:0/1648291488 conn(0x7fe5300cf6e0 msgr2=0x7fe5300b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:57.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.789+0000 7fe5387ff700 1 -- 172.21.15.111:0/1648291488 shutdown_connections 2023-03-29T10:57:57.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.789+0000 7fe5387ff700 1 -- 172.21.15.111:0/1648291488 wait complete. 2023-03-29T10:57:57.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.801+0000 7f760d7fa700 1 -- 172.21.15.111:0/3535907615 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7600005860 con 0x7f762806a3e0 2023-03-29T10:57:57.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.801+0000 7f760f7fe700 1 -- 172.21.15.111:0/3535907615 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7618059b80 con 0x7f762806a3e0 2023-03-29T10:57:57.801 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2023-03-29T10:57:57.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.805+0000 7f760d7fa700 1 -- 172.21.15.111:0/3535907615 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f76140adec0 msgr2=0x7f76140b0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.805+0000 7f760d7fa700 1 --2- 172.21.15.111:0/3535907615 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f76140adec0 0x7f76140b0370 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f76280c9270 tx=0x7f761c009040 comp rx=0 tx=0).stop 2023-03-29T10:57:57.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.805+0000 7f760d7fa700 1 -- 172.21.15.111:0/3535907615 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f762806a3e0 msgr2=0x7f76280af070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.805+0000 7f760d7fa700 1 --2- 172.21.15.111:0/3535907615 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f762806a3e0 0x7f76280af070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f76180090b0 tx=0x7f76180047c0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.805+0000 7f760d7fa700 1 -- 172.21.15.111:0/3535907615 shutdown_connections 2023-03-29T10:57:57.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.805+0000 7f760d7fa700 1 --2- 172.21.15.111:0/3535907615 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f76140adec0 0x7f76140b0370 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.805+0000 7f760d7fa700 1 --2- 172.21.15.111:0/3535907615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f762815cdb0 0x7f76280aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.805+0000 7f760d7fa700 1 --2- 172.21.15.111:0/3535907615 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f762806a3e0 0x7f76280af070 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.805+0000 7f760d7fa700 1 --2- 172.21.15.111:0/3535907615 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f76280cacf0 0x7f76280c47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.805+0000 7f760d7fa700 1 -- 172.21.15.111:0/3535907615 >> 172.21.15.111:0/3535907615 conn(0x7f76280cf6e0 msgr2=0x7f762806a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:57.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.806+0000 7f760d7fa700 1 -- 172.21.15.111:0/3535907615 shutdown_connections 2023-03-29T10:57:57.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.806+0000 7f760d7fa700 1 -- 172.21.15.111:0/3535907615 wait complete. 2023-03-29T10:57:57.807 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.2 2023-03-29T10:57:57.831 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2023-03-29T10:57:57.832 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2023-03-29T10:57:57.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.832+0000 7f0ca31ff700 1 -- 172.21.15.111:0/1971019728 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f0c9c160fc0 con 0x7f0c9c0cfbf0 2023-03-29T10:57:57.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.832+0000 7f0c997fa700 1 -- 172.21.15.111:0/1971019728 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0c90011e60 con 0x7f0c9c0cfbf0 2023-03-29T10:57:57.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.836+0000 7f0ca31ff700 1 -- 172.21.15.111:0/1971019728 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0c88069eb0 msgr2=0x7f0c8806c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.836+0000 7f0ca31ff700 1 --2- 172.21.15.111:0/1971019728 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0c88069eb0 0x7f0c8806c360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009910 tx=0x7f0c8c008040 comp rx=0 tx=0).stop 2023-03-29T10:57:57.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.836+0000 7f0ca31ff700 1 -- 172.21.15.111:0/1971019728 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c9c0cfbf0 msgr2=0x7f0c9c0c7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.836+0000 7f0ca31ff700 1 --2- 172.21.15.111:0/1971019728 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c9c0cfbf0 0x7f0c9c0c7a30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c072760 tx=0x7f0c90004750 comp rx=0 tx=0).stop 2023-03-29T10:57:57.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.836+0000 7f0ca31ff700 1 -- 172.21.15.111:0/1971019728 shutdown_connections 2023-03-29T10:57:57.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.836+0000 7f0ca31ff700 1 --2- 172.21.15.111:0/1971019728 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0c88069eb0 0x7f0c8806c360 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.836+0000 7f0ca31ff700 1 --2- 172.21.15.111:0/1971019728 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c9c0b87a0 0x7f0c9c0ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.836+0000 7f0ca31ff700 1 --2- 172.21.15.111:0/1971019728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c9c0d1100 0x7f0c9c0c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.836+0000 7f0ca31ff700 1 --2- 172.21.15.111:0/1971019728 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c9c0cfbf0 0x7f0c9c0c7a30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.836+0000 7f0ca31ff700 1 -- 172.21.15.111:0/1971019728 >> 172.21.15.111:0/1971019728 conn(0x7f0c9c071b10 msgr2=0x7f0c9c0b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:57.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.836+0000 7f2af1ffb700 1 -- 172.21.15.111:0/2808584674 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f2afc200990 con 0x7f2afc06a3e0 2023-03-29T10:57:57.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.836+0000 7f0ca31ff700 1 -- 172.21.15.111:0/1971019728 shutdown_connections 2023-03-29T10:57:57.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.836+0000 7f0ca31ff700 1 -- 172.21.15.111:0/1971019728 wait complete. 2023-03-29T10:57:57.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.836+0000 7f2af8ff9700 1 -- 172.21.15.111:0/2808584674 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2aec058950 con 0x7f2afc06a3e0 2023-03-29T10:57:57.838 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2023-03-29T10:57:57.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.839+0000 7f2af1ffb700 1 -- 172.21.15.111:0/2808584674 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2adc069eb0 msgr2=0x7f2adc06c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.839+0000 7f2af1ffb700 1 --2- 172.21.15.111:0/2808584674 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2adc069eb0 0x7f2adc06c360 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2afc15c8b0 tx=0x7f2ae4008040 comp rx=0 tx=0).stop 2023-03-29T10:57:57.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.839+0000 7f2af1ffb700 1 -- 172.21.15.111:0/2808584674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2afc06a3e0 msgr2=0x7f2afc1f99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.839+0000 7f2af1ffb700 1 --2- 172.21.15.111:0/2808584674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2afc06a3e0 0x7f2afc1f99d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2aec016010 tx=0x7f2aec004ad0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.839+0000 7f2af1ffb700 1 -- 172.21.15.111:0/2808584674 shutdown_connections 2023-03-29T10:57:57.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.839+0000 7f2af1ffb700 1 --2- 172.21.15.111:0/2808584674 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2adc069eb0 0x7f2adc06c360 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.839+0000 7f2af1ffb700 1 --2- 172.21.15.111:0/2808584674 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2afc0ccb20 0x7f2afc1ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.839+0000 7f2af1ffb700 1 --2- 172.21.15.111:0/2808584674 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2afc06a3e0 0x7f2afc1f99d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.839+0000 7f2af1ffb700 1 --2- 172.21.15.111:0/2808584674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2afc0cc6b0 0x7f2afc0b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.839+0000 7f2af1ffb700 1 -- 172.21.15.111:0/2808584674 >> 172.21.15.111:0/2808584674 conn(0x7f2afc0cf6e0 msgr2=0x7f2afc15ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:57.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.839+0000 7f2af1ffb700 1 -- 172.21.15.111:0/2808584674 shutdown_connections 2023-03-29T10:57:57.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.839+0000 7f2af1ffb700 1 -- 172.21.15.111:0/2808584674 wait complete. 2023-03-29T10:57:57.856 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.5 2023-03-29T10:57:57.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.857+0000 7f7801fff700 1 -- 172.21.15.111:0/1170026446 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f77dc005590 con 0x7f77fc0b3e50 2023-03-29T10:57:57.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.858+0000 7f77e3fff700 1 -- 172.21.15.111:0/1170026446 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f77f40587d0 con 0x7f77fc0b3e50 2023-03-29T10:57:57.857 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2023-03-29T10:57:57.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.861+0000 7f7801fff700 1 -- 172.21.15.111:0/1170026446 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f77e8069eb0 msgr2=0x7f77e806c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.861+0000 7f7801fff700 1 --2- 172.21.15.111:0/1170026446 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f77e8069eb0 0x7f77e806c360 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009910 tx=0x7f77ec008040 comp rx=0 tx=0).stop 2023-03-29T10:57:57.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.861+0000 7f7801fff700 1 -- 172.21.15.111:0/1170026446 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77fc0b3e50 msgr2=0x7f77fc14bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:57.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.861+0000 7f7801fff700 1 --2- 172.21.15.111:0/1170026446 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77fc0b3e50 0x7f77fc14bb60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f77f400b730 tx=0x7f77f4004ad0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.861+0000 7f7801fff700 1 -- 172.21.15.111:0/1170026446 shutdown_connections 2023-03-29T10:57:57.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.861+0000 7f7801fff700 1 --2- 172.21.15.111:0/1170026446 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f77e8069eb0 0x7f77e806c360 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.861+0000 7f7801fff700 1 --2- 172.21.15.111:0/1170026446 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f77fc0cacf0 0x7f77fc0c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.861+0000 7f7801fff700 1 --2- 172.21.15.111:0/1170026446 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f77fc0b3e50 0x7f77fc14bb60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.861+0000 7f7801fff700 1 --2- 172.21.15.111:0/1170026446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77fc158aa0 0x7f77fc0c3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:57.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.861+0000 7f7801fff700 1 -- 172.21.15.111:0/1170026446 >> 172.21.15.111:0/1170026446 conn(0x7f77fc0cf6e0 msgr2=0x7f77fc06a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:57.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.861+0000 7f7801fff700 1 -- 172.21.15.111:0/1170026446 shutdown_connections 2023-03-29T10:57:57.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:57.861+0000 7f7801fff700 1 -- 172.21.15.111:0/1170026446 wait complete. 2023-03-29T10:57:57.863 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.4 2023-03-29T10:57:57.878 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.7 2023-03-29T10:57:58.747 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-03-29T10:57:58.748 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-03-29T10:57:58.770 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-03-29T10:57:58.808 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-03-29T10:57:58.832 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-03-29T10:57:58.857 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-03-29T10:57:58.864 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-03-29T10:57:58.879 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-03-29T10:57:58.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:58.991+0000 7f03a75ff700 1 Processor -- start 2023-03-29T10:57:58.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:58.992+0000 7f03a75ff700 1 -- start start 2023-03-29T10:57:58.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:58.994+0000 7f03a75ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f03a00ad930 0x7f03a00add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:58.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:58.994+0000 7f03a75ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f03a006c6c0 0x7f03a0070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:58.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:58.994+0000 7f03a75ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03a0070af0 con 0x7f03a0160fd0 2023-03-29T10:57:58.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:58.994+0000 7f03a75ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03a00ae240 con 0x7f03a006c6c0 2023-03-29T10:57:58.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:58.994+0000 7f03a75ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a0161530 con 0x7f03a00ad930 2023-03-29T10:57:58.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:58.997+0000 7f03a5b9c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f03a006c6c0 0x7f03a0070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:58.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:58.997+0000 7f03a5b9c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f03a006c6c0 0x7f03a0070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50252/0 (socket says 172.21.15.111:50252) 2023-03-29T10:57:58.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:58.997+0000 7f03a5b9c700 1 -- 172.21.15.111:0/1445992074 learned_addr learned my addr 172.21.15.111:0/1445992074 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:58.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:58.998+0000 7f03a5b9c700 1 -- 172.21.15.111:0/1445992074 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f03a00ad930 msgr2=0x7f03a00add00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-03-29T10:57:58.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:58.998+0000 7f03a5b9c700 1 --2- 172.21.15.111:0/1445992074 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f03a00ad930 0x7f03a00add00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:58.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:58.998+0000 7f03a5b9c700 1 -- 172.21.15.111:0/1445992074 >> v1:172.21.15.119:6789/0 conn(0x7f03a0160fd0 legacy=0x7f03a006a7a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-03-29T10:57:58.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:58.998+0000 7f03a5b9c700 1 -- 172.21.15.111:0/1445992074 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a0161810 con 0x7f03a006c6c0 2023-03-29T10:57:58.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:58.999+0000 7f03a5b9c700 1 --2- 172.21.15.111:0/1445992074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a006c6c0 0x7f03a0070ff0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f039c0098f0 tx=0x7f039c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d03f09f274b0f8 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.000+0000 7f0397fff700 1 -- 172.21.15.111:0/1445992074 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c004230 con 0x7f03a006c6c0 2023-03-29T10:57:59.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.000+0000 7f0397fff700 1 -- 172.21.15.111:0/1445992074 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c004390 con 0x7f03a006c6c0 2023-03-29T10:57:59.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.000+0000 7f0397fff700 1 -- 172.21.15.111:0/1445992074 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c0036a0 con 0x7f03a006c6c0 2023-03-29T10:57:59.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.001+0000 7f03a75ff700 1 -- 172.21.15.111:0/1445992074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a006c6c0 msgr2=0x7f03a0070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.001+0000 7f03a75ff700 1 --2- 172.21.15.111:0/1445992074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a006c6c0 0x7f03a0070ff0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f039c0098f0 tx=0x7f039c009c00 comp rx=0 tx=0).stop 2023-03-29T10:57:59.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.002+0000 7f03a75ff700 1 -- 172.21.15.111:0/1445992074 shutdown_connections 2023-03-29T10:57:59.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.002+0000 7f03a75ff700 1 --2- 172.21.15.111:0/1445992074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a006c6c0 0x7f03a0070ff0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.002+0000 7f03a75ff700 1 --2- 172.21.15.111:0/1445992074 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f03a00ad930 0x7f03a00add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.002+0000 7f03a75ff700 1 -- 172.21.15.111:0/1445992074 >> 172.21.15.111:0/1445992074 conn(0x7f03a00cf6e0 msgr2=0x7f03a00cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:59.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.002+0000 7f03a75ff700 1 -- 172.21.15.111:0/1445992074 shutdown_connections 2023-03-29T10:57:59.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.003+0000 7f03a75ff700 1 -- 172.21.15.111:0/1445992074 wait complete. 2023-03-29T10:57:59.009 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.003+0000 7f14be5ff700 1 Processor -- start 2023-03-29T10:57:59.009 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.005+0000 7f14be5ff700 1 -- start start 2023-03-29T10:57:59.009 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.007+0000 7f14be5ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f14b80ad930 0x7f14b80add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.009 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.007+0000 7f14be5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14b806c6c0 0x7f14b8070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.009 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.007+0000 7f14be5ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b8070af0 con 0x7f14b8160fd0 2023-03-29T10:57:59.009 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.007+0000 7f14be5ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b80ae240 con 0x7f14b80ad930 2023-03-29T10:57:59.009 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.007+0000 7f14be5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b8161530 con 0x7f14b806c6c0 2023-03-29T10:57:59.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.005+0000 7f03a75ff700 1 Processor -- start 2023-03-29T10:57:59.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.006+0000 7f03a75ff700 1 -- start start 2023-03-29T10:57:59.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.007+0000 7f03a75ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a006c6c0 0x7f03a01576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.007+0000 7f03a75ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f03a00ad930 0x7f03a015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.007+0000 7f03a75ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a0160fd0 0x7f03a015f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.007+0000 7f03a75ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a0157c10 con 0x7f03a006c6c0 2023-03-29T10:57:59.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.007+0000 7f03a75ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a015b3f0 con 0x7f03a0160fd0 2023-03-29T10:57:59.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.007+0000 7f03a75ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03a01fa460 con 0x7f03a00ad930 2023-03-29T10:57:59.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.008+0000 7f03a539b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a006c6c0 0x7f03a01576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.008+0000 7f03a4b9a700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f03a00ad930 0x7f03a015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.008+0000 7f03a539b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a006c6c0 0x7f03a01576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50264/0 (socket says 172.21.15.111:50264) 2023-03-29T10:57:59.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.008+0000 7f03a539b700 1 -- 172.21.15.111:0/880857222 learned_addr learned my addr 172.21.15.111:0/880857222 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:59.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.008+0000 7f03a5b9c700 1 --2- 172.21.15.111:0/880857222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a0160fd0 0x7f03a015f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.009+0000 7f03a539b700 1 -- 172.21.15.111:0/880857222 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f03a00ad930 msgr2=0x7f03a015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.009+0000 7f03a539b700 1 --2- 172.21.15.111:0/880857222 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f03a00ad930 0x7f03a015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.009+0000 7f03a539b700 1 -- 172.21.15.111:0/880857222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a0160fd0 msgr2=0x7f03a015f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.009+0000 7f03a539b700 1 --2- 172.21.15.111:0/880857222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a0160fd0 0x7f03a015f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.009+0000 7f03a539b700 1 -- 172.21.15.111:0/880857222 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c009550 con 0x7f03a006c6c0 2023-03-29T10:57:59.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.009+0000 7f03a539b700 1 --2- 172.21.15.111:0/880857222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a006c6c0 0x7f03a01576d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f039000b470 tx=0x7f039000b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.008+0000 7f14b77fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f14b8160fd0 0x7f14b806a7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36920/0 (socket says 172.21.15.111:36920) 2023-03-29T10:57:59.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.008+0000 7f14b77fe700 1 -- 172.21.15.111:0/606526213 learned_addr learned my addr 172.21.15.111:0/606526213 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:59.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.008+0000 7f14b7fff700 1 --2- 172.21.15.111:0/606526213 >> v2:172.21.15.111:3300/0 conn(0x7f14b80ad930 0x7f14b80add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.009+0000 7f14bcb9c700 1 --2- 172.21.15.111:0/606526213 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14b806c6c0 0x7f14b8070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.010+0000 7f14b7fff700 1 -- 172.21.15.111:0/606526213 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14b806c6c0 msgr2=0x7f14b8070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:59.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.010+0000 7f14b7fff700 1 --2- 172.21.15.111:0/606526213 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14b806c6c0 0x7f14b8070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.010+0000 7f14b7fff700 1 -- 172.21.15.111:0/606526213 >> v1:172.21.15.111:6789/0 conn(0x7f14b8160fd0 legacy=0x7f14b806a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.010+0000 7f14b7fff700 1 -- 172.21.15.111:0/606526213 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b8161810 con 0x7f14b80ad930 2023-03-29T10:57:59.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.011+0000 7f14b7fff700 1 --2- 172.21.15.111:0/606526213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14b80ad930 0x7f14b80add00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009d90 tx=0x7f14a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52d51c60ff4bc44 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.011+0000 7f14b6ffd700 1 -- 172.21.15.111:0/606526213 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800e070 con 0x7f14b80ad930 2023-03-29T10:57:59.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.011+0000 7f14b6ffd700 1 -- 172.21.15.111:0/606526213 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a8003a20 con 0x7f14b80ad930 2023-03-29T10:57:59.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.011+0000 7f14b6ffd700 1 -- 172.21.15.111:0/606526213 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8003dd0 con 0x7f14b80ad930 2023-03-29T10:57:59.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.010+0000 7f0395ffb700 1 -- 172.21.15.111:0/880857222 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390011830 con 0x7f03a006c6c0 2023-03-29T10:57:59.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.011+0000 7f0395ffb700 1 -- 172.21.15.111:0/880857222 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03900119b0 con 0x7f03a006c6c0 2023-03-29T10:57:59.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.011+0000 7f0395ffb700 1 -- 172.21.15.111:0/880857222 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0390011d60 con 0x7f03a006c6c0 2023-03-29T10:57:59.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.011+0000 7f03a75ff700 1 -- 172.21.15.111:0/880857222 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a0161dd0 con 0x7f03a006c6c0 2023-03-29T10:57:59.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.011+0000 7f03a75ff700 1 -- 172.21.15.111:0/880857222 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a0161af0 con 0x7f03a006c6c0 2023-03-29T10:57:59.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.012+0000 7f14be5ff700 1 -- 172.21.15.111:0/606526213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14b80ad930 msgr2=0x7f14b80add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.012+0000 7f14be5ff700 1 --2- 172.21.15.111:0/606526213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14b80ad930 0x7f14b80add00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009d90 tx=0x7f14a800b330 comp rx=0 tx=0).stop 2023-03-29T10:57:59.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.014+0000 7f14be5ff700 1 -- 172.21.15.111:0/606526213 shutdown_connections 2023-03-29T10:57:59.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.014+0000 7f14be5ff700 1 --2- 172.21.15.111:0/606526213 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14b806c6c0 0x7f14b8070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.014+0000 7f14be5ff700 1 --2- 172.21.15.111:0/606526213 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14b80ad930 0x7f14b80add00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.014+0000 7f14be5ff700 1 -- 172.21.15.111:0/606526213 >> 172.21.15.111:0/606526213 conn(0x7f14b80cf6e0 msgr2=0x7f14b80cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:59.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.015+0000 7f14be5ff700 1 -- 172.21.15.111:0/606526213 shutdown_connections 2023-03-29T10:57:59.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.015+0000 7f14be5ff700 1 -- 172.21.15.111:0/606526213 wait complete. 2023-03-29T10:57:59.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.015+0000 7f03a75ff700 1 -- 172.21.15.111:0/880857222 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03800052e0 con 0x7f03a006c6c0 2023-03-29T10:57:59.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.017+0000 7f14be5ff700 1 Processor -- start 2023-03-29T10:57:59.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.019+0000 7f14be5ff700 1 -- start start 2023-03-29T10:57:59.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.021+0000 7f14be5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b806c6c0 0x7f14b815e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.021+0000 7f14be5ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14b80ad930 0x7f14b8150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.021+0000 7f14be5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14b8160fd0 0x7f14b81ff390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.021+0000 7f14be5ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b80c5f80 con 0x7f14b80ad930 2023-03-29T10:57:59.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.021+0000 7f14be5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b81512f0 con 0x7f14b806c6c0 2023-03-29T10:57:59.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.021+0000 7f14be5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b80c3950 con 0x7f14b8160fd0 2023-03-29T10:57:59.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.022+0000 7f14b7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b806c6c0 0x7f14b815e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.022+0000 7f14b7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b806c6c0 0x7f14b815e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:33958/0 (socket says 172.21.15.111:33958) 2023-03-29T10:57:59.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.022+0000 7f14b7fff700 1 -- 172.21.15.111:0/3947059983 learned_addr learned my addr 172.21.15.111:0/3947059983 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:59.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.022+0000 7f14bcb9c700 1 --2- 172.21.15.111:0/3947059983 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14b8160fd0 0x7f14b81ff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.022 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.022+0000 7f14b77fe700 1 --2- 172.21.15.111:0/3947059983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14b80ad930 0x7f14b8150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.023+0000 7f14b7fff700 1 -- 172.21.15.111:0/3947059983 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14b8160fd0 msgr2=0x7f14b81ff390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.023+0000 7f14b7fff700 1 --2- 172.21.15.111:0/3947059983 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14b8160fd0 0x7f14b81ff390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.023+0000 7f14b7fff700 1 -- 172.21.15.111:0/3947059983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14b80ad930 msgr2=0x7f14b8150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.023+0000 7f14b7fff700 1 --2- 172.21.15.111:0/3947059983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14b80ad930 0x7f14b8150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.023+0000 7f14b7fff700 1 -- 172.21.15.111:0/3947059983 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a8009a70 con 0x7f14b806c6c0 2023-03-29T10:57:59.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.024+0000 7f14b7fff700 1 --2- 172.21.15.111:0/3947059983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b806c6c0 0x7f14b815e250 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f14b806b8d0 tx=0x7f14a8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.026+0000 7f14b4ff9700 1 -- 172.21.15.111:0/3947059983 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8004670 con 0x7f14b806c6c0 2023-03-29T10:57:59.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.026+0000 7f14b4ff9700 1 -- 172.21.15.111:0/3947059983 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a80047d0 con 0x7f14b806c6c0 2023-03-29T10:57:59.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.026+0000 7f14b4ff9700 1 -- 172.21.15.111:0/3947059983 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8012650 con 0x7f14b806c6c0 2023-03-29T10:57:59.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.026+0000 7f14be5ff700 1 -- 172.21.15.111:0/3947059983 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b8161dd0 con 0x7f14b806c6c0 2023-03-29T10:57:59.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.026+0000 7f14be5ff700 1 -- 172.21.15.111:0/3947059983 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b8161af0 con 0x7f14b806c6c0 2023-03-29T10:57:59.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.028+0000 7f0395ffb700 1 -- 172.21.15.111:0/880857222 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f039001a400 con 0x7f03a006c6c0 2023-03-29T10:57:59.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.030+0000 7f14be5ff700 1 -- 172.21.15.111:0/3947059983 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14980052e0 con 0x7f14b806c6c0 2023-03-29T10:57:59.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.029+0000 7f0395ffb700 1 --2- 172.21.15.111:0/880857222 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0384069eb0 0x7f038406c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.030+0000 7f0395ffb700 1 -- 172.21.15.111:0/880857222 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f03900886b0 con 0x7f03a006c6c0 2023-03-29T10:57:59.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.030+0000 7f0395ffb700 1 -- 172.21.15.111:0/880857222 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f0390088b30 con 0x7f03a006c6c0 2023-03-29T10:57:59.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.033+0000 7f14b4ff9700 1 -- 172.21.15.111:0/3947059983 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f14a80127b0 con 0x7f14b806c6c0 2023-03-29T10:57:59.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.034+0000 7f14b4ff9700 1 --2- 172.21.15.111:0/3947059983 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f14a4069eb0 0x7f14a406c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.034+0000 7f14b4ff9700 1 -- 172.21.15.111:0/3947059983 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f14a808a3b0 con 0x7f14b806c6c0 2023-03-29T10:57:59.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.034+0000 7f03a4b9a700 1 --2- 172.21.15.111:0/880857222 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0384069eb0 0x7f038406c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.035+0000 7f03a4b9a700 1 --2- 172.21.15.111:0/880857222 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0384069eb0 0x7f038406c360 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0398009fd0 tx=0x7f039800d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.035+0000 7f14b77fe700 1 --2- 172.21.15.111:0/3947059983 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f14a4069eb0 0x7f14a406c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.037+0000 7f14b77fe700 1 --2- 172.21.15.111:0/3947059983 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f14a4069eb0 0x7f14a406c360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f14b80b38c0 tx=0x7f14b0007d50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.039+0000 7f14b4ff9700 1 -- 172.21.15.111:0/3947059983 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f14a8055980 con 0x7f14b806c6c0 2023-03-29T10:57:59.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.076+0000 7f5f05dff700 1 Processor -- start 2023-03-29T10:57:59.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.077+0000 7f5f05dff700 1 -- start start 2023-03-29T10:57:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.079+0000 7f5f05dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5f0006aaf0 0x7f5f0015cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.079+0000 7f5f05dff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f00158d10 con 0x7f5f000cacf0 2023-03-29T10:57:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.079+0000 7f5f05dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f0015d2f0 con 0x7f5f000add70 2023-03-29T10:57:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.079+0000 7f5f05dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000c97d0 con 0x7f5f0006aaf0 2023-03-29T10:57:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.080+0000 7f5efeffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5f000add70 0x7f5f0006a3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.111:58828/0 (socket says 172.21.15.111:58828) 2023-03-29T10:57:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.080+0000 7f5efeffd700 1 -- 172.21.15.111:0/2943405208 learned_addr learned my addr 172.21.15.111:0/2943405208 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.080+0000 7f5efffff700 1 --2- 172.21.15.111:0/2943405208 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5f0006aaf0 0x7f5f0015cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.081+0000 7f5efe7fc700 1 -- 172.21.15.111:0/2943405208 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2444771659 0 0) 0x7f5ef8004d10 con 0x7f5f000add70 2023-03-29T10:57:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.081+0000 7f5efe7fc700 1 -- 172.21.15.111:0/2943405208 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ee80035b0 con 0x7f5f000add70 2023-03-29T10:57:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.081+0000 7f5efffff700 1 -- 172.21.15.111:0/2943405208 >> v1:172.21.15.119:6789/0 conn(0x7f5f000add70 legacy=0x7f5f0006a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.081+0000 7f5efffff700 1 -- 172.21.15.111:0/2943405208 >> v1:172.21.15.111:6789/0 conn(0x7f5f000cacf0 legacy=0x7f5f000c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.081+0000 7f5efffff700 1 -- 172.21.15.111:0/2943405208 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f000cd5b0 con 0x7f5f0006aaf0 2023-03-29T10:57:59.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.082+0000 7f5efffff700 1 --2- 172.21.15.111:0/2943405208 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5f0006aaf0 0x7f5f0015cdb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009870 tx=0x7f5ef4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0f88a9ad6c92b1e server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.084+0000 7f5efe7fc700 1 -- 172.21.15.111:0/2943405208 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4004230 con 0x7f5f0006aaf0 2023-03-29T10:57:59.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.084+0000 7f5efe7fc700 1 -- 172.21.15.111:0/2943405208 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef4004390 con 0x7f5f0006aaf0 2023-03-29T10:57:59.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.084+0000 7f5efe7fc700 1 -- 172.21.15.111:0/2943405208 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef400f740 con 0x7f5f0006aaf0 2023-03-29T10:57:59.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.085+0000 7f5f05dff700 1 -- 172.21.15.111:0/2943405208 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5f0006aaf0 msgr2=0x7f5f0015cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.085+0000 7f5f05dff700 1 --2- 172.21.15.111:0/2943405208 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5f0006aaf0 0x7f5f0015cdb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009870 tx=0x7f5ef4009b80 comp rx=0 tx=0).stop 2023-03-29T10:57:59.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.086+0000 7f5f05dff700 1 -- 172.21.15.111:0/2943405208 shutdown_connections 2023-03-29T10:57:59.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.086+0000 7f5f05dff700 1 --2- 172.21.15.111:0/2943405208 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5f0006aaf0 0x7f5f0015cdb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.086+0000 7f5f05dff700 1 -- 172.21.15.111:0/2943405208 >> 172.21.15.111:0/2943405208 conn(0x7f5f000cf6e0 msgr2=0x7f5f000cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:59.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.086+0000 7f5f05dff700 1 -- 172.21.15.111:0/2943405208 shutdown_connections 2023-03-29T10:57:59.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.086+0000 7f5f05dff700 1 -- 172.21.15.111:0/2943405208 wait complete. 2023-03-29T10:57:59.091 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.089+0000 7f5f05dff700 1 Processor -- start 2023-03-29T10:57:59.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.089+0000 7f5f05dff700 1 -- start start 2023-03-29T10:57:59.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.090+0000 7f5f05dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f0006aaf0 0x7f5f00151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.090+0000 7f5f05dff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f000add70 0x7f5f0014e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.090+0000 7f5f05dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5f000cacf0 0x7f5f000c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.090+0000 7f5f05dff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f00152390 con 0x7f5f000add70 2023-03-29T10:57:59.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.091+0000 7f5f05dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000c92a0 con 0x7f5f0006aaf0 2023-03-29T10:57:59.093 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.091+0000 7f5f05dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000c6270 con 0x7f5f000cacf0 2023-03-29T10:57:59.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.093+0000 7f5efeffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f000add70 0x7f5f0014e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.094+0000 7f5efeffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f000add70 0x7f5f0014e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50280/0 (socket says 172.21.15.111:50280) 2023-03-29T10:57:59.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.094+0000 7f5efeffd700 1 -- 172.21.15.111:0/2640804139 learned_addr learned my addr 172.21.15.111:0/2640804139 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:59.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.095+0000 7f5efeffd700 1 -- 172.21.15.111:0/2640804139 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5f000cacf0 msgr2=0x7f5f000c8d60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-03-29T10:57:59.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.095+0000 7f5efeffd700 1 --2- 172.21.15.111:0/2640804139 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5f000cacf0 0x7f5f000c8d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.095+0000 7f5efeffd700 1 -- 172.21.15.111:0/2640804139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f0006aaf0 msgr2=0x7f5f00151b30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-03-29T10:57:59.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.095+0000 7f5efeffd700 1 --2- 172.21.15.111:0/2640804139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f0006aaf0 0x7f5f00151b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.095+0000 7f5efeffd700 1 -- 172.21.15.111:0/2640804139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef40094a0 con 0x7f5f000add70 2023-03-29T10:57:59.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.095+0000 7f5efeffd700 1 --2- 172.21.15.111:0/2640804139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f000add70 0x7f5f0014e020 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8001a40 tx=0x7f5ef8001e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.099+0000 7f5ee7fff700 1 -- 172.21.15.111:0/2640804139 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8006550 con 0x7f5f000add70 2023-03-29T10:57:59.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.099+0000 7f5ee7fff700 1 -- 172.21.15.111:0/2640804139 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef80066d0 con 0x7f5f000add70 2023-03-29T10:57:59.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.099+0000 7f5ee7fff700 1 -- 172.21.15.111:0/2640804139 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8006a80 con 0x7f5f000add70 2023-03-29T10:57:59.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.099+0000 7f5f05dff700 1 -- 172.21.15.111:0/2640804139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f000cd2d0 con 0x7f5f000add70 2023-03-29T10:57:59.100 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.099+0000 7f5f05dff700 1 -- 172.21.15.111:0/2640804139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f000ccff0 con 0x7f5f000add70 2023-03-29T10:57:59.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.104+0000 7f5ee7fff700 1 -- 172.21.15.111:0/2640804139 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f5ef8006be0 con 0x7f5f000add70 2023-03-29T10:57:59.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.105+0000 7f5ee7fff700 1 --2- 172.21.15.111:0/2640804139 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5eec069eb0 0x7f5eec06c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.106+0000 7f5ee7fff700 1 -- 172.21.15.111:0/2640804139 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5ef8087a30 con 0x7f5f000add70 2023-03-29T10:57:59.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.107+0000 7f5f05dff700 1 -- 172.21.15.111:0/2640804139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f0006cc40 con 0x7f5f000add70 2023-03-29T10:57:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.111+0000 7f5eff7fe700 1 --2- 172.21.15.111:0/2640804139 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5eec069eb0 0x7f5eec06c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.112+0000 7f5eff7fe700 1 --2- 172.21.15.111:0/2640804139 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5eec069eb0 0x7f5eec06c360 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0006fd0 tx=0x7f5ef0006bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.117+0000 7f5ee7fff700 1 -- 172.21.15.111:0/2640804139 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f5ef8052fd0 con 0x7f5f000add70 2023-03-29T10:57:59.163 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.163+0000 7fd48c5ff700 1 Processor -- start 2023-03-29T10:57:59.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.165+0000 7f5228dff700 1 Processor -- start 2023-03-29T10:57:59.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.163+0000 7fd48c5ff700 1 -- start start 2023-03-29T10:57:59.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.165+0000 7fd48c5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4840cfbf0 0x7fd4841608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.165+0000 7fd48c5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4840d1100 0x7fd4840d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.165+0000 7fd48c5ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4840d0020 con 0x7fd4840b87a0 2023-03-29T10:57:59.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.165+0000 7fd48c5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4840d02d0 con 0x7fd4840d1100 2023-03-29T10:57:59.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.165+0000 7fd48c5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4840d0cb0 con 0x7fd4840cfbf0 2023-03-29T10:57:59.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.166+0000 7fd48ab9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4840d1100 0x7fd4840d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.166+0000 7fd48ab9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4840d1100 0x7fd4840d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:33984/0 (socket says 172.21.15.111:33984) 2023-03-29T10:57:59.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.166+0000 7fd48ab9c700 1 -- 172.21.15.111:0/3748972483 learned_addr learned my addr 172.21.15.111:0/3748972483 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:59.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.167+0000 7fd489b9a700 1 --2- 172.21.15.111:0/3748972483 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4840cfbf0 0x7fd4841608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.167+0000 7fd48ab9c700 1 -- 172.21.15.111:0/3748972483 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4840cfbf0 msgr2=0x7fd4841608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:59.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.167+0000 7fd48ab9c700 1 --2- 172.21.15.111:0/3748972483 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4840cfbf0 0x7fd4841608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.167+0000 7fd48ab9c700 1 -- 172.21.15.111:0/3748972483 >> v1:172.21.15.111:6789/0 conn(0x7fd4840b87a0 legacy=0x7fd4841601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:59.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.167+0000 7fd48ab9c700 1 -- 172.21.15.111:0/3748972483 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4840cdc60 con 0x7fd4840d1100 2023-03-29T10:57:59.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.168+0000 7fd48ab9c700 1 --2- 172.21.15.111:0/3748972483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4840d1100 0x7fd4840d1580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009e50 tx=0x7fd47c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=830176209de118d1 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.168+0000 7fd489399700 1 -- 172.21.15.111:0/3748972483 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c00e070 con 0x7fd4840d1100 2023-03-29T10:57:59.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.168+0000 7fd489399700 1 -- 172.21.15.111:0/3748972483 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd47c003a20 con 0x7fd4840d1100 2023-03-29T10:57:59.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.168+0000 7fd489399700 1 -- 172.21.15.111:0/3748972483 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47c003db0 con 0x7fd4840d1100 2023-03-29T10:57:59.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.169+0000 7fd48c5ff700 1 -- 172.21.15.111:0/3748972483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4840d1100 msgr2=0x7fd4840d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.169+0000 7fd48c5ff700 1 --2- 172.21.15.111:0/3748972483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4840d1100 0x7fd4840d1580 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009e50 tx=0x7fd47c00b330 comp rx=0 tx=0).stop 2023-03-29T10:57:59.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.168+0000 7f5228dff700 1 -- start start 2023-03-29T10:57:59.170 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.170+0000 7f5228dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f52240ad690 0x7f52240adb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.170 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.170+0000 7f5228dff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52240c19c0 0x7f52240c1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.170 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.170+0000 7f5228dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52240cd1f0 con 0x7f5224160d20 2023-03-29T10:57:59.170 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.170+0000 7f5228dff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52241610f0 con 0x7f52240c19c0 2023-03-29T10:57:59.170 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.170+0000 7f5228dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52240c22d0 con 0x7f52240ad690 2023-03-29T10:57:59.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.170+0000 7fd48c5ff700 1 -- 172.21.15.111:0/3748972483 shutdown_connections 2023-03-29T10:57:59.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.170+0000 7fd48c5ff700 1 --2- 172.21.15.111:0/3748972483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4840d1100 0x7fd4840d1580 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.170+0000 7fd48c5ff700 1 --2- 172.21.15.111:0/3748972483 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4840cfbf0 0x7fd4841608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.170+0000 7fd48c5ff700 1 -- 172.21.15.111:0/3748972483 >> 172.21.15.111:0/3748972483 conn(0x7fd484071b10 msgr2=0x7fd484071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:59.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.171+0000 7fd48c5ff700 1 -- 172.21.15.111:0/3748972483 shutdown_connections 2023-03-29T10:57:59.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.171+0000 7fd48c5ff700 1 -- 172.21.15.111:0/3748972483 wait complete. 2023-03-29T10:57:59.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.172+0000 7f5222d9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52240c19c0 0x7f52240c1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.172+0000 7f5222d9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f52240c19c0 0x7f52240c1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50290/0 (socket says 172.21.15.111:50290) 2023-03-29T10:57:59.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.172+0000 7f5222d9d700 1 -- 172.21.15.111:0/78269786 learned_addr learned my addr 172.21.15.111:0/78269786 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:59.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.173+0000 7f5222d9d700 1 -- 172.21.15.111:0/78269786 >> v2:172.21.15.119:3300/0 conn(0x7f52240ad690 msgr2=0x7f52240adb10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-03-29T10:57:59.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.173+0000 7f5222d9d700 1 --2- 172.21.15.111:0/78269786 >> v2:172.21.15.119:3300/0 conn(0x7f52240ad690 0x7f52240adb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.173+0000 7f5222d9d700 1 -- 172.21.15.111:0/78269786 >> v1:172.21.15.119:6789/0 conn(0x7f5224160d20 legacy=0x7f522406cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:59.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.173+0000 7f5222d9d700 1 -- 172.21.15.111:0/78269786 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52240cc670 con 0x7f52240c19c0 2023-03-29T10:57:59.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.174+0000 7f5222d9d700 1 --2- 172.21.15.111:0/78269786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52240c19c0 0x7f52240c1d90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5224071b90 tx=0x7f5218009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb14056752915a19 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.174+0000 7fd48c5ff700 1 Processor -- start 2023-03-29T10:57:59.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.174+0000 7fd48c5ff700 1 -- start start 2023-03-29T10:57:59.175 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.176+0000 7f522159a700 1 -- 172.21.15.111:0/78269786 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5218004230 con 0x7f52240c19c0 2023-03-29T10:57:59.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.176+0000 7f522159a700 1 -- 172.21.15.111:0/78269786 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5218004390 con 0x7f52240c19c0 2023-03-29T10:57:59.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.176+0000 7f522159a700 1 -- 172.21.15.111:0/78269786 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52180036a0 con 0x7f52240c19c0 2023-03-29T10:57:59.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.175+0000 7fd48c5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4840b87a0 0x7fd4840ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.175+0000 7fd48c5ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4840cfbf0 0x7fd4840b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.175+0000 7fd48c5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4840d1100 0x7fd4840b7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.175+0000 7fd48c5ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4840cb330 con 0x7fd4840cfbf0 2023-03-29T10:57:59.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.175+0000 7fd48c5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48414c1a0 con 0x7fd4840d1100 2023-03-29T10:57:59.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.175+0000 7fd48c5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48414b620 con 0x7fd4840b87a0 2023-03-29T10:57:59.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.176+0000 7fd489b9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4840cfbf0 0x7fd4840b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.176+0000 7fd489b9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4840cfbf0 0x7fd4840b6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50294/0 (socket says 172.21.15.111:50294) 2023-03-29T10:57:59.177 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.176+0000 7fd489b9a700 1 -- 172.21.15.111:0/1825929892 learned_addr learned my addr 172.21.15.111:0/1825929892 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:59.177 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.177+0000 7f5228dff700 1 -- 172.21.15.111:0/78269786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52240c19c0 msgr2=0x7f52240c1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.177 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.177+0000 7f5228dff700 1 --2- 172.21.15.111:0/78269786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52240c19c0 0x7f52240c1d90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5224071b90 tx=0x7f5218009bd0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.177 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.177+0000 7f5228dff700 1 -- 172.21.15.111:0/78269786 shutdown_connections 2023-03-29T10:57:59.177 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.177+0000 7f5228dff700 1 --2- 172.21.15.111:0/78269786 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52240c19c0 0x7f52240c1d90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.177 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.177+0000 7f5228dff700 1 --2- 172.21.15.111:0/78269786 >> v2:172.21.15.119:3300/0 conn(0x7f52240ad690 0x7f52240adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.177 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.177+0000 7f5228dff700 1 -- 172.21.15.111:0/78269786 >> 172.21.15.111:0/78269786 conn(0x7f52240cf720 msgr2=0x7f52240cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:59.177 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.177+0000 7fd489b9a700 1 -- 172.21.15.111:0/1825929892 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4840b87a0 msgr2=0x7fd4840ccd80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-03-29T10:57:59.177 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.177+0000 7fd489b9a700 1 --2- 172.21.15.111:0/1825929892 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4840b87a0 0x7fd4840ccd80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.178 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.177+0000 7fd489b9a700 1 -- 172.21.15.111:0/1825929892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4840d1100 msgr2=0x7fd4840b7230 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-03-29T10:57:59.178 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.177+0000 7fd489b9a700 1 --2- 172.21.15.111:0/1825929892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4840d1100 0x7fd4840b7230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.178 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.177+0000 7fd489b9a700 1 -- 172.21.15.111:0/1825929892 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c009b30 con 0x7fd4840cfbf0 2023-03-29T10:57:59.178 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.178+0000 7fd489b9a700 1 --2- 172.21.15.111:0/1825929892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4840cfbf0 0x7fd4840b6cf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd48000c7f0 tx=0x7fd48000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.179+0000 7fd476ffd700 1 -- 172.21.15.111:0/1825929892 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480011800 con 0x7fd4840cfbf0 2023-03-29T10:57:59.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.179+0000 7fd476ffd700 1 -- 172.21.15.111:0/1825929892 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd480011960 con 0x7fd4840cfbf0 2023-03-29T10:57:59.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.179+0000 7fd476ffd700 1 -- 172.21.15.111:0/1825929892 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480011d10 con 0x7fd4840cfbf0 2023-03-29T10:57:59.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.179+0000 7fd48c5ff700 1 -- 172.21.15.111:0/1825929892 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4840cd980 con 0x7fd4840cfbf0 2023-03-29T10:57:59.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.179+0000 7fd48c5ff700 1 -- 172.21.15.111:0/1825929892 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4840cd6a0 con 0x7fd4840cfbf0 2023-03-29T10:57:59.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.184+0000 7f5228dff700 1 -- 172.21.15.111:0/78269786 shutdown_connections 2023-03-29T10:57:59.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.181+0000 7fd48c5ff700 1 -- 172.21.15.111:0/1825929892 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd484160e30 con 0x7fd4840cfbf0 2023-03-29T10:57:59.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.190+0000 7f5228dff700 1 -- 172.21.15.111:0/78269786 wait complete. 2023-03-29T10:57:59.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.193+0000 7f5228dff700 1 Processor -- start 2023-03-29T10:57:59.198 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.195+0000 7f5228dff700 1 -- start start 2023-03-29T10:57:59.198 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.197+0000 7f5228dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52240ad690 0x7f52241f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.198 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.197+0000 7f5228dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52240c19c0 0x7f52241f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.198 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.197+0000 7f5228dff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52240bd650 0x7f52241623d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.198 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.197+0000 7f5228dff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52241f7710 con 0x7f52240bd650 2023-03-29T10:57:59.198 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.197+0000 7f5228dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52241fa140 con 0x7f52240ad690 2023-03-29T10:57:59.198 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.197+0000 7f5228dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52241fa420 con 0x7f52240c19c0 2023-03-29T10:57:59.198 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.193+0000 7fd476ffd700 1 -- 172.21.15.111:0/1825929892 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fd480024480 con 0x7fd4840cfbf0 2023-03-29T10:57:59.198 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.195+0000 7fd476ffd700 1 --2- 172.21.15.111:0/1825929892 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd468069eb0 0x7fd46806c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.198 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.195+0000 7fd476ffd700 1 -- 172.21.15.111:0/1825929892 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd4800886a0 con 0x7fd4840cfbf0 2023-03-29T10:57:59.198 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.195+0000 7fd476ffd700 1 -- 172.21.15.111:0/1825929892 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fd480088b20 con 0x7fd4840cfbf0 2023-03-29T10:57:59.199 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.198+0000 7f5221d9b700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52240c19c0 0x7f52241f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.199 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.199+0000 7f5221d9b700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52240c19c0 0x7f52241f5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:38736/0 (socket says 172.21.15.111:38736) 2023-03-29T10:57:59.199 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.199+0000 7f5221d9b700 1 -- 172.21.15.111:0/671331489 learned_addr learned my addr 172.21.15.111:0/671331489 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:59.199 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.199+0000 7f5222d9d700 1 --2- 172.21.15.111:0/671331489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52240bd650 0x7f52241623d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.199 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.200+0000 7f5222d9d700 1 -- 172.21.15.111:0/671331489 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52240c19c0 msgr2=0x7f52241f5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.199 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.200+0000 7f5222d9d700 1 --2- 172.21.15.111:0/671331489 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52240c19c0 0x7f52241f5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.200+0000 7f5222d9d700 1 -- 172.21.15.111:0/671331489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52240ad690 msgr2=0x7f52241f71d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-03-29T10:57:59.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.200+0000 7f5222d9d700 1 --2- 172.21.15.111:0/671331489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52240ad690 0x7f52241f71d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.200+0000 7f5222d9d700 1 -- 172.21.15.111:0/671331489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5218009550 con 0x7f52240bd650 2023-03-29T10:57:59.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.200+0000 7f5222d9d700 1 --2- 172.21.15.111:0/671331489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52240bd650 0x7f52241623d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5218015000 tx=0x7f5218004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.201+0000 7fd48a39b700 1 --2- 172.21.15.111:0/1825929892 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd468069eb0 0x7fd46806c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.202+0000 7f5212ffd700 1 -- 172.21.15.111:0/671331489 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5218011630 con 0x7f52240bd650 2023-03-29T10:57:59.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.202+0000 7f5212ffd700 1 -- 172.21.15.111:0/671331489 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52180117b0 con 0x7f52240bd650 2023-03-29T10:57:59.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.202+0000 7f5212ffd700 1 -- 172.21.15.111:0/671331489 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5218011b60 con 0x7f52240bd650 2023-03-29T10:57:59.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.202+0000 7f5228dff700 1 -- 172.21.15.111:0/671331489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52240ccc30 con 0x7f52240bd650 2023-03-29T10:57:59.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.205+0000 7fd48a39b700 1 --2- 172.21.15.111:0/1825929892 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd468069eb0 0x7fd46806c360 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd478006fd0 tx=0x7fd478006bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.202+0000 7f5228dff700 1 -- 172.21.15.111:0/671331489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52240cc950 con 0x7f52240bd650 2023-03-29T10:57:59.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.210+0000 7f5212ffd700 1 -- 172.21.15.111:0/671331489 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f5218021020 con 0x7f52240bd650 2023-03-29T10:57:59.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.211+0000 7f5212ffd700 1 --2- 172.21.15.111:0/671331489 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5204069eb0 0x7f520406c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.212+0000 7f5212ffd700 1 -- 172.21.15.111:0/671331489 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f52180894d0 con 0x7f52240bd650 2023-03-29T10:57:59.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.213+0000 7f5210ff9700 1 -- 172.21.15.111:0/671331489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51fc0052e0 con 0x7f52240bd650 2023-03-29T10:57:59.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.223+0000 7f522259c700 1 --2- 172.21.15.111:0/671331489 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5204069eb0 0x7f520406c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.224+0000 7f5212ffd700 1 -- 172.21.15.111:0/671331489 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f5218054b00 con 0x7f52240bd650 2023-03-29T10:57:59.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.226+0000 7f522259c700 1 --2- 172.21.15.111:0/671331489 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5204069eb0 0x7f520406c360 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52240709f0 tx=0x7f5214007a40 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.231+0000 7feda29ff700 1 Processor -- start 2023-03-29T10:57:59.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.232+0000 7feda29ff700 1 -- start start 2023-03-29T10:57:59.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.234+0000 7feda29ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fed9c0add70 0x7fed9c15cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.234+0000 7feda29ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fed9c0c90c0 0x7fed9c06a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.234+0000 7feda29ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed9c158d10 con 0x7fed9c0cbc20 2023-03-29T10:57:59.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.234+0000 7feda29ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed9c15d2f0 con 0x7fed9c0c90c0 2023-03-29T10:57:59.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.234+0000 7feda29ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed9c0cd5b0 con 0x7fed9c0add70 2023-03-29T10:57:59.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.235+0000 7fed93fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fed9c0add70 0x7fed9c15cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.235+0000 7fed93fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fed9c0add70 0x7fed9c15cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:34004/0 (socket says 172.21.15.111:34004) 2023-03-29T10:57:59.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.235+0000 7fed93fff700 1 -- 172.21.15.111:0/1778586601 learned_addr learned my addr 172.21.15.111:0/1778586601 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:59.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.236+0000 7fed93fff700 1 -- 172.21.15.111:0/1778586601 >> v2:172.21.15.111:3300/0 conn(0x7fed9c0c90c0 msgr2=0x7fed9c06a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:59.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.236+0000 7fed93fff700 1 --2- 172.21.15.111:0/1778586601 >> v2:172.21.15.111:3300/0 conn(0x7fed9c0c90c0 0x7fed9c06a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.236+0000 7fed93fff700 1 -- 172.21.15.111:0/1778586601 >> v1:172.21.15.119:6789/0 conn(0x7fed9c0cbc20 legacy=0x7fed9c0cacf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-03-29T10:57:59.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.236+0000 7fed93fff700 1 -- 172.21.15.111:0/1778586601 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9c0ccff0 con 0x7fed9c0add70 2023-03-29T10:57:59.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.237+0000 7fed93fff700 1 --2- 172.21.15.111:0/1778586601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed9c0add70 0x7fed9c15cdb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fed9c06b7d0 tx=0x7fed8c009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed8e3296ef01aa75 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.239+0000 7fed9b7fe700 1 -- 172.21.15.111:0/1778586601 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c004230 con 0x7fed9c0add70 2023-03-29T10:57:59.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.239+0000 7fed9b7fe700 1 -- 172.21.15.111:0/1778586601 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8c004390 con 0x7fed9c0add70 2023-03-29T10:57:59.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.239+0000 7fed9b7fe700 1 -- 172.21.15.111:0/1778586601 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c0036a0 con 0x7fed9c0add70 2023-03-29T10:57:59.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.240+0000 7feda29ff700 1 -- 172.21.15.111:0/1778586601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed9c0add70 msgr2=0x7fed9c15cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.240+0000 7feda29ff700 1 --2- 172.21.15.111:0/1778586601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed9c0add70 0x7fed9c15cdb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fed9c06b7d0 tx=0x7fed8c009bd0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.240+0000 7f354cdff700 1 Processor -- start 2023-03-29T10:57:59.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.240+0000 7f354cdff700 1 -- start start 2023-03-29T10:57:59.244 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.242+0000 7f354cdff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f35480cfbf0 0x7f35481608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.244 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.243+0000 7f354cdff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f35480d1100 0x7f35480d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.244 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.243+0000 7f354cdff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35480d0020 con 0x7f35480b87a0 2023-03-29T10:57:59.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.243+0000 7f354cdff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35480d02d0 con 0x7f35480cfbf0 2023-03-29T10:57:59.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.243+0000 7f354cdff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35480d0cb0 con 0x7f35480d1100 2023-03-29T10:57:59.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.244+0000 7f3546d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f35480d1100 0x7f35480d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.244+0000 7f3546d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f35480d1100 0x7f35480d1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:34020/0 (socket says 172.21.15.111:34020) 2023-03-29T10:57:59.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.244+0000 7f3546d9d700 1 -- 172.21.15.111:0/2996891193 learned_addr learned my addr 172.21.15.111:0/2996891193 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:59.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.244+0000 7f3546d9d700 1 -- 172.21.15.111:0/2996891193 >> v2:172.21.15.111:3300/0 conn(0x7f35480cfbf0 msgr2=0x7f35481608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:59.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.244+0000 7f3546d9d700 1 --2- 172.21.15.111:0/2996891193 >> v2:172.21.15.111:3300/0 conn(0x7f35480cfbf0 0x7f35481608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.244+0000 7f3546d9d700 1 -- 172.21.15.111:0/2996891193 >> v1:172.21.15.119:6789/0 conn(0x7f35480b87a0 legacy=0x7f35481601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:59.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.244+0000 7f3546d9d700 1 -- 172.21.15.111:0/2996891193 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35480cdc60 con 0x7f35480d1100 2023-03-29T10:57:59.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.245+0000 7f3546d9d700 1 --2- 172.21.15.111:0/2996891193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35480d1100 0x7f35480d1520 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f35480d0820 tx=0x7f354000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7236d0ad10d9359 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.241+0000 7feda29ff700 1 -- 172.21.15.111:0/1778586601 shutdown_connections 2023-03-29T10:57:59.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.241+0000 7feda29ff700 1 --2- 172.21.15.111:0/1778586601 >> v2:172.21.15.111:3300/0 conn(0x7fed9c0c90c0 0x7fed9c06a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.246 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.241+0000 7feda29ff700 1 --2- 172.21.15.111:0/1778586601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed9c0add70 0x7fed9c15cdb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.246 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.241+0000 7feda29ff700 1 -- 172.21.15.111:0/1778586601 >> 172.21.15.111:0/1778586601 conn(0x7fed9c0cf6e0 msgr2=0x7fed9c0cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:59.246 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.242+0000 7feda29ff700 1 -- 172.21.15.111:0/1778586601 shutdown_connections 2023-03-29T10:57:59.246 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.242+0000 7feda29ff700 1 -- 172.21.15.111:0/1778586601 wait complete. 2023-03-29T10:57:59.246 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.244+0000 7feda29ff700 1 Processor -- start 2023-03-29T10:57:59.246 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.245+0000 7feda29ff700 1 -- start start 2023-03-29T10:57:59.246 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.246+0000 7feda29ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fed9c0add70 0x7fed9c0b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.246 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.246+0000 7feda29ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed9c0c90c0 0x7fed9c1486a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.246 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.246+0000 7feda29ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed9c0cbc20 0x7fed9c070d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.246 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.246+0000 7feda29ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c148be0 con 0x7fed9c0c90c0 2023-03-29T10:57:59.246 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.246+0000 7feda29ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c0b3080 con 0x7fed9c0cbc20 2023-03-29T10:57:59.246 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.246+0000 7feda29ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c0b3340 con 0x7fed9c0add70 2023-03-29T10:57:59.247 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.246+0000 7f3545d9b700 1 -- 172.21.15.111:0/2996891193 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354000e070 con 0x7f35480d1100 2023-03-29T10:57:59.247 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.246+0000 7f3545d9b700 1 -- 172.21.15.111:0/2996891193 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3540003a20 con 0x7f35480d1100 2023-03-29T10:57:59.247 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.246+0000 7f3545d9b700 1 -- 172.21.15.111:0/2996891193 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3540003dd0 con 0x7f35480d1100 2023-03-29T10:57:59.247 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.247+0000 7f354cdff700 1 -- 172.21.15.111:0/2996891193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35480d1100 msgr2=0x7f35480d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.247 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.247+0000 7f354cdff700 1 --2- 172.21.15.111:0/2996891193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35480d1100 0x7f35480d1520 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f35480d0820 tx=0x7f354000b330 comp rx=0 tx=0).stop 2023-03-29T10:57:59.247 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.247+0000 7fed9bfff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fed9c0add70 0x7fed9c0b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.247 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.247+0000 7fed9bfff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fed9c0add70 0x7fed9c0b43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:38746/0 (socket says 172.21.15.111:38746) 2023-03-29T10:57:59.247 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.247+0000 7fed9bfff700 1 -- 172.21.15.111:0/3404615683 learned_addr learned my addr 172.21.15.111:0/3404615683 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:59.247 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.247+0000 7fed93fff700 1 --2- 172.21.15.111:0/3404615683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed9c0c90c0 0x7fed9c1486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.247+0000 7feda0f9c700 1 --2- 172.21.15.111:0/3404615683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed9c0cbc20 0x7fed9c070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.247+0000 7fed9bfff700 1 -- 172.21.15.111:0/3404615683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed9c0cbc20 msgr2=0x7fed9c070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.247+0000 7fed9bfff700 1 --2- 172.21.15.111:0/3404615683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed9c0cbc20 0x7fed9c070d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.247+0000 7fed9bfff700 1 -- 172.21.15.111:0/3404615683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed9c0c90c0 msgr2=0x7fed9c1486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.247+0000 7fed9bfff700 1 --2- 172.21.15.111:0/3404615683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed9c0c90c0 0x7fed9c1486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.247+0000 7fed9bfff700 1 -- 172.21.15.111:0/3404615683 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8c009550 con 0x7fed9c0add70 2023-03-29T10:57:59.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.248+0000 7fed9bfff700 1 --2- 172.21.15.111:0/3404615683 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fed9c0add70 0x7fed9c0b43e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fed8400b470 tx=0x7fed8400b830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.249+0000 7fed997fa700 1 -- 172.21.15.111:0/3404615683 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed84011830 con 0x7fed9c0add70 2023-03-29T10:57:59.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.249+0000 7fed997fa700 1 -- 172.21.15.111:0/3404615683 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed84011990 con 0x7fed9c0add70 2023-03-29T10:57:59.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.249+0000 7fed997fa700 1 -- 172.21.15.111:0/3404615683 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed84011d40 con 0x7fed9c0add70 2023-03-29T10:57:59.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.249+0000 7feda29ff700 1 -- 172.21.15.111:0/3404615683 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed9c06cc40 con 0x7fed9c0add70 2023-03-29T10:57:59.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.249+0000 7feda29ff700 1 -- 172.21.15.111:0/3404615683 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9c0cd2d0 con 0x7fed9c0add70 2023-03-29T10:57:59.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.249+0000 7f354cdff700 1 -- 172.21.15.111:0/2996891193 shutdown_connections 2023-03-29T10:57:59.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.249+0000 7f354cdff700 1 --2- 172.21.15.111:0/2996891193 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35480d1100 0x7f35480d1520 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.249+0000 7f354cdff700 1 --2- 172.21.15.111:0/2996891193 >> v2:172.21.15.111:3300/0 conn(0x7f35480cfbf0 0x7f35481608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.249+0000 7f354cdff700 1 -- 172.21.15.111:0/2996891193 >> 172.21.15.111:0/2996891193 conn(0x7f3548071b10 msgr2=0x7f3548071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:59.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.250+0000 7f2a5e9ff700 1 Processor -- start 2023-03-29T10:57:59.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.250+0000 7f2a5e9ff700 1 -- start start 2023-03-29T10:57:59.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.253+0000 7f2a5e9ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2a58160d20 0x7f2a5806cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.253+0000 7f2a5e9ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a580c20d0 0x7f2a580adb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.253+0000 7f2a5e9ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a580cd1f0 con 0x7f2a580ad690 2023-03-29T10:57:59.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.253+0000 7f2a5e9ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a581610f0 con 0x7f2a58160d20 2023-03-29T10:57:59.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.253+0000 7f2a5e9ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580cc670 con 0x7f2a580c20d0 2023-03-29T10:57:59.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.253+0000 7f2a5cf9c700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a580c20d0 0x7f2a580adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.253+0000 7f2a5cf9c700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a580c20d0 0x7f2a580adb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:38748/0 (socket says 172.21.15.111:38748) 2023-03-29T10:57:59.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.253+0000 7f2a5cf9c700 1 -- 172.21.15.111:0/3472242151 learned_addr learned my addr 172.21.15.111:0/3472242151 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:59.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.254+0000 7f2a57fff700 1 --2- 172.21.15.111:0/3472242151 >> v2:172.21.15.111:3300/0 conn(0x7f2a58160d20 0x7f2a5806cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.254+0000 7f2a5cf9c700 1 -- 172.21.15.111:0/3472242151 >> v2:172.21.15.111:3300/0 conn(0x7f2a58160d20 msgr2=0x7f2a5806cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:59.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.254+0000 7f2a5cf9c700 1 --2- 172.21.15.111:0/3472242151 >> v2:172.21.15.111:3300/0 conn(0x7f2a58160d20 0x7f2a5806cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.254+0000 7f2a5cf9c700 1 -- 172.21.15.111:0/3472242151 >> v1:172.21.15.111:6789/0 conn(0x7f2a580ad690 legacy=0x7f2a580c19c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-03-29T10:57:59.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.254+0000 7f2a5cf9c700 1 -- 172.21.15.111:0/3472242151 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a580cc950 con 0x7f2a580c20d0 2023-03-29T10:57:59.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.253+0000 7fed997fa700 1 -- 172.21.15.111:0/3404615683 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fed8401a400 con 0x7fed9c0add70 2023-03-29T10:57:59.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.254+0000 7fed997fa700 1 --2- 172.21.15.111:0/3404615683 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fed7c069de0 0x7fed7c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.254+0000 7fed997fa700 1 -- 172.21.15.111:0/3404615683 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fed84088340 con 0x7fed9c0add70 2023-03-29T10:57:59.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.255+0000 7f2a5cf9c700 1 --2- 172.21.15.111:0/3472242151 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a580c20d0 0x7f2a580adb10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2a500098f0 tx=0x7f2a50009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d44ca402961d7d2 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.255+0000 7f354cdff700 1 -- 172.21.15.111:0/2996891193 shutdown_connections 2023-03-29T10:57:59.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.256+0000 7fed93fff700 1 --2- 172.21.15.111:0/3404615683 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fed7c069de0 0x7fed7c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.255+0000 7f354cdff700 1 -- 172.21.15.111:0/2996891193 wait complete. 2023-03-29T10:57:59.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.256+0000 7f2a56ffd700 1 -- 172.21.15.111:0/3472242151 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50004230 con 0x7f2a580c20d0 2023-03-29T10:57:59.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.256+0000 7f2a56ffd700 1 -- 172.21.15.111:0/3472242151 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a50004390 con 0x7f2a580c20d0 2023-03-29T10:57:59.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.256+0000 7f2a56ffd700 1 -- 172.21.15.111:0/3472242151 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a500036a0 con 0x7f2a580c20d0 2023-03-29T10:57:59.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.257+0000 7feda29ff700 1 -- 172.21.15.111:0/3404615683 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed800052e0 con 0x7fed9c0add70 2023-03-29T10:57:59.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.257+0000 7fed93fff700 1 --2- 172.21.15.111:0/3404615683 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fed7c069de0 0x7fed7c06c290 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fed8c000c00 tx=0x7fed8c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.257+0000 7f2a5e9ff700 1 -- 172.21.15.111:0/3472242151 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a580c20d0 msgr2=0x7f2a580adb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.257+0000 7f2a5e9ff700 1 --2- 172.21.15.111:0/3472242151 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a580c20d0 0x7f2a580adb10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2a500098f0 tx=0x7f2a50009c00 comp rx=0 tx=0).stop 2023-03-29T10:57:59.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.259+0000 7f354cdff700 1 Processor -- start 2023-03-29T10:57:59.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.258+0000 7f2a5e9ff700 1 -- 172.21.15.111:0/3472242151 shutdown_connections 2023-03-29T10:57:59.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.258+0000 7f2a5e9ff700 1 --2- 172.21.15.111:0/3472242151 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a580c20d0 0x7f2a580adb10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.258+0000 7f2a5e9ff700 1 --2- 172.21.15.111:0/3472242151 >> v2:172.21.15.111:3300/0 conn(0x7f2a58160d20 0x7f2a5806cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.258+0000 7f2a5e9ff700 1 -- 172.21.15.111:0/3472242151 >> 172.21.15.111:0/3472242151 conn(0x7f2a580cf720 msgr2=0x7f2a580cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:59.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.259+0000 7f2a5e9ff700 1 -- 172.21.15.111:0/3472242151 shutdown_connections 2023-03-29T10:57:59.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.259+0000 7f2a5e9ff700 1 -- 172.21.15.111:0/3472242151 wait complete. 2023-03-29T10:57:59.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.260+0000 7f354cdff700 1 -- start start 2023-03-29T10:57:59.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.261+0000 7f354cdff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35480b87a0 0x7f354815f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.261+0000 7f354cdff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35480cfbf0 0x7f3548159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.261+0000 7f354cdff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35480d1100 0x7f35480cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.261+0000 7f354cdff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35480cd300 con 0x7f35480d1100 2023-03-29T10:57:59.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.261+0000 7f354cdff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354815a520 con 0x7f35480cfbf0 2023-03-29T10:57:59.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.261+0000 7f354cdff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35480cc6f0 con 0x7f35480b87a0 2023-03-29T10:57:59.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.262+0000 7f354659c700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35480b87a0 0x7f354815f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.262+0000 7f353ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35480cfbf0 0x7f3548159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.262+0000 7f353ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35480cfbf0 0x7f3548159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:34036/0 (socket says 172.21.15.111:34036) 2023-03-29T10:57:59.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.262+0000 7f353ffff700 1 -- 172.21.15.111:0/388662874 learned_addr learned my addr 172.21.15.111:0/388662874 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:59.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.261+0000 7f2a5e9ff700 1 Processor -- start 2023-03-29T10:57:59.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.263+0000 7f3546d9d700 1 --2- 172.21.15.111:0/388662874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35480d1100 0x7f35480cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.263+0000 7f353ffff700 1 -- 172.21.15.111:0/388662874 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35480b87a0 msgr2=0x7f354815f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.263+0000 7f353ffff700 1 --2- 172.21.15.111:0/388662874 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35480b87a0 0x7f354815f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.263+0000 7f353ffff700 1 -- 172.21.15.111:0/388662874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35480d1100 msgr2=0x7f35480cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.263+0000 7f353ffff700 1 --2- 172.21.15.111:0/388662874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35480d1100 0x7f35480cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.263+0000 7f353ffff700 1 -- 172.21.15.111:0/388662874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3540009b30 con 0x7f35480cfbf0 2023-03-29T10:57:59.264 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.263+0000 7f3546d9d700 1 --2- 172.21.15.111:0/388662874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35480d1100 0x7f35480cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:59.264 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.262+0000 7f2a5e9ff700 1 -- start start 2023-03-29T10:57:59.264 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.263+0000 7f353ffff700 1 --2- 172.21.15.111:0/388662874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35480cfbf0 0x7f3548159fe0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f353800b530 tx=0x7f353800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.264 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.263+0000 7f2a5e9ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a580ad690 0x7f2a580bef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.264 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.263+0000 7f2a5e9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a580c20d0 0x7f2a580bd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.264+0000 7f2a577fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a580c20d0 0x7f2a580bd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.264+0000 7f2a577fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a580c20d0 0x7f2a580bd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:34044/0 (socket says 172.21.15.111:34044) 2023-03-29T10:57:59.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.264+0000 7f2a57fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a580ad690 0x7f2a580bef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.264+0000 7f2a57fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a580ad690 0x7f2a580bef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:38770/0 (socket says 172.21.15.111:38770) 2023-03-29T10:57:59.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.264+0000 7f2a5e9ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a58160d20 0x7f2a58155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.264+0000 7f2a5e9ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580bf4c0 con 0x7f2a58160d20 2023-03-29T10:57:59.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.264+0000 7f2a5e9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580beaf0 con 0x7f2a580c20d0 2023-03-29T10:57:59.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.264+0000 7f2a5e9ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580bdee0 con 0x7f2a580ad690 2023-03-29T10:57:59.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.264+0000 7f2a577fe700 1 -- 172.21.15.111:0/4058920897 learned_addr learned my addr 172.21.15.111:0/4058920897 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:59.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.265+0000 7f2a5cf9c700 1 --2- 172.21.15.111:0/4058920897 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a58160d20 0x7f2a58155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.265 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.265+0000 7fed997fa700 1 -- 172.21.15.111:0/3404615683 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fed84053860 con 0x7fed9c0add70 2023-03-29T10:57:59.266 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.265+0000 7f2a5cf9c700 1 -- 172.21.15.111:0/4058920897 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a580ad690 msgr2=0x7f2a580bef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.266 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.265+0000 7f2a5cf9c700 1 --2- 172.21.15.111:0/4058920897 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a580ad690 0x7f2a580bef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.266 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.265+0000 7f2a5cf9c700 1 -- 172.21.15.111:0/4058920897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a580c20d0 msgr2=0x7f2a580bd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.266 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.265+0000 7f2a5cf9c700 1 --2- 172.21.15.111:0/4058920897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a580c20d0 0x7f2a580bd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.266 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.265+0000 7f2a5cf9c700 1 -- 172.21.15.111:0/4058920897 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a50009550 con 0x7f2a58160d20 2023-03-29T10:57:59.266 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.265+0000 7f2a577fe700 1 --2- 172.21.15.111:0/4058920897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a580c20d0 0x7f2a580bd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:59.266 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.267+0000 7f2a5cf9c700 1 --2- 172.21.15.111:0/4058920897 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a58160d20 0x7f2a58155920 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2a50015000 tx=0x7f2a50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.267+0000 7f353effd700 1 -- 172.21.15.111:0/388662874 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3538011830 con 0x7f35480cfbf0 2023-03-29T10:57:59.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.267+0000 7f353effd700 1 -- 172.21.15.111:0/388662874 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3538011990 con 0x7f35480cfbf0 2023-03-29T10:57:59.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.267+0000 7f353effd700 1 -- 172.21.15.111:0/388662874 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3538011d20 con 0x7f35480cfbf0 2023-03-29T10:57:59.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.267+0000 7f354cdff700 1 -- 172.21.15.111:0/388662874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35480cd980 con 0x7f35480cfbf0 2023-03-29T10:57:59.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.268+0000 7f354cdff700 1 -- 172.21.15.111:0/388662874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35480cd6a0 con 0x7f35480cfbf0 2023-03-29T10:57:59.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.268+0000 7f2a54ff9700 1 -- 172.21.15.111:0/4058920897 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50003e60 con 0x7f2a58160d20 2023-03-29T10:57:59.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.268+0000 7f2a54ff9700 1 -- 172.21.15.111:0/4058920897 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a500045d0 con 0x7f2a58160d20 2023-03-29T10:57:59.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.268+0000 7f2a54ff9700 1 -- 172.21.15.111:0/4058920897 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50011850 con 0x7f2a58160d20 2023-03-29T10:57:59.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.268+0000 7f2a5e9ff700 1 -- 172.21.15.111:0/4058920897 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a580ccf10 con 0x7f2a58160d20 2023-03-29T10:57:59.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.268+0000 7f2a5e9ff700 1 -- 172.21.15.111:0/4058920897 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a580ccc30 con 0x7f2a58160d20 2023-03-29T10:57:59.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.272+0000 7f353effd700 1 -- 172.21.15.111:0/388662874 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f3538033800 con 0x7f35480cfbf0 2023-03-29T10:57:59.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.273+0000 7f353effd700 1 --2- 172.21.15.111:0/388662874 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3528069eb0 0x7f352806c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.273+0000 7f353effd700 1 -- 172.21.15.111:0/388662874 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f35380876f0 con 0x7f35480cfbf0 2023-03-29T10:57:59.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.274+0000 7f2a54ff9700 1 -- 172.21.15.111:0/4058920897 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f2a50021070 con 0x7f2a58160d20 2023-03-29T10:57:59.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.275+0000 7f354659c700 1 --2- 172.21.15.111:0/388662874 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3528069eb0 0x7f352806c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.276+0000 7f354659c700 1 --2- 172.21.15.111:0/388662874 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3528069eb0 0x7f352806c360 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f35480b35e0 tx=0x7f35300090e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.276+0000 7f2a54ff9700 1 --2- 172.21.15.111:0/4058920897 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2a38069eb0 0x7f2a3806c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.276+0000 7f2a54ff9700 1 -- 172.21.15.111:0/4058920897 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2a500896e0 con 0x7f2a58160d20 2023-03-29T10:57:59.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.276+0000 7f354cdff700 1 -- 172.21.15.111:0/388662874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35480cd4b0 con 0x7f35480cfbf0 2023-03-29T10:57:59.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.277+0000 7f2a57fff700 1 --2- 172.21.15.111:0/4058920897 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2a38069eb0 0x7f2a3806c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.278+0000 7f2a57fff700 1 --2- 172.21.15.111:0/4058920897 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2a38069eb0 0x7f2a3806c360 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009be0 tx=0x7f2a40009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.278+0000 7f2a5e9ff700 1 -- 172.21.15.111:0/4058920897 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3c0052e0 con 0x7f2a58160d20 2023-03-29T10:57:59.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.282+0000 7f353effd700 1 -- 172.21.15.111:0/388662874 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f35480cd4b0 con 0x7f35480cfbf0 2023-03-29T10:57:59.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.288+0000 7f2a54ff9700 1 -- 172.21.15.111:0/4058920897 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f2a50054d10 con 0x7f2a58160d20 2023-03-29T10:57:59.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.454+0000 7f14be5ff700 1 -- 172.21.15.111:0/3947059983 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1498005590 con 0x7f14b806c6c0 2023-03-29T10:57:59.457 INFO:teuthology.orchestra.run.smithi111.stdout:55834574860 2023-03-29T10:57:59.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.454+0000 7f14b4ff9700 1 -- 172.21.15.111:0/3947059983 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f14a8058fa0 con 0x7f14b806c6c0 2023-03-29T10:57:59.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.458+0000 7f14be5ff700 1 -- 172.21.15.111:0/3947059983 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f14a4069eb0 msgr2=0x7f14a406c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.458+0000 7f14be5ff700 1 --2- 172.21.15.111:0/3947059983 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f14a4069eb0 0x7f14a406c360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f14b80b38c0 tx=0x7f14b0007d50 comp rx=0 tx=0).stop 2023-03-29T10:57:59.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.458+0000 7f14be5ff700 1 -- 172.21.15.111:0/3947059983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b806c6c0 msgr2=0x7f14b815e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.458+0000 7f14be5ff700 1 --2- 172.21.15.111:0/3947059983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b806c6c0 0x7f14b815e250 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f14b806b8d0 tx=0x7f14a8004ad0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.459+0000 7f14be5ff700 1 -- 172.21.15.111:0/3947059983 shutdown_connections 2023-03-29T10:57:59.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.459+0000 7f14be5ff700 1 --2- 172.21.15.111:0/3947059983 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f14a4069eb0 0x7f14a406c360 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.459+0000 7f14be5ff700 1 --2- 172.21.15.111:0/3947059983 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14b8160fd0 0x7f14b81ff390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.459+0000 7f14be5ff700 1 --2- 172.21.15.111:0/3947059983 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b806c6c0 0x7f14b815e250 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.459+0000 7f14be5ff700 1 --2- 172.21.15.111:0/3947059983 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14b80ad930 0x7f14b8150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.459+0000 7f14be5ff700 1 -- 172.21.15.111:0/3947059983 >> 172.21.15.111:0/3947059983 conn(0x7f14b80cf6e0 msgr2=0x7f14b80b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:59.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.458+0000 7f03a75ff700 1 -- 172.21.15.111:0/880857222 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0380005860 con 0x7f03a006c6c0 2023-03-29T10:57:59.459 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.459+0000 7f14be5ff700 1 -- 172.21.15.111:0/3947059983 shutdown_connections 2023-03-29T10:57:59.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.460+0000 7f14be5ff700 1 -- 172.21.15.111:0/3947059983 wait complete. 2023-03-29T10:57:59.460 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.460+0000 7f0395ffb700 1 -- 172.21.15.111:0/880857222 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0390053d30 con 0x7f03a006c6c0 2023-03-29T10:57:59.461 INFO:teuthology.orchestra.run.smithi111.stdout:55834574860 2023-03-29T10:57:59.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.464+0000 7f03a75ff700 1 -- 172.21.15.111:0/880857222 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0384069eb0 msgr2=0x7f038406c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.464+0000 7f03a75ff700 1 --2- 172.21.15.111:0/880857222 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0384069eb0 0x7f038406c360 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0398009fd0 tx=0x7f039800d040 comp rx=0 tx=0).stop 2023-03-29T10:57:59.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.464+0000 7f03a75ff700 1 -- 172.21.15.111:0/880857222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a006c6c0 msgr2=0x7f03a01576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.464 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.464+0000 7f03a75ff700 1 --2- 172.21.15.111:0/880857222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a006c6c0 0x7f03a01576d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f039000b470 tx=0x7f039000b780 comp rx=0 tx=0).stop 2023-03-29T10:57:59.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.465+0000 7f03a75ff700 1 -- 172.21.15.111:0/880857222 shutdown_connections 2023-03-29T10:57:59.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.465+0000 7f03a75ff700 1 --2- 172.21.15.111:0/880857222 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0384069eb0 0x7f038406c360 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0398009fd0 tx=0x7f039800d040 comp rx=0 tx=0).stop 2023-03-29T10:57:59.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.465+0000 7f03a75ff700 1 --2- 172.21.15.111:0/880857222 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f03a006c6c0 0x7f03a01576d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.465+0000 7f03a75ff700 1 --2- 172.21.15.111:0/880857222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03a0160fd0 0x7f03a015f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.465+0000 7f03a75ff700 1 --2- 172.21.15.111:0/880857222 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f03a00ad930 0x7f03a015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.465+0000 7f03a75ff700 1 -- 172.21.15.111:0/880857222 >> 172.21.15.111:0/880857222 conn(0x7f03a00cf6e0 msgr2=0x7f03a006dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:59.465 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.466+0000 7f03a75ff700 1 -- 172.21.15.111:0/880857222 shutdown_connections 2023-03-29T10:57:59.466 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.466+0000 7f03a75ff700 1 -- 172.21.15.111:0/880857222 wait complete. 2023-03-29T10:57:59.490 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.1 2023-03-29T10:57:59.490 DEBUG:teuthology.parallel:result is None 2023-03-29T10:57:59.495 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.0 2023-03-29T10:57:59.495 DEBUG:teuthology.parallel:result is None 2023-03-29T10:57:59.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.538+0000 7f5f05dff700 1 -- 172.21.15.111:0/2640804139 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f5f0015d2f0 con 0x7f5f000add70 2023-03-29T10:57:59.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.539+0000 7f5ee7fff700 1 -- 172.21.15.111:0/2640804139 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5f0015d2f0 con 0x7f5f000add70 2023-03-29T10:57:59.540 INFO:teuthology.orchestra.run.smithi111.stdout:55834574858 2023-03-29T10:57:59.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.543+0000 7f5f05dff700 1 -- 172.21.15.111:0/2640804139 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5eec069eb0 msgr2=0x7f5eec06c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.543+0000 7f5f05dff700 1 --2- 172.21.15.111:0/2640804139 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5eec069eb0 0x7f5eec06c360 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0006fd0 tx=0x7f5ef0006bc0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.543+0000 7f5f05dff700 1 -- 172.21.15.111:0/2640804139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f000add70 msgr2=0x7f5f0014e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.543+0000 7f5f05dff700 1 --2- 172.21.15.111:0/2640804139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f000add70 0x7f5f0014e020 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8001a40 tx=0x7f5ef8001e00 comp rx=0 tx=0).stop 2023-03-29T10:57:59.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.543+0000 7f5f05dff700 1 -- 172.21.15.111:0/2640804139 shutdown_connections 2023-03-29T10:57:59.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.543+0000 7f5f05dff700 1 --2- 172.21.15.111:0/2640804139 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5eec069eb0 0x7f5eec06c360 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.543+0000 7f5f05dff700 1 --2- 172.21.15.111:0/2640804139 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5f000cacf0 0x7f5f000c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.543+0000 7f5f05dff700 1 --2- 172.21.15.111:0/2640804139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f0006aaf0 0x7f5f00151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.543+0000 7f5f05dff700 1 --2- 172.21.15.111:0/2640804139 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f000add70 0x7f5f0014e020 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.543+0000 7f5f05dff700 1 -- 172.21.15.111:0/2640804139 >> 172.21.15.111:0/2640804139 conn(0x7f5f000cf6e0 msgr2=0x7f5f0006a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:59.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.543+0000 7f5f05dff700 1 -- 172.21.15.111:0/2640804139 shutdown_connections 2023-03-29T10:57:59.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.543+0000 7f5f05dff700 1 -- 172.21.15.111:0/2640804139 wait complete. 2023-03-29T10:57:59.570 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2023-03-29T10:57:59.571 DEBUG:teuthology.parallel:result is None 2023-03-29T10:57:59.573 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.574+0000 7f5210ff9700 1 -- 172.21.15.111:0/671331489 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f51fc005590 con 0x7f52240bd650 2023-03-29T10:57:59.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.574+0000 7f5212ffd700 1 -- 172.21.15.111:0/671331489 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5218029030 con 0x7f52240bd650 2023-03-29T10:57:59.574 INFO:teuthology.orchestra.run.smithi111.stdout:55834574860 2023-03-29T10:57:59.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.578+0000 7f5210ff9700 1 -- 172.21.15.111:0/671331489 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5204069eb0 msgr2=0x7f520406c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.578+0000 7f5210ff9700 1 --2- 172.21.15.111:0/671331489 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5204069eb0 0x7f520406c360 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f52240709f0 tx=0x7f5214007a40 comp rx=0 tx=0).stop 2023-03-29T10:57:59.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.578+0000 7f5210ff9700 1 -- 172.21.15.111:0/671331489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52240bd650 msgr2=0x7f52241623d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.578+0000 7f5210ff9700 1 --2- 172.21.15.111:0/671331489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52240bd650 0x7f52241623d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f5218015000 tx=0x7f5218004750 comp rx=0 tx=0).stop 2023-03-29T10:57:59.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.578+0000 7f5210ff9700 1 -- 172.21.15.111:0/671331489 shutdown_connections 2023-03-29T10:57:59.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.578+0000 7f5210ff9700 1 --2- 172.21.15.111:0/671331489 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5204069eb0 0x7f520406c360 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.578+0000 7f5210ff9700 1 --2- 172.21.15.111:0/671331489 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52240bd650 0x7f52241623d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.578+0000 7f5210ff9700 1 --2- 172.21.15.111:0/671331489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52240ad690 0x7f52241f71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.578+0000 7f5210ff9700 1 --2- 172.21.15.111:0/671331489 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52240c19c0 0x7f52241f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.578 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.578+0000 7f5210ff9700 1 -- 172.21.15.111:0/671331489 >> 172.21.15.111:0/671331489 conn(0x7f52240cf720 msgr2=0x7f5224163780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:59.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.579+0000 7f5210ff9700 1 -- 172.21.15.111:0/671331489 shutdown_connections 2023-03-29T10:57:59.579 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.579+0000 7f5210ff9700 1 -- 172.21.15.111:0/671331489 wait complete. 2023-03-29T10:57:59.596 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.2 2023-03-29T10:57:59.596 DEBUG:teuthology.parallel:result is None 2023-03-29T10:57:59.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.603+0000 7f2a5e9ff700 1 -- 172.21.15.111:0/4058920897 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2a3c005590 con 0x7f2a58160d20 2023-03-29T10:57:59.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.603+0000 7f2a54ff9700 1 -- 172.21.15.111:0/4058920897 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2a50058330 con 0x7f2a58160d20 2023-03-29T10:57:59.603 INFO:teuthology.orchestra.run.smithi111.stdout:55834574860 2023-03-29T10:57:59.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.605+0000 7f2a5e9ff700 1 -- 172.21.15.111:0/4058920897 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2a38069eb0 msgr2=0x7f2a3806c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.605+0000 7f2a5e9ff700 1 --2- 172.21.15.111:0/4058920897 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2a38069eb0 0x7f2a3806c360 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009be0 tx=0x7f2a40009210 comp rx=0 tx=0).stop 2023-03-29T10:57:59.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.605+0000 7f2a5e9ff700 1 -- 172.21.15.111:0/4058920897 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a58160d20 msgr2=0x7f2a58155920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.605+0000 7f2a5e9ff700 1 --2- 172.21.15.111:0/4058920897 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a58160d20 0x7f2a58155920 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2a50015000 tx=0x7f2a50004750 comp rx=0 tx=0).stop 2023-03-29T10:57:59.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.606+0000 7f2a5e9ff700 1 -- 172.21.15.111:0/4058920897 shutdown_connections 2023-03-29T10:57:59.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.606+0000 7f2a5e9ff700 1 --2- 172.21.15.111:0/4058920897 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2a38069eb0 0x7f2a3806c360 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.606+0000 7f2a5e9ff700 1 --2- 172.21.15.111:0/4058920897 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2a58160d20 0x7f2a58155920 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.606+0000 7f2a5e9ff700 1 --2- 172.21.15.111:0/4058920897 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2a580ad690 0x7f2a580bef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.606+0000 7f2a5e9ff700 1 --2- 172.21.15.111:0/4058920897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a580c20d0 0x7f2a580bd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.606+0000 7f2a5e9ff700 1 -- 172.21.15.111:0/4058920897 >> 172.21.15.111:0/4058920897 conn(0x7f2a580cf720 msgr2=0x7f2a580cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:59.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.606+0000 7f2a5e9ff700 1 -- 172.21.15.111:0/4058920897 shutdown_connections 2023-03-29T10:57:59.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.606+0000 7f2a5e9ff700 1 -- 172.21.15.111:0/4058920897 wait complete. 2023-03-29T10:57:59.623 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.7 2023-03-29T10:57:59.623 DEBUG:teuthology.parallel:result is None 2023-03-29T10:57:59.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.632+0000 7fd48c5ff700 1 -- 172.21.15.111:0/1825929892 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd484074430 con 0x7fd4840cfbf0 2023-03-29T10:57:59.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.633+0000 7fd476ffd700 1 -- 172.21.15.111:0/1825929892 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd480053be0 con 0x7fd4840cfbf0 2023-03-29T10:57:59.632 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2023-03-29T10:57:59.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.636+0000 7fd48c5ff700 1 -- 172.21.15.111:0/1825929892 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd468069eb0 msgr2=0x7fd46806c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.636+0000 7fd48c5ff700 1 --2- 172.21.15.111:0/1825929892 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd468069eb0 0x7fd46806c360 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd478006fd0 tx=0x7fd478006bc0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.636+0000 7fd48c5ff700 1 -- 172.21.15.111:0/1825929892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4840cfbf0 msgr2=0x7fd4840b6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.636+0000 7fd48c5ff700 1 --2- 172.21.15.111:0/1825929892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4840cfbf0 0x7fd4840b6cf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd48000c7f0 tx=0x7fd48000cbb0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.636+0000 7fd48c5ff700 1 -- 172.21.15.111:0/1825929892 shutdown_connections 2023-03-29T10:57:59.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.636+0000 7fd48c5ff700 1 --2- 172.21.15.111:0/1825929892 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd468069eb0 0x7fd46806c360 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.636+0000 7fd48c5ff700 1 --2- 172.21.15.111:0/1825929892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4840d1100 0x7fd4840b7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.636+0000 7fd48c5ff700 1 --2- 172.21.15.111:0/1825929892 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4840b87a0 0x7fd4840ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.636+0000 7fd48c5ff700 1 --2- 172.21.15.111:0/1825929892 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4840cfbf0 0x7fd4840b6cf0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.636+0000 7fd48c5ff700 1 -- 172.21.15.111:0/1825929892 >> 172.21.15.111:0/1825929892 conn(0x7fd484071b10 msgr2=0x7fd4840cf460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:59.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.636+0000 7fd48c5ff700 1 -- 172.21.15.111:0/1825929892 shutdown_connections 2023-03-29T10:57:59.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.636+0000 7fd48c5ff700 1 -- 172.21.15.111:0/1825929892 wait complete. 2023-03-29T10:57:59.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.637+0000 7f354cdff700 1 -- 172.21.15.111:0/388662874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f35480cd4b0 con 0x7f35480cfbf0 2023-03-29T10:57:59.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.638+0000 7f353effd700 1 -- 172.21.15.111:0/388662874 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f35480cd4b0 con 0x7f35480cfbf0 2023-03-29T10:57:59.637 INFO:teuthology.orchestra.run.smithi111.stdout:55834574860 2023-03-29T10:57:59.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.640+0000 7f354cdff700 1 -- 172.21.15.111:0/388662874 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3528069eb0 msgr2=0x7f352806c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.640+0000 7f354cdff700 1 --2- 172.21.15.111:0/388662874 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3528069eb0 0x7f352806c360 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f35480b35e0 tx=0x7f35300090e0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.640+0000 7f354cdff700 1 -- 172.21.15.111:0/388662874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35480cfbf0 msgr2=0x7f3548159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.640+0000 7f354cdff700 1 --2- 172.21.15.111:0/388662874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35480cfbf0 0x7f3548159fe0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f353800b530 tx=0x7f353800b840 comp rx=0 tx=0).stop 2023-03-29T10:57:59.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.640+0000 7f354cdff700 1 -- 172.21.15.111:0/388662874 shutdown_connections 2023-03-29T10:57:59.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.640+0000 7f354cdff700 1 --2- 172.21.15.111:0/388662874 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3528069eb0 0x7f352806c360 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.640+0000 7f354cdff700 1 --2- 172.21.15.111:0/388662874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35480d1100 0x7f35480cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.640+0000 7f354cdff700 1 --2- 172.21.15.111:0/388662874 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35480b87a0 0x7f354815f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.640+0000 7f354cdff700 1 --2- 172.21.15.111:0/388662874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35480cfbf0 0x7f3548159fe0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.640+0000 7f354cdff700 1 -- 172.21.15.111:0/388662874 >> 172.21.15.111:0/388662874 conn(0x7f3548071b10 msgr2=0x7f35480b5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:59.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.640+0000 7f354cdff700 1 -- 172.21.15.111:0/388662874 shutdown_connections 2023-03-29T10:57:59.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.640+0000 7f354cdff700 1 -- 172.21.15.111:0/388662874 wait complete. 2023-03-29T10:57:59.654 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2023-03-29T10:57:59.654 DEBUG:teuthology.parallel:result is None 2023-03-29T10:57:59.657 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.4 2023-03-29T10:57:59.657 DEBUG:teuthology.parallel:result is None 2023-03-29T10:57:59.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.670+0000 7feda29ff700 1 -- 172.21.15.111:0/3404615683 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fed80005590 con 0x7fed9c0add70 2023-03-29T10:57:59.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.671+0000 7fed997fa700 1 -- 172.21.15.111:0/3404615683 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fed84015070 con 0x7fed9c0add70 2023-03-29T10:57:59.670 INFO:teuthology.orchestra.run.smithi111.stdout:55834574859 2023-03-29T10:57:59.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.673+0000 7feda29ff700 1 -- 172.21.15.111:0/3404615683 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fed7c069de0 msgr2=0x7fed7c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.673+0000 7feda29ff700 1 --2- 172.21.15.111:0/3404615683 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fed7c069de0 0x7fed7c06c290 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fed8c000c00 tx=0x7fed8c008030 comp rx=0 tx=0).stop 2023-03-29T10:57:59.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.673+0000 7feda29ff700 1 -- 172.21.15.111:0/3404615683 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fed9c0add70 msgr2=0x7fed9c0b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.673+0000 7feda29ff700 1 --2- 172.21.15.111:0/3404615683 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fed9c0add70 0x7fed9c0b43e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fed8400b470 tx=0x7fed8400b830 comp rx=0 tx=0).stop 2023-03-29T10:57:59.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.673+0000 7feda29ff700 1 -- 172.21.15.111:0/3404615683 shutdown_connections 2023-03-29T10:57:59.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.673+0000 7feda29ff700 1 --2- 172.21.15.111:0/3404615683 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fed7c069de0 0x7fed7c06c290 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.673+0000 7feda29ff700 1 --2- 172.21.15.111:0/3404615683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed9c0cbc20 0x7fed9c070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.673+0000 7feda29ff700 1 --2- 172.21.15.111:0/3404615683 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fed9c0add70 0x7fed9c0b43e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.673+0000 7feda29ff700 1 --2- 172.21.15.111:0/3404615683 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fed9c0c90c0 0x7fed9c1486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.673+0000 7feda29ff700 1 -- 172.21.15.111:0/3404615683 >> 172.21.15.111:0/3404615683 conn(0x7fed9c0cf6e0 msgr2=0x7fed9c0cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:59.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.673+0000 7feda29ff700 1 -- 172.21.15.111:0/3404615683 shutdown_connections 2023-03-29T10:57:59.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.673+0000 7feda29ff700 1 -- 172.21.15.111:0/3404615683 wait complete. 2023-03-29T10:57:59.690 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2023-03-29T10:57:59.690 DEBUG:teuthology.parallel:result is None 2023-03-29T10:57:59.690 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-03-29T10:57:59.691 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T10:57:59.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.916+0000 7fc42c1ff700 1 Processor -- start 2023-03-29T10:57:59.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.916+0000 7fc42c1ff700 1 -- start start 2023-03-29T10:57:59.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.917+0000 7fc42c1ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4241594c0 0x7fc4240cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.917+0000 7fc42c1ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4240cd9f0 0x7fc4240cde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.917+0000 7fc42c1ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4240ce350 con 0x7fc42415dce0 2023-03-29T10:57:59.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.917+0000 7fc42c1ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4240b0dd0 con 0x7fc4241594c0 2023-03-29T10:57:59.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.917+0000 7fc42c1ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240b0fc0 con 0x7fc4240cd9f0 2023-03-29T10:57:59.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.918+0000 7fc42979a700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc42415dce0 0x7fc4240b06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36950/0 (socket says 172.21.15.111:36950) 2023-03-29T10:57:59.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.918+0000 7fc42979a700 1 -- 172.21.15.111:0/3622427638 learned_addr learned my addr 172.21.15.111:0/3622427638 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:57:59.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.918+0000 7fc42a79c700 1 --2- 172.21.15.111:0/3622427638 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4240cd9f0 0x7fc4240cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.918+0000 7fc429f9b700 1 --2- 172.21.15.111:0/3622427638 >> v2:172.21.15.119:3300/0 conn(0x7fc4241594c0 0x7fc4240cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.918+0000 7fc428f99700 1 -- 172.21.15.111:0/3622427638 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2367418997 0 0) 0x7fc4240ce350 con 0x7fc42415dce0 2023-03-29T10:57:59.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.918+0000 7fc428f99700 1 -- 172.21.15.111:0/3622427638 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4100035b0 con 0x7fc42415dce0 2023-03-29T10:57:59.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.918+0000 7fc42a79c700 1 -- 172.21.15.111:0/3622427638 >> v2:172.21.15.119:3300/0 conn(0x7fc4241594c0 msgr2=0x7fc4240cd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:57:59.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.918+0000 7fc42a79c700 1 --2- 172.21.15.111:0/3622427638 >> v2:172.21.15.119:3300/0 conn(0x7fc4241594c0 0x7fc4240cd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.918+0000 7fc42a79c700 1 -- 172.21.15.111:0/3622427638 >> v1:172.21.15.111:6789/0 conn(0x7fc42415dce0 legacy=0x7fc4240b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.918+0000 7fc42a79c700 1 -- 172.21.15.111:0/3622427638 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4240cc9b0 con 0x7fc4240cd9f0 2023-03-29T10:57:59.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.918+0000 7fc429f9b700 1 --2- 172.21.15.111:0/3622427638 >> v2:172.21.15.119:3300/0 conn(0x7fc4241594c0 0x7fc4240cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:57:59.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.919+0000 7fc42a79c700 1 --2- 172.21.15.111:0/3622427638 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4240cd9f0 0x7fc4240cde10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc41c009840 tx=0x7fc41c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=646264d5de0c156d server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.919+0000 7fc428f99700 1 -- 172.21.15.111:0/3622427638 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c011830 con 0x7fc4240cd9f0 2023-03-29T10:57:59.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.919+0000 7fc428f99700 1 -- 172.21.15.111:0/3622427638 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41c011990 con 0x7fc4240cd9f0 2023-03-29T10:57:59.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.919+0000 7fc428f99700 1 -- 172.21.15.111:0/3622427638 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c011d20 con 0x7fc4240cd9f0 2023-03-29T10:57:59.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.919+0000 7fc42c1ff700 1 -- 172.21.15.111:0/3622427638 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4240cd9f0 msgr2=0x7fc4240cde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.919+0000 7fc42c1ff700 1 --2- 172.21.15.111:0/3622427638 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4240cd9f0 0x7fc4240cde10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc41c009840 tx=0x7fc41c009b50 comp rx=0 tx=0).stop 2023-03-29T10:57:59.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.919+0000 7fc42c1ff700 1 -- 172.21.15.111:0/3622427638 shutdown_connections 2023-03-29T10:57:59.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.919+0000 7fc42c1ff700 1 --2- 172.21.15.111:0/3622427638 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc4240cd9f0 0x7fc4240cde10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.919+0000 7fc42c1ff700 1 --2- 172.21.15.111:0/3622427638 >> v2:172.21.15.119:3300/0 conn(0x7fc4241594c0 0x7fc4240cd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.919+0000 7fc42c1ff700 1 -- 172.21.15.111:0/3622427638 >> 172.21.15.111:0/3622427638 conn(0x7fc4240cb6c0 msgr2=0x7fc4240cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:57:59.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.920+0000 7fc42c1ff700 1 -- 172.21.15.111:0/3622427638 shutdown_connections 2023-03-29T10:57:59.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.920+0000 7fc42c1ff700 1 -- 172.21.15.111:0/3622427638 wait complete. 2023-03-29T10:57:59.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.921+0000 7fc42c1ff700 1 Processor -- start 2023-03-29T10:57:59.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.921+0000 7fc42c1ff700 1 -- start start 2023-03-29T10:57:59.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.922+0000 7fc42c1ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4240cd9f0 0x7fc4240ac180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.922+0000 7fc42c1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4241594c0 0x7fc42414e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.922+0000 7fc42c1ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc42415dce0 0x7fc4240c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.922+0000 7fc42c1ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240ad070 con 0x7fc4240cd9f0 2023-03-29T10:57:59.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.922+0000 7fc42c1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240ac6c0 con 0x7fc4241594c0 2023-03-29T10:57:59.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.922+0000 7fc42c1ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240b43d0 con 0x7fc42415dce0 2023-03-29T10:57:59.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.923+0000 7fc429f9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4240cd9f0 0x7fc4240ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.923+0000 7fc429f9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4240cd9f0 0x7fc4240ac180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50368/0 (socket says 172.21.15.111:50368) 2023-03-29T10:57:59.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.923+0000 7fc429f9b700 1 -- 172.21.15.111:0/3257221390 learned_addr learned my addr 172.21.15.111:0/3257221390 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:57:59.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.923+0000 7fc42a79c700 1 --2- 172.21.15.111:0/3257221390 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc42415dce0 0x7fc4240c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.923+0000 7fc429f9b700 1 -- 172.21.15.111:0/3257221390 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc42415dce0 msgr2=0x7fc4240c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.923+0000 7fc42979a700 1 --2- 172.21.15.111:0/3257221390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4241594c0 0x7fc42414e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.923+0000 7fc429f9b700 1 --2- 172.21.15.111:0/3257221390 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc42415dce0 0x7fc4240c37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.923+0000 7fc429f9b700 1 -- 172.21.15.111:0/3257221390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4241594c0 msgr2=0x7fc42414e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:57:59.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.923+0000 7fc429f9b700 1 --2- 172.21.15.111:0/3257221390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4241594c0 0x7fc42414e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:57:59.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.923+0000 7fc429f9b700 1 -- 172.21.15.111:0/3257221390 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc420009a70 con 0x7fc4240cd9f0 2023-03-29T10:57:59.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.923+0000 7fc42a79c700 1 --2- 172.21.15.111:0/3257221390 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc42415dce0 0x7fc4240c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:59.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.923+0000 7fc42979a700 1 --2- 172.21.15.111:0/3257221390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4241594c0 0x7fc42414e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:57:59.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.923+0000 7fc429f9b700 1 --2- 172.21.15.111:0/3257221390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4240cd9f0 0x7fc4240ac180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc42000efc0 tx=0x7fc42000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.923+0000 7fc41a7fc700 1 -- 172.21.15.111:0/3257221390 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42000cee0 con 0x7fc4240cd9f0 2023-03-29T10:57:59.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.923+0000 7fc41a7fc700 1 -- 172.21.15.111:0/3257221390 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42000f040 con 0x7fc4240cd9f0 2023-03-29T10:57:59.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.923+0000 7fc42c1ff700 1 -- 172.21.15.111:0/3257221390 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41c0094a0 con 0x7fc4240cd9f0 2023-03-29T10:57:59.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.923+0000 7fc41a7fc700 1 -- 172.21.15.111:0/3257221390 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420014680 con 0x7fc4240cd9f0 2023-03-29T10:57:59.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.924+0000 7fc42c1ff700 1 -- 172.21.15.111:0/3257221390 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4240ccf70 con 0x7fc4240cd9f0 2023-03-29T10:57:59.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.925+0000 7fc42c1ff700 1 -- 172.21.15.111:0/3257221390 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4240ccc90 con 0x7fc4240cd9f0 2023-03-29T10:57:59.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.925+0000 7fc41a7fc700 1 -- 172.21.15.111:0/3257221390 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fc4200147e0 con 0x7fc4240cd9f0 2023-03-29T10:57:59.926 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.926+0000 7fc41a7fc700 1 --2- 172.21.15.111:0/3257221390 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc408069de0 0x7fc40806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:57:59.926 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.926+0000 7fc41a7fc700 1 -- 172.21.15.111:0/3257221390 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc420089120 con 0x7fc4240cd9f0 2023-03-29T10:57:59.926 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.927+0000 7fc42979a700 1 --2- 172.21.15.111:0/3257221390 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc408069de0 0x7fc40806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:57:59.926 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.927+0000 7fc42979a700 1 --2- 172.21.15.111:0/3257221390 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc408069de0 0x7fc40806c290 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc424155c40 tx=0x7fc414009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:57:59.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:57:59.931+0000 7fc41a7fc700 1 -- 172.21.15.111:0/3257221390 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fc420058100 con 0x7fc4240cd9f0 2023-03-29T10:58:00.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.118+0000 7fc42c1ff700 1 -- 172.21.15.111:0/3257221390 --> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc424159890 con 0x7fc408069de0 2023-03-29T10:58:00.128 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.128+0000 7fc41a7fc700 1 -- 172.21.15.111:0/3257221390 <== mgr.4103 v2:172.21.15.111:6832/110141 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367293 (secure 0 0 0) 0x7fc424159890 con 0x7fc408069de0 2023-03-29T10:58:00.133 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:00.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.133+0000 7fc42c1ff700 1 -- 172.21.15.111:0/3257221390 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc408069de0 msgr2=0x7fc40806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:00.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.133+0000 7fc42c1ff700 1 --2- 172.21.15.111:0/3257221390 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc408069de0 0x7fc40806c290 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc424155c40 tx=0x7fc414009040 comp rx=0 tx=0).stop 2023-03-29T10:58:00.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.133+0000 7fc42c1ff700 1 -- 172.21.15.111:0/3257221390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4240cd9f0 msgr2=0x7fc4240ac180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:00.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.133+0000 7fc42c1ff700 1 --2- 172.21.15.111:0/3257221390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4240cd9f0 0x7fc4240ac180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc42000efc0 tx=0x7fc42000c3a0 comp rx=0 tx=0).stop 2023-03-29T10:58:00.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.133+0000 7fc42c1ff700 1 -- 172.21.15.111:0/3257221390 shutdown_connections 2023-03-29T10:58:00.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.133+0000 7fc42c1ff700 1 --2- 172.21.15.111:0/3257221390 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc408069de0 0x7fc40806c290 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:00.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.133+0000 7fc42c1ff700 1 --2- 172.21.15.111:0/3257221390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc4240cd9f0 0x7fc4240ac180 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:00.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.133+0000 7fc42c1ff700 1 --2- 172.21.15.111:0/3257221390 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc42415dce0 0x7fc4240c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:00.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.133+0000 7fc42c1ff700 1 --2- 172.21.15.111:0/3257221390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4241594c0 0x7fc42414e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:00.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.133+0000 7fc42c1ff700 1 -- 172.21.15.111:0/3257221390 >> 172.21.15.111:0/3257221390 conn(0x7fc4240cb6c0 msgr2=0x7fc4240ae680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:00.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.133+0000 7fc42c1ff700 1 -- 172.21.15.111:0/3257221390 shutdown_connections 2023-03-29T10:58:00.136 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.133+0000 7fc42c1ff700 1 -- 172.21.15.111:0/3257221390 wait complete. 2023-03-29T10:58:00.136 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2023-03-29T10:58:00.150 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2023-03-29T10:57:58.132508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594966,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67584,"kb_used_data":3584,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":754907136,"statfs":{"total":773094113280,"available":773024907264,"internally_reserved":0,"allocated":3670016,"data_stored":2185270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003141"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.845704+0000","last_change":"2023-03-29T10:57:27.854795+0000","last_active":"2023-03-29T10:57:31.845704+0000","last_peered":"2023-03-29T10:57:31.845704+0000","last_clean":"2023-03-29T10:57:31.845704+0000","last_became_active":"2023-03-29T10:57:27.854521+0000","last_became_peered":"2023-03-29T10:57:27.854521+0000","last_unstale":"2023-03-29T10:57:31.845704+0000","last_undegraded":"2023-03-29T10:57:31.845704+0000","last_fullsized":"2023-03-29T10:57:31.845704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:40:19.611213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.862071+0000","last_change":"2023-03-29T10:57:27.850091+0000","last_active":"2023-03-29T10:57:31.862071+0000","last_peered":"2023-03-29T10:57:31.862071+0000","last_clean":"2023-03-29T10:57:31.862071+0000","last_became_active":"2023-03-29T10:57:27.849795+0000","last_became_peered":"2023-03-29T10:57:27.849795+0000","last_unstale":"2023-03-29T10:57:31.862071+0000","last_undegraded":"2023-03-29T10:57:31.862071+0000","last_fullsized":"2023-03-29T10:57:31.862071+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:36:09.200571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853118+0000","last_change":"2023-03-29T10:57:27.864238+0000","last_active":"2023-03-29T10:57:31.853118+0000","last_peered":"2023-03-29T10:57:31.853118+0000","last_clean":"2023-03-29T10:57:31.853118+0000","last_became_active":"2023-03-29T10:57:27.863878+0000","last_became_peered":"2023-03-29T10:57:27.863878+0000","last_unstale":"2023-03-29T10:57:31.853118+0000","last_undegraded":"2023-03-29T10:57:31.853118+0000","last_fullsized":"2023-03-29T10:57:31.853118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:01:58.615750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854397+0000","last_change":"2023-03-29T10:57:27.861054+0000","last_active":"2023-03-29T10:57:31.854397+0000","last_peered":"2023-03-29T10:57:31.854397+0000","last_clean":"2023-03-29T10:57:31.854397+0000","last_became_active":"2023-03-29T10:57:27.859547+0000","last_became_peered":"2023-03-29T10:57:27.859547+0000","last_unstale":"2023-03-29T10:57:31.854397+0000","last_undegraded":"2023-03-29T10:57:31.854397+0000","last_fullsized":"2023-03-29T10:57:31.854397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:24:00.034416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854082+0000","last_change":"2023-03-29T10:57:27.855728+0000","last_active":"2023-03-29T10:57:31.854082+0000","last_peered":"2023-03-29T10:57:31.854082+0000","last_clean":"2023-03-29T10:57:31.854082+0000","last_became_active":"2023-03-29T10:57:27.855438+0000","last_became_peered":"2023-03-29T10:57:27.855438+0000","last_unstale":"2023-03-29T10:57:31.854082+0000","last_undegraded":"2023-03-29T10:57:31.854082+0000","last_fullsized":"2023-03-29T10:57:31.854082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:33:37.107795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852905+0000","last_change":"2023-03-29T10:57:27.861795+0000","last_active":"2023-03-29T10:57:31.852905+0000","last_peered":"2023-03-29T10:57:31.852905+0000","last_clean":"2023-03-29T10:57:31.852905+0000","last_became_active":"2023-03-29T10:57:27.860751+0000","last_became_peered":"2023-03-29T10:57:27.860751+0000","last_unstale":"2023-03-29T10:57:31.852905+0000","last_undegraded":"2023-03-29T10:57:31.852905+0000","last_fullsized":"2023-03-29T10:57:31.852905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:41.882164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847170+0000","last_change":"2023-03-29T10:57:27.855143+0000","last_active":"2023-03-29T10:57:31.847170+0000","last_peered":"2023-03-29T10:57:31.847170+0000","last_clean":"2023-03-29T10:57:31.847170+0000","last_became_active":"2023-03-29T10:57:27.854804+0000","last_became_peered":"2023-03-29T10:57:27.854804+0000","last_unstale":"2023-03-29T10:57:31.847170+0000","last_undegraded":"2023-03-29T10:57:31.847170+0000","last_fullsized":"2023-03-29T10:57:31.847170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:24:08.253556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861416+0000","last_change":"2023-03-29T10:57:27.864339+0000","last_active":"2023-03-29T10:57:31.861416+0000","last_peered":"2023-03-29T10:57:31.861416+0000","last_clean":"2023-03-29T10:57:31.861416+0000","last_became_active":"2023-03-29T10:57:27.864034+0000","last_became_peered":"2023-03-29T10:57:27.864034+0000","last_unstale":"2023-03-29T10:57:31.861416+0000","last_undegraded":"2023-03-29T10:57:31.861416+0000","last_fullsized":"2023-03-29T10:57:31.861416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:10:11.279944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851971+0000","last_change":"2023-03-29T10:57:27.849118+0000","last_active":"2023-03-29T10:57:31.851971+0000","last_peered":"2023-03-29T10:57:31.851971+0000","last_clean":"2023-03-29T10:57:31.851971+0000","last_became_active":"2023-03-29T10:57:27.848724+0000","last_became_peered":"2023-03-29T10:57:27.848724+0000","last_unstale":"2023-03-29T10:57:31.851971+0000","last_undegraded":"2023-03-29T10:57:31.851971+0000","last_fullsized":"2023-03-29T10:57:31.851971+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:36:09.200571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.862264+0000","last_change":"2023-03-29T10:57:27.848373+0000","last_active":"2023-03-29T10:57:31.862264+0000","last_peered":"2023-03-29T10:57:31.862264+0000","last_clean":"2023-03-29T10:57:31.862264+0000","last_became_active":"2023-03-29T10:57:27.847792+0000","last_became_peered":"2023-03-29T10:57:27.847792+0000","last_unstale":"2023-03-29T10:57:31.862264+0000","last_undegraded":"2023-03-29T10:57:31.862264+0000","last_fullsized":"2023-03-29T10:57:31.862264+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:46:29.403965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853207+0000","last_change":"2023-03-29T10:57:27.864182+0000","last_active":"2023-03-29T10:57:31.853207+0000","last_peered":"2023-03-29T10:57:31.853207+0000","last_clean":"2023-03-29T10:57:31.853207+0000","last_became_active":"2023-03-29T10:57:27.863567+0000","last_became_peered":"2023-03-29T10:57:27.863567+0000","last_unstale":"2023-03-29T10:57:31.853207+0000","last_undegraded":"2023-03-29T10:57:31.853207+0000","last_fullsized":"2023-03-29T10:57:31.853207+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:52:36.730410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.845640+0000","last_change":"2023-03-29T10:57:27.863745+0000","last_active":"2023-03-29T10:57:31.845640+0000","last_peered":"2023-03-29T10:57:31.845640+0000","last_clean":"2023-03-29T10:57:31.845640+0000","last_became_active":"2023-03-29T10:57:27.863377+0000","last_became_peered":"2023-03-29T10:57:27.863377+0000","last_unstale":"2023-03-29T10:57:31.845640+0000","last_undegraded":"2023-03-29T10:57:31.845640+0000","last_fullsized":"2023-03-29T10:57:31.845640+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:53:39.143611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846141+0000","last_change":"2023-03-29T10:57:27.856601+0000","last_active":"2023-03-29T10:57:31.846141+0000","last_peered":"2023-03-29T10:57:31.846141+0000","last_clean":"2023-03-29T10:57:31.846141+0000","last_became_active":"2023-03-29T10:57:27.856323+0000","last_became_peered":"2023-03-29T10:57:27.856323+0000","last_unstale":"2023-03-29T10:57:31.846141+0000","last_undegraded":"2023-03-29T10:57:31.846141+0000","last_fullsized":"2023-03-29T10:57:31.846141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:12:15.104266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.863135+0000","last_change":"2023-03-29T10:57:27.850723+0000","last_active":"2023-03-29T10:57:31.863135+0000","last_peered":"2023-03-29T10:57:31.863135+0000","last_clean":"2023-03-29T10:57:31.863135+0000","last_became_active":"2023-03-29T10:57:27.844480+0000","last_became_peered":"2023-03-29T10:57:27.844480+0000","last_unstale":"2023-03-29T10:57:31.863135+0000","last_undegraded":"2023-03-29T10:57:31.863135+0000","last_fullsized":"2023-03-29T10:57:31.863135+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:07:19.154573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847070+0000","last_change":"2023-03-29T10:57:27.855233+0000","last_active":"2023-03-29T10:57:31.847070+0000","last_peered":"2023-03-29T10:57:31.847070+0000","last_clean":"2023-03-29T10:57:31.847070+0000","last_became_active":"2023-03-29T10:57:27.854970+0000","last_became_peered":"2023-03-29T10:57:27.854970+0000","last_unstale":"2023-03-29T10:57:31.847070+0000","last_undegraded":"2023-03-29T10:57:31.847070+0000","last_fullsized":"2023-03-29T10:57:31.847070+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:46:29.403965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860076+0000","last_change":"2023-03-29T10:57:27.860037+0000","last_active":"2023-03-29T10:57:31.860076+0000","last_peered":"2023-03-29T10:57:31.860076+0000","last_clean":"2023-03-29T10:57:31.860076+0000","last_became_active":"2023-03-29T10:57:27.859764+0000","last_became_peered":"2023-03-29T10:57:27.859764+0000","last_unstale":"2023-03-29T10:57:31.860076+0000","last_undegraded":"2023-03-29T10:57:31.860076+0000","last_fullsized":"2023-03-29T10:57:31.860076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:25:48.508072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851770+0000","last_change":"2023-03-29T10:57:27.848952+0000","last_active":"2023-03-29T10:57:31.851770+0000","last_peered":"2023-03-29T10:57:31.851770+0000","last_clean":"2023-03-29T10:57:31.851770+0000","last_became_active":"2023-03-29T10:57:27.848390+0000","last_became_peered":"2023-03-29T10:57:27.848390+0000","last_unstale":"2023-03-29T10:57:31.851770+0000","last_undegraded":"2023-03-29T10:57:31.851770+0000","last_fullsized":"2023-03-29T10:57:31.851770+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:07:19.154573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854647+0000","last_change":"2023-03-29T10:57:27.862653+0000","last_active":"2023-03-29T10:57:31.854647+0000","last_peered":"2023-03-29T10:57:31.854647+0000","last_clean":"2023-03-29T10:57:31.854647+0000","last_became_active":"2023-03-29T10:57:27.862108+0000","last_became_peered":"2023-03-29T10:57:27.862108+0000","last_unstale":"2023-03-29T10:57:31.854647+0000","last_undegraded":"2023-03-29T10:57:31.854647+0000","last_fullsized":"2023-03-29T10:57:31.854647+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:06.878410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846508+0000","last_change":"2023-03-29T10:57:27.859887+0000","last_active":"2023-03-29T10:57:31.846508+0000","last_peered":"2023-03-29T10:57:31.846508+0000","last_clean":"2023-03-29T10:57:31.846508+0000","last_became_active":"2023-03-29T10:57:27.859632+0000","last_became_peered":"2023-03-29T10:57:27.859632+0000","last_unstale":"2023-03-29T10:57:31.846508+0000","last_undegraded":"2023-03-29T10:57:31.846508+0000","last_fullsized":"2023-03-29T10:57:31.846508+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:17.321564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854180+0000","last_change":"2023-03-29T10:57:27.865228+0000","last_active":"2023-03-29T10:57:31.854180+0000","last_peered":"2023-03-29T10:57:31.854180+0000","last_clean":"2023-03-29T10:57:31.854180+0000","last_became_active":"2023-03-29T10:57:27.864919+0000","last_became_peered":"2023-03-29T10:57:27.864919+0000","last_unstale":"2023-03-29T10:57:31.854180+0000","last_undegraded":"2023-03-29T10:57:31.854180+0000","last_fullsized":"2023-03-29T10:57:31.854180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:44:47.553332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846978+0000","last_change":"2023-03-29T10:57:27.853377+0000","last_active":"2023-03-29T10:57:31.846978+0000","last_peered":"2023-03-29T10:57:31.846978+0000","last_clean":"2023-03-29T10:57:31.846978+0000","last_became_active":"2023-03-29T10:57:27.853039+0000","last_became_peered":"2023-03-29T10:57:27.853039+0000","last_unstale":"2023-03-29T10:57:31.846978+0000","last_undegraded":"2023-03-29T10:57:31.846978+0000","last_fullsized":"2023-03-29T10:57:31.846978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:23:24.586546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851362+0000","last_change":"2023-03-29T10:57:27.849931+0000","last_active":"2023-03-29T10:57:31.851362+0000","last_peered":"2023-03-29T10:57:31.851362+0000","last_clean":"2023-03-29T10:57:31.851362+0000","last_became_active":"2023-03-29T10:57:27.849662+0000","last_became_peered":"2023-03-29T10:57:27.849662+0000","last_unstale":"2023-03-29T10:57:31.851362+0000","last_undegraded":"2023-03-29T10:57:31.851362+0000","last_fullsized":"2023-03-29T10:57:31.851362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:25:48.508072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852622+0000","last_change":"2023-03-29T10:57:27.850427+0000","last_active":"2023-03-29T10:57:31.852622+0000","last_peered":"2023-03-29T10:57:31.852622+0000","last_clean":"2023-03-29T10:57:31.852622+0000","last_became_active":"2023-03-29T10:57:27.850140+0000","last_became_peered":"2023-03-29T10:57:27.850140+0000","last_unstale":"2023-03-29T10:57:31.852622+0000","last_undegraded":"2023-03-29T10:57:31.852622+0000","last_fullsized":"2023-03-29T10:57:31.852622+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:01:58.615750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847710+0000","last_change":"2023-03-29T10:57:27.854437+0000","last_active":"2023-03-29T10:57:31.847710+0000","last_peered":"2023-03-29T10:57:31.847710+0000","last_clean":"2023-03-29T10:57:31.847710+0000","last_became_active":"2023-03-29T10:57:27.853876+0000","last_became_peered":"2023-03-29T10:57:27.853876+0000","last_unstale":"2023-03-29T10:57:31.847710+0000","last_undegraded":"2023-03-29T10:57:31.847710+0000","last_fullsized":"2023-03-29T10:57:31.847710+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:33:37.107795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851569+0000","last_change":"2023-03-29T10:57:27.856343+0000","last_active":"2023-03-29T10:57:31.851569+0000","last_peered":"2023-03-29T10:57:31.851569+0000","last_clean":"2023-03-29T10:57:31.851569+0000","last_became_active":"2023-03-29T10:57:27.855843+0000","last_became_peered":"2023-03-29T10:57:27.855843+0000","last_unstale":"2023-03-29T10:57:31.851569+0000","last_undegraded":"2023-03-29T10:57:31.851569+0000","last_fullsized":"2023-03-29T10:57:31.851569+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:46:29.403965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852481+0000","last_change":"2023-03-29T10:57:27.846853+0000","last_active":"2023-03-29T10:57:31.852481+0000","last_peered":"2023-03-29T10:57:31.852481+0000","last_clean":"2023-03-29T10:57:31.852481+0000","last_became_active":"2023-03-29T10:57:27.846503+0000","last_became_peered":"2023-03-29T10:57:27.846503+0000","last_unstale":"2023-03-29T10:57:31.852481+0000","last_undegraded":"2023-03-29T10:57:31.852481+0000","last_fullsized":"2023-03-29T10:57:31.852481+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:37:35.403032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.864802+0000","last_change":"2023-03-29T10:57:27.850277+0000","last_active":"2023-03-29T10:57:31.864802+0000","last_peered":"2023-03-29T10:57:31.864802+0000","last_clean":"2023-03-29T10:57:31.864802+0000","last_became_active":"2023-03-29T10:57:27.849726+0000","last_became_peered":"2023-03-29T10:57:27.849726+0000","last_unstale":"2023-03-29T10:57:31.864802+0000","last_undegraded":"2023-03-29T10:57:31.864802+0000","last_fullsized":"2023-03-29T10:57:31.864802+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:25:48.508072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340375+0000","last_change":"2023-03-29T10:57:31.874510+0000","last_active":"2023-03-29T10:57:32.340375+0000","last_peered":"2023-03-29T10:57:32.340375+0000","last_clean":"2023-03-29T10:57:32.340375+0000","last_became_active":"2023-03-29T10:57:31.874055+0000","last_became_peered":"2023-03-29T10:57:31.874055+0000","last_unstale":"2023-03-29T10:57:32.340375+0000","last_undegraded":"2023-03-29T10:57:32.340375+0000","last_fullsized":"2023-03-29T10:57:32.340375+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:53:20.999237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852831+0000","last_change":"2023-03-29T10:57:27.366688+0000","last_active":"2023-03-29T10:57:31.852831+0000","last_peered":"2023-03-29T10:57:31.852831+0000","last_clean":"2023-03-29T10:57:31.852831+0000","last_became_active":"2023-03-29T10:57:27.366396+0000","last_became_peered":"2023-03-29T10:57:27.366396+0000","last_unstale":"2023-03-29T10:57:31.852831+0000","last_undegraded":"2023-03-29T10:57:31.852831+0000","last_fullsized":"2023-03-29T10:57:31.852831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:16.312585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853217+0000","last_change":"2023-03-29T10:57:27.851695+0000","last_active":"2023-03-29T10:57:31.853217+0000","last_peered":"2023-03-29T10:57:31.853217+0000","last_clean":"2023-03-29T10:57:31.853217+0000","last_became_active":"2023-03-29T10:57:27.851344+0000","last_became_peered":"2023-03-29T10:57:27.851344+0000","last_unstale":"2023-03-29T10:57:31.853217+0000","last_undegraded":"2023-03-29T10:57:31.853217+0000","last_fullsized":"2023-03-29T10:57:31.853217+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:53:38.186916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606792+0000","last_change":"2023-03-29T10:57:31.878011+0000","last_active":"2023-03-29T10:57:32.606792+0000","last_peered":"2023-03-29T10:57:32.606792+0000","last_clean":"2023-03-29T10:57:32.606792+0000","last_became_active":"2023-03-29T10:57:31.877524+0000","last_became_peered":"2023-03-29T10:57:31.877524+0000","last_unstale":"2023-03-29T10:57:32.606792+0000","last_undegraded":"2023-03-29T10:57:32.606792+0000","last_fullsized":"2023-03-29T10:57:32.606792+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:44:51.056937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.850681+0000","last_change":"2023-03-29T10:57:27.368907+0000","last_active":"2023-03-29T10:57:31.850681+0000","last_peered":"2023-03-29T10:57:31.850681+0000","last_clean":"2023-03-29T10:57:31.850681+0000","last_became_active":"2023-03-29T10:57:27.367137+0000","last_became_peered":"2023-03-29T10:57:27.367137+0000","last_unstale":"2023-03-29T10:57:31.850681+0000","last_undegraded":"2023-03-29T10:57:31.850681+0000","last_fullsized":"2023-03-29T10:57:31.850681+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:12:14.095287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853885+0000","last_change":"2023-03-29T10:57:27.844281+0000","last_active":"2023-03-29T10:57:31.853885+0000","last_peered":"2023-03-29T10:57:31.853885+0000","last_clean":"2023-03-29T10:57:31.853885+0000","last_became_active":"2023-03-29T10:57:27.844012+0000","last_became_peered":"2023-03-29T10:57:27.844012+0000","last_unstale":"2023-03-29T10:57:31.853885+0000","last_undegraded":"2023-03-29T10:57:31.853885+0000","last_fullsized":"2023-03-29T10:57:31.853885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:23:24.586546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.336612+0000","last_change":"2023-03-29T10:57:31.872397+0000","last_active":"2023-03-29T10:57:32.336612+0000","last_peered":"2023-03-29T10:57:32.336612+0000","last_clean":"2023-03-29T10:57:32.336612+0000","last_became_active":"2023-03-29T10:57:31.872073+0000","last_became_peered":"2023-03-29T10:57:31.872073+0000","last_unstale":"2023-03-29T10:57:32.336612+0000","last_undegraded":"2023-03-29T10:57:32.336612+0000","last_fullsized":"2023-03-29T10:57:32.336612+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:24:14.450310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849730+0000","last_change":"2023-03-29T10:57:27.362811+0000","last_active":"2023-03-29T10:57:31.849730+0000","last_peered":"2023-03-29T10:57:31.849730+0000","last_clean":"2023-03-29T10:57:31.849730+0000","last_became_active":"2023-03-29T10:57:27.362313+0000","last_became_peered":"2023-03-29T10:57:27.362313+0000","last_unstale":"2023-03-29T10:57:31.849730+0000","last_undegraded":"2023-03-29T10:57:31.849730+0000","last_fullsized":"2023-03-29T10:57:31.849730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:57:18.786945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.857696+0000","last_change":"2023-03-29T10:57:27.853328+0000","last_active":"2023-03-29T10:57:31.857696+0000","last_peered":"2023-03-29T10:57:31.857696+0000","last_clean":"2023-03-29T10:57:31.857696+0000","last_became_active":"2023-03-29T10:57:27.852980+0000","last_became_peered":"2023-03-29T10:57:27.852980+0000","last_unstale":"2023-03-29T10:57:31.857696+0000","last_undegraded":"2023-03-29T10:57:31.857696+0000","last_fullsized":"2023-03-29T10:57:31.857696+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:35:25.612159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606385+0000","last_change":"2023-03-29T10:57:31.880406+0000","last_active":"2023-03-29T10:57:32.606385+0000","last_peered":"2023-03-29T10:57:32.606385+0000","last_clean":"2023-03-29T10:57:32.606385+0000","last_became_active":"2023-03-29T10:57:31.879910+0000","last_became_peered":"2023-03-29T10:57:31.879910+0000","last_unstale":"2023-03-29T10:57:32.606385+0000","last_undegraded":"2023-03-29T10:57:32.606385+0000","last_fullsized":"2023-03-29T10:57:32.606385+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:50:03.793655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852066+0000","last_change":"2023-03-29T10:57:27.361203+0000","last_active":"2023-03-29T10:57:31.852066+0000","last_peered":"2023-03-29T10:57:31.852066+0000","last_clean":"2023-03-29T10:57:31.852066+0000","last_became_active":"2023-03-29T10:57:27.360905+0000","last_became_peered":"2023-03-29T10:57:27.360905+0000","last_unstale":"2023-03-29T10:57:31.852066+0000","last_undegraded":"2023-03-29T10:57:31.852066+0000","last_fullsized":"2023-03-29T10:57:31.852066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:53:38.134633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849652+0000","last_change":"2023-03-29T10:57:27.848599+0000","last_active":"2023-03-29T10:57:31.849652+0000","last_peered":"2023-03-29T10:57:31.849652+0000","last_clean":"2023-03-29T10:57:31.849652+0000","last_became_active":"2023-03-29T10:57:27.848300+0000","last_became_peered":"2023-03-29T10:57:27.848300+0000","last_unstale":"2023-03-29T10:57:31.849652+0000","last_undegraded":"2023-03-29T10:57:31.849652+0000","last_fullsized":"2023-03-29T10:57:31.849652+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:44:47.553332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340886+0000","last_change":"2023-03-29T10:57:31.875820+0000","last_active":"2023-03-29T10:57:32.340886+0000","last_peered":"2023-03-29T10:57:32.340886+0000","last_clean":"2023-03-29T10:57:32.340886+0000","last_became_active":"2023-03-29T10:57:31.875285+0000","last_became_peered":"2023-03-29T10:57:31.875285+0000","last_unstale":"2023-03-29T10:57:32.340886+0000","last_undegraded":"2023-03-29T10:57:32.340886+0000","last_fullsized":"2023-03-29T10:57:32.340886+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:05:02.661508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.862158+0000","last_change":"2023-03-29T10:57:27.369635+0000","last_active":"2023-03-29T10:57:31.862158+0000","last_peered":"2023-03-29T10:57:31.862158+0000","last_clean":"2023-03-29T10:57:31.862158+0000","last_became_active":"2023-03-29T10:57:27.369232+0000","last_became_peered":"2023-03-29T10:57:27.369232+0000","last_unstale":"2023-03-29T10:57:31.862158+0000","last_undegraded":"2023-03-29T10:57:31.862158+0000","last_fullsized":"2023-03-29T10:57:31.862158+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:45:12.326563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852705+0000","last_change":"2023-03-29T10:57:27.841973+0000","last_active":"2023-03-29T10:57:31.852705+0000","last_peered":"2023-03-29T10:57:31.852705+0000","last_clean":"2023-03-29T10:57:31.852705+0000","last_became_active":"2023-03-29T10:57:27.841668+0000","last_became_peered":"2023-03-29T10:57:27.841668+0000","last_unstale":"2023-03-29T10:57:31.852705+0000","last_undegraded":"2023-03-29T10:57:31.852705+0000","last_fullsized":"2023-03-29T10:57:31.852705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:25:48.508072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.608208+0000","last_change":"2023-03-29T10:57:31.867435+0000","last_active":"2023-03-29T10:57:32.608208+0000","last_peered":"2023-03-29T10:57:32.608208+0000","last_clean":"2023-03-29T10:57:32.608208+0000","last_became_active":"2023-03-29T10:57:31.867024+0000","last_became_peered":"2023-03-29T10:57:31.867024+0000","last_unstale":"2023-03-29T10:57:32.608208+0000","last_undegraded":"2023-03-29T10:57:32.608208+0000","last_fullsized":"2023-03-29T10:57:32.608208+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:33:59.083704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846915+0000","last_change":"2023-03-29T10:57:27.367580+0000","last_active":"2023-03-29T10:57:31.846915+0000","last_peered":"2023-03-29T10:57:31.846915+0000","last_clean":"2023-03-29T10:57:31.846915+0000","last_became_active":"2023-03-29T10:57:27.367297+0000","last_became_peered":"2023-03-29T10:57:27.367297+0000","last_unstale":"2023-03-29T10:57:31.846915+0000","last_undegraded":"2023-03-29T10:57:31.846915+0000","last_fullsized":"2023-03-29T10:57:31.846915+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:35:24.603180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851381+0000","last_change":"2023-03-29T10:57:27.868674+0000","last_active":"2023-03-29T10:57:31.851381+0000","last_peered":"2023-03-29T10:57:31.851381+0000","last_clean":"2023-03-29T10:57:31.851381+0000","last_became_active":"2023-03-29T10:57:27.868480+0000","last_became_peered":"2023-03-29T10:57:27.868480+0000","last_unstale":"2023-03-29T10:57:31.851381+0000","last_undegraded":"2023-03-29T10:57:31.851381+0000","last_fullsized":"2023-03-29T10:57:31.851381+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:24:08.253556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.543504+0000","last_change":"2023-03-29T10:57:31.865358+0000","last_active":"2023-03-29T10:57:32.543504+0000","last_peered":"2023-03-29T10:57:32.543504+0000","last_clean":"2023-03-29T10:57:32.543504+0000","last_became_active":"2023-03-29T10:57:31.864834+0000","last_became_peered":"2023-03-29T10:57:31.864834+0000","last_unstale":"2023-03-29T10:57:32.543504+0000","last_undegraded":"2023-03-29T10:57:32.543504+0000","last_fullsized":"2023-03-29T10:57:32.543504+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:40:55.333034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:44.820956+0000","last_change":"2023-03-29T10:57:27.361696+0000","last_active":"2023-03-29T10:57:44.820956+0000","last_peered":"2023-03-29T10:57:44.820956+0000","last_clean":"2023-03-29T10:57:44.820956+0000","last_became_active":"2023-03-29T10:57:27.360021+0000","last_became_peered":"2023-03-29T10:57:27.360021+0000","last_unstale":"2023-03-29T10:57:44.820956+0000","last_undegraded":"2023-03-29T10:57:44.820956+0000","last_fullsized":"2023-03-29T10:57:44.820956+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:57:18.786945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853468+0000","last_change":"2023-03-29T10:57:27.860824+0000","last_active":"2023-03-29T10:57:31.853468+0000","last_peered":"2023-03-29T10:57:31.853468+0000","last_clean":"2023-03-29T10:57:31.853468+0000","last_became_active":"2023-03-29T10:57:27.860139+0000","last_became_peered":"2023-03-29T10:57:27.860139+0000","last_unstale":"2023-03-29T10:57:31.853468+0000","last_undegraded":"2023-03-29T10:57:31.853468+0000","last_fullsized":"2023-03-29T10:57:31.853468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:10:11.279944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606214+0000","last_change":"2023-03-29T10:57:31.869234+0000","last_active":"2023-03-29T10:57:32.606214+0000","last_peered":"2023-03-29T10:57:32.606214+0000","last_clean":"2023-03-29T10:57:32.606214+0000","last_became_active":"2023-03-29T10:57:31.868479+0000","last_became_peered":"2023-03-29T10:57:31.868479+0000","last_unstale":"2023-03-29T10:57:32.606214+0000","last_undegraded":"2023-03-29T10:57:32.606214+0000","last_fullsized":"2023-03-29T10:57:32.606214+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:29:07.043419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846875+0000","last_change":"2023-03-29T10:57:27.363811+0000","last_active":"2023-03-29T10:57:31.846875+0000","last_peered":"2023-03-29T10:57:31.846875+0000","last_clean":"2023-03-29T10:57:31.846875+0000","last_became_active":"2023-03-29T10:57:27.362755+0000","last_became_peered":"2023-03-29T10:57:27.362755+0000","last_unstale":"2023-03-29T10:57:31.846875+0000","last_undegraded":"2023-03-29T10:57:31.846875+0000","last_fullsized":"2023-03-29T10:57:31.846875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:45:12.326563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849249+0000","last_change":"2023-03-29T10:57:27.848024+0000","last_active":"2023-03-29T10:57:31.849249+0000","last_peered":"2023-03-29T10:57:31.849249+0000","last_clean":"2023-03-29T10:57:31.849249+0000","last_became_active":"2023-03-29T10:57:27.847742+0000","last_became_peered":"2023-03-29T10:57:27.847742+0000","last_unstale":"2023-03-29T10:57:31.849249+0000","last_undegraded":"2023-03-29T10:57:31.849249+0000","last_fullsized":"2023-03-29T10:57:31.849249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:07:19.154573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.608195+0000","last_change":"2023-03-29T10:57:31.874176+0000","last_active":"2023-03-29T10:57:32.608195+0000","last_peered":"2023-03-29T10:57:32.608195+0000","last_clean":"2023-03-29T10:57:32.608195+0000","last_became_active":"2023-03-29T10:57:31.873748+0000","last_became_peered":"2023-03-29T10:57:31.873748+0000","last_unstale":"2023-03-29T10:57:32.608195+0000","last_undegraded":"2023-03-29T10:57:32.608195+0000","last_fullsized":"2023-03-29T10:57:32.608195+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:29:07.043419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851278+0000","last_change":"2023-03-29T10:57:27.353170+0000","last_active":"2023-03-29T10:57:31.851278+0000","last_peered":"2023-03-29T10:57:31.851278+0000","last_clean":"2023-03-29T10:57:31.851278+0000","last_became_active":"2023-03-29T10:57:27.352891+0000","last_became_peered":"2023-03-29T10:57:27.352891+0000","last_unstale":"2023-03-29T10:57:31.851278+0000","last_undegraded":"2023-03-29T10:57:31.851278+0000","last_fullsized":"2023-03-29T10:57:31.851278+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:42:03.146529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.864725+0000","last_change":"2023-03-29T10:57:27.850083+0000","last_active":"2023-03-29T10:57:31.864725+0000","last_peered":"2023-03-29T10:57:31.864725+0000","last_clean":"2023-03-29T10:57:31.864725+0000","last_became_active":"2023-03-29T10:57:27.849341+0000","last_became_peered":"2023-03-29T10:57:27.849341+0000","last_unstale":"2023-03-29T10:57:31.864725+0000","last_undegraded":"2023-03-29T10:57:31.864725+0000","last_fullsized":"2023-03-29T10:57:31.864725+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:41.882164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.608429+0000","last_change":"2023-03-29T10:57:31.874626+0000","last_active":"2023-03-29T10:57:32.608429+0000","last_peered":"2023-03-29T10:57:32.608429+0000","last_clean":"2023-03-29T10:57:32.608429+0000","last_became_active":"2023-03-29T10:57:31.874288+0000","last_became_peered":"2023-03-29T10:57:31.874288+0000","last_unstale":"2023-03-29T10:57:32.608429+0000","last_undegraded":"2023-03-29T10:57:32.608429+0000","last_fullsized":"2023-03-29T10:57:32.608429+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:10:08.859801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847314+0000","last_change":"2023-03-29T10:57:27.371046+0000","last_active":"2023-03-29T10:57:31.847314+0000","last_peered":"2023-03-29T10:57:31.847314+0000","last_clean":"2023-03-29T10:57:31.847314+0000","last_became_active":"2023-03-29T10:57:27.370703+0000","last_became_peered":"2023-03-29T10:57:27.370703+0000","last_unstale":"2023-03-29T10:57:31.847314+0000","last_undegraded":"2023-03-29T10:57:31.847314+0000","last_fullsized":"2023-03-29T10:57:31.847314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:00:38.166723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853955+0000","last_change":"2023-03-29T10:57:27.857805+0000","last_active":"2023-03-29T10:57:31.853955+0000","last_peered":"2023-03-29T10:57:31.853955+0000","last_clean":"2023-03-29T10:57:31.853955+0000","last_became_active":"2023-03-29T10:57:27.857520+0000","last_became_peered":"2023-03-29T10:57:27.857520+0000","last_unstale":"2023-03-29T10:57:31.853955+0000","last_undegraded":"2023-03-29T10:57:31.853955+0000","last_fullsized":"2023-03-29T10:57:31.853955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:24:08.253556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605664+0000","last_change":"2023-03-29T10:57:31.868093+0000","last_active":"2023-03-29T10:57:32.605664+0000","last_peered":"2023-03-29T10:57:32.605664+0000","last_clean":"2023-03-29T10:57:32.605664+0000","last_became_active":"2023-03-29T10:57:31.867516+0000","last_became_peered":"2023-03-29T10:57:31.867516+0000","last_unstale":"2023-03-29T10:57:32.605664+0000","last_undegraded":"2023-03-29T10:57:32.605664+0000","last_fullsized":"2023-03-29T10:57:32.605664+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:56:12.713461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852187+0000","last_change":"2023-03-29T10:57:27.366626+0000","last_active":"2023-03-29T10:57:31.852187+0000","last_peered":"2023-03-29T10:57:31.852187+0000","last_clean":"2023-03-29T10:57:31.852187+0000","last_became_active":"2023-03-29T10:57:27.366259+0000","last_became_peered":"2023-03-29T10:57:27.366259+0000","last_unstale":"2023-03-29T10:57:31.852187+0000","last_undegraded":"2023-03-29T10:57:31.852187+0000","last_fullsized":"2023-03-29T10:57:31.852187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:01:05.024349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852636+0000","last_change":"2023-03-29T10:57:27.842305+0000","last_active":"2023-03-29T10:57:31.852636+0000","last_peered":"2023-03-29T10:57:31.852636+0000","last_clean":"2023-03-29T10:57:31.852636+0000","last_became_active":"2023-03-29T10:57:27.841993+0000","last_became_peered":"2023-03-29T10:57:27.841993+0000","last_unstale":"2023-03-29T10:57:31.852636+0000","last_undegraded":"2023-03-29T10:57:31.852636+0000","last_fullsized":"2023-03-29T10:57:31.852636+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:41.882164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340179+0000","last_change":"2023-03-29T10:57:31.874901+0000","last_active":"2023-03-29T10:57:32.340179+0000","last_peered":"2023-03-29T10:57:32.340179+0000","last_clean":"2023-03-29T10:57:32.340179+0000","last_became_active":"2023-03-29T10:57:31.874412+0000","last_became_peered":"2023-03-29T10:57:31.874412+0000","last_unstale":"2023-03-29T10:57:32.340179+0000","last_undegraded":"2023-03-29T10:57:32.340179+0000","last_fullsized":"2023-03-29T10:57:32.340179+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:19:12.936732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847695+0000","last_change":"2023-03-29T10:57:27.363942+0000","last_active":"2023-03-29T10:57:31.847695+0000","last_peered":"2023-03-29T10:57:31.847695+0000","last_clean":"2023-03-29T10:57:31.847695+0000","last_became_active":"2023-03-29T10:57:27.363509+0000","last_became_peered":"2023-03-29T10:57:27.363509+0000","last_unstale":"2023-03-29T10:57:31.847695+0000","last_undegraded":"2023-03-29T10:57:31.847695+0000","last_fullsized":"2023-03-29T10:57:31.847695+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:23:59.025438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861310+0000","last_change":"2023-03-29T10:57:27.849632+0000","last_active":"2023-03-29T10:57:31.861310+0000","last_peered":"2023-03-29T10:57:31.861310+0000","last_clean":"2023-03-29T10:57:31.861310+0000","last_became_active":"2023-03-29T10:57:27.849341+0000","last_became_peered":"2023-03-29T10:57:27.849341+0000","last_unstale":"2023-03-29T10:57:31.861310+0000","last_undegraded":"2023-03-29T10:57:31.861310+0000","last_fullsized":"2023-03-29T10:57:31.861310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:01:58.615750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.604803+0000","last_change":"2023-03-29T10:57:31.864416+0000","last_active":"2023-03-29T10:57:32.604803+0000","last_peered":"2023-03-29T10:57:32.604803+0000","last_clean":"2023-03-29T10:57:32.604803+0000","last_became_active":"2023-03-29T10:57:31.864021+0000","last_became_peered":"2023-03-29T10:57:31.864021+0000","last_unstale":"2023-03-29T10:57:32.604803+0000","last_undegraded":"2023-03-29T10:57:32.604803+0000","last_fullsized":"2023-03-29T10:57:32.604803+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:44:51.056937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847268+0000","last_change":"2023-03-29T10:57:27.365113+0000","last_active":"2023-03-29T10:57:31.847268+0000","last_peered":"2023-03-29T10:57:31.847268+0000","last_clean":"2023-03-29T10:57:31.847268+0000","last_became_active":"2023-03-29T10:57:27.363593+0000","last_became_peered":"2023-03-29T10:57:27.363593+0000","last_unstale":"2023-03-29T10:57:31.847268+0000","last_undegraded":"2023-03-29T10:57:31.847268+0000","last_fullsized":"2023-03-29T10:57:31.847268+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:10:38.094232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860900+0000","last_change":"2023-03-29T10:57:27.861487+0000","last_active":"2023-03-29T10:57:31.860900+0000","last_peered":"2023-03-29T10:57:31.860900+0000","last_clean":"2023-03-29T10:57:31.860900+0000","last_became_active":"2023-03-29T10:57:27.861218+0000","last_became_peered":"2023-03-29T10:57:27.861218+0000","last_unstale":"2023-03-29T10:57:31.860900+0000","last_undegraded":"2023-03-29T10:57:31.860900+0000","last_fullsized":"2023-03-29T10:57:31.860900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:04:59.157904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.604937+0000","last_change":"2023-03-29T10:57:31.871267+0000","last_active":"2023-03-29T10:57:32.604937+0000","last_peered":"2023-03-29T10:57:32.604937+0000","last_clean":"2023-03-29T10:57:32.604937+0000","last_became_active":"2023-03-29T10:57:31.870814+0000","last_became_peered":"2023-03-29T10:57:31.870814+0000","last_unstale":"2023-03-29T10:57:32.604937+0000","last_undegraded":"2023-03-29T10:57:32.604937+0000","last_fullsized":"2023-03-29T10:57:32.604937+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:57:20.392149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847436+0000","last_change":"2023-03-29T10:57:27.367428+0000","last_active":"2023-03-29T10:57:31.847436+0000","last_peered":"2023-03-29T10:57:31.847436+0000","last_clean":"2023-03-29T10:57:31.847436+0000","last_became_active":"2023-03-29T10:57:27.367100+0000","last_became_peered":"2023-03-29T10:57:27.367100+0000","last_unstale":"2023-03-29T10:57:31.847436+0000","last_undegraded":"2023-03-29T10:57:31.847436+0000","last_fullsized":"2023-03-29T10:57:31.847436+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:52:59.436152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.858073+0000","last_change":"2023-03-29T10:57:27.863156+0000","last_active":"2023-03-29T10:57:31.858073+0000","last_peered":"2023-03-29T10:57:31.858073+0000","last_clean":"2023-03-29T10:57:31.858073+0000","last_became_active":"2023-03-29T10:57:27.862805+0000","last_became_peered":"2023-03-29T10:57:27.862805+0000","last_unstale":"2023-03-29T10:57:31.858073+0000","last_undegraded":"2023-03-29T10:57:31.858073+0000","last_fullsized":"2023-03-29T10:57:31.858073+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:32:05.407357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605019+0000","last_change":"2023-03-29T10:57:31.878616+0000","last_active":"2023-03-29T10:57:32.605019+0000","last_peered":"2023-03-29T10:57:32.605019+0000","last_clean":"2023-03-29T10:57:32.605019+0000","last_became_active":"2023-03-29T10:57:31.878248+0000","last_became_peered":"2023-03-29T10:57:31.878248+0000","last_unstale":"2023-03-29T10:57:32.605019+0000","last_undegraded":"2023-03-29T10:57:32.605019+0000","last_fullsized":"2023-03-29T10:57:32.605019+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:53:41.690520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861723+0000","last_change":"2023-03-29T10:57:27.370499+0000","last_active":"2023-03-29T10:57:31.861723+0000","last_peered":"2023-03-29T10:57:31.861723+0000","last_clean":"2023-03-29T10:57:31.861723+0000","last_became_active":"2023-03-29T10:57:27.370048+0000","last_became_peered":"2023-03-29T10:57:27.370048+0000","last_unstale":"2023-03-29T10:57:31.861723+0000","last_undegraded":"2023-03-29T10:57:31.861723+0000","last_fullsized":"2023-03-29T10:57:31.861723+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:52:59.436152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854114+0000","last_change":"2023-03-29T10:57:27.848004+0000","last_active":"2023-03-29T10:57:31.854114+0000","last_peered":"2023-03-29T10:57:31.854114+0000","last_clean":"2023-03-29T10:57:31.854114+0000","last_became_active":"2023-03-29T10:57:27.847731+0000","last_became_peered":"2023-03-29T10:57:27.847731+0000","last_unstale":"2023-03-29T10:57:31.854114+0000","last_undegraded":"2023-03-29T10:57:31.854114+0000","last_fullsized":"2023-03-29T10:57:31.854114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:41.882164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605202+0000","last_change":"2023-03-29T10:57:31.868289+0000","last_active":"2023-03-29T10:57:32.605202+0000","last_peered":"2023-03-29T10:57:32.605202+0000","last_clean":"2023-03-29T10:57:32.605202+0000","last_became_active":"2023-03-29T10:57:31.867905+0000","last_became_peered":"2023-03-29T10:57:31.867905+0000","last_unstale":"2023-03-29T10:57:32.605202+0000","last_undegraded":"2023-03-29T10:57:32.605202+0000","last_fullsized":"2023-03-29T10:57:32.605202+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:23:28.090151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851809+0000","last_change":"2023-03-29T10:57:27.368350+0000","last_active":"2023-03-29T10:57:31.851809+0000","last_peered":"2023-03-29T10:57:31.851809+0000","last_clean":"2023-03-29T10:57:31.851809+0000","last_became_active":"2023-03-29T10:57:27.367743+0000","last_became_peered":"2023-03-29T10:57:27.367743+0000","last_unstale":"2023-03-29T10:57:31.851809+0000","last_undegraded":"2023-03-29T10:57:31.851809+0000","last_fullsized":"2023-03-29T10:57:31.851809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:16.312585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853838+0000","last_change":"2023-03-29T10:57:27.862273+0000","last_active":"2023-03-29T10:57:31.853838+0000","last_peered":"2023-03-29T10:57:31.853838+0000","last_clean":"2023-03-29T10:57:31.853838+0000","last_became_active":"2023-03-29T10:57:27.861893+0000","last_became_peered":"2023-03-29T10:57:27.861893+0000","last_unstale":"2023-03-29T10:57:31.853838+0000","last_undegraded":"2023-03-29T10:57:31.853838+0000","last_fullsized":"2023-03-29T10:57:31.853838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:34:37.113840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.543869+0000","last_change":"2023-03-29T10:57:31.877007+0000","last_active":"2023-03-29T10:57:32.543869+0000","last_peered":"2023-03-29T10:57:32.543869+0000","last_clean":"2023-03-29T10:57:32.543869+0000","last_became_active":"2023-03-29T10:57:31.876649+0000","last_became_peered":"2023-03-29T10:57:31.876649+0000","last_unstale":"2023-03-29T10:57:32.543869+0000","last_undegraded":"2023-03-29T10:57:32.543869+0000","last_fullsized":"2023-03-29T10:57:32.543869+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:19:12.936732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.858178+0000","last_change":"2023-03-29T10:57:27.370469+0000","last_active":"2023-03-29T10:57:31.858178+0000","last_peered":"2023-03-29T10:57:31.858178+0000","last_clean":"2023-03-29T10:57:31.858178+0000","last_became_active":"2023-03-29T10:57:27.370194+0000","last_became_peered":"2023-03-29T10:57:27.370194+0000","last_unstale":"2023-03-29T10:57:31.858178+0000","last_undegraded":"2023-03-29T10:57:31.858178+0000","last_fullsized":"2023-03-29T10:57:31.858178+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:42:03.146529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853028+0000","last_change":"2023-03-29T10:57:27.863980+0000","last_active":"2023-03-29T10:57:31.853028+0000","last_peered":"2023-03-29T10:57:31.853028+0000","last_clean":"2023-03-29T10:57:31.853028+0000","last_became_active":"2023-03-29T10:57:27.863503+0000","last_became_peered":"2023-03-29T10:57:27.863503+0000","last_unstale":"2023-03-29T10:57:31.853028+0000","last_undegraded":"2023-03-29T10:57:31.853028+0000","last_fullsized":"2023-03-29T10:57:31.853028+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:36:09.200571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606340+0000","last_change":"2023-03-29T10:57:31.880859+0000","last_active":"2023-03-29T10:57:32.606340+0000","last_peered":"2023-03-29T10:57:32.606340+0000","last_clean":"2023-03-29T10:57:32.606340+0000","last_became_active":"2023-03-29T10:57:31.879872+0000","last_became_peered":"2023-03-29T10:57:31.879872+0000","last_unstale":"2023-03-29T10:57:32.606340+0000","last_undegraded":"2023-03-29T10:57:32.606340+0000","last_fullsized":"2023-03-29T10:57:32.606340+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:40:04.971172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860797+0000","last_change":"2023-03-29T10:57:27.359363+0000","last_active":"2023-03-29T10:57:31.860797+0000","last_peered":"2023-03-29T10:57:31.860797+0000","last_clean":"2023-03-29T10:57:31.860797+0000","last_became_active":"2023-03-29T10:57:27.359054+0000","last_became_peered":"2023-03-29T10:57:27.359054+0000","last_unstale":"2023-03-29T10:57:31.860797+0000","last_undegraded":"2023-03-29T10:57:31.860797+0000","last_fullsized":"2023-03-29T10:57:31.860797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:42:03.146529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854683+0000","last_change":"2023-03-29T10:57:27.862443+0000","last_active":"2023-03-29T10:57:31.854683+0000","last_peered":"2023-03-29T10:57:31.854683+0000","last_clean":"2023-03-29T10:57:31.854683+0000","last_became_active":"2023-03-29T10:57:27.861796+0000","last_became_peered":"2023-03-29T10:57:27.861796+0000","last_unstale":"2023-03-29T10:57:31.854683+0000","last_undegraded":"2023-03-29T10:57:31.854683+0000","last_fullsized":"2023-03-29T10:57:31.854683+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:50:00.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340645+0000","last_change":"2023-03-29T10:57:31.875674+0000","last_active":"2023-03-29T10:57:32.340645+0000","last_peered":"2023-03-29T10:57:32.340645+0000","last_clean":"2023-03-29T10:57:32.340645+0000","last_became_active":"2023-03-29T10:57:31.875240+0000","last_became_peered":"2023-03-29T10:57:31.875240+0000","last_unstale":"2023-03-29T10:57:32.340645+0000","last_undegraded":"2023-03-29T10:57:32.340645+0000","last_fullsized":"2023-03-29T10:57:32.340645+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:36:10.088665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849570+0000","last_change":"2023-03-29T10:57:27.366426+0000","last_active":"2023-03-29T10:57:31.849570+0000","last_peered":"2023-03-29T10:57:31.849570+0000","last_clean":"2023-03-29T10:57:31.849570+0000","last_became_active":"2023-03-29T10:57:27.366129+0000","last_became_peered":"2023-03-29T10:57:27.366129+0000","last_unstale":"2023-03-29T10:57:31.849570+0000","last_undegraded":"2023-03-29T10:57:31.849570+0000","last_fullsized":"2023-03-29T10:57:31.849570+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:42:03.146529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851728+0000","last_change":"2023-03-29T10:57:27.856138+0000","last_active":"2023-03-29T10:57:31.851728+0000","last_peered":"2023-03-29T10:57:31.851728+0000","last_clean":"2023-03-29T10:57:31.851728+0000","last_became_active":"2023-03-29T10:57:27.855392+0000","last_became_peered":"2023-03-29T10:57:27.855392+0000","last_unstale":"2023-03-29T10:57:31.851728+0000","last_undegraded":"2023-03-29T10:57:31.851728+0000","last_fullsized":"2023-03-29T10:57:31.851728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:12:10.758718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.336574+0000","last_change":"2023-03-29T10:57:31.872511+0000","last_active":"2023-03-29T10:57:32.336574+0000","last_peered":"2023-03-29T10:57:32.336574+0000","last_clean":"2023-03-29T10:57:32.336574+0000","last_became_active":"2023-03-29T10:57:31.872088+0000","last_became_peered":"2023-03-29T10:57:31.872088+0000","last_unstale":"2023-03-29T10:57:32.336574+0000","last_undegraded":"2023-03-29T10:57:32.336574+0000","last_fullsized":"2023-03-29T10:57:32.336574+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:28:41.336878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852880+0000","last_change":"2023-03-29T10:57:22.318583+0000","last_active":"2023-03-29T10:57:31.852880+0000","last_peered":"2023-03-29T10:57:31.852880+0000","last_clean":"2023-03-29T10:57:31.852880+0000","last_became_active":"2023-03-29T10:57:22.318223+0000","last_became_peered":"2023-03-29T10:57:22.318223+0000","last_unstale":"2023-03-29T10:57:31.852880+0000","last_undegraded":"2023-03-29T10:57:31.852880+0000","last_fullsized":"2023-03-29T10:57:31.852880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:31:03.349241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.864963+0000","last_change":"2023-03-29T10:57:27.369314+0000","last_active":"2023-03-29T10:57:31.864963+0000","last_peered":"2023-03-29T10:57:31.864963+0000","last_clean":"2023-03-29T10:57:31.864963+0000","last_became_active":"2023-03-29T10:57:27.368989+0000","last_became_peered":"2023-03-29T10:57:27.368989+0000","last_unstale":"2023-03-29T10:57:31.864963+0000","last_undegraded":"2023-03-29T10:57:31.864963+0000","last_fullsized":"2023-03-29T10:57:31.864963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:35:24.603180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851286+0000","last_change":"2023-03-29T10:57:27.860136+0000","last_active":"2023-03-29T10:57:31.851286+0000","last_peered":"2023-03-29T10:57:31.851286+0000","last_clean":"2023-03-29T10:57:31.851286+0000","last_became_active":"2023-03-29T10:57:27.859888+0000","last_became_peered":"2023-03-29T10:57:27.859888+0000","last_unstale":"2023-03-29T10:57:31.851286+0000","last_undegraded":"2023-03-29T10:57:31.851286+0000","last_fullsized":"2023-03-29T10:57:31.851286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:23:24.586546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.336910+0000","last_change":"2023-03-29T10:57:31.877812+0000","last_active":"2023-03-29T10:57:32.336910+0000","last_peered":"2023-03-29T10:57:32.336910+0000","last_clean":"2023-03-29T10:57:32.336910+0000","last_became_active":"2023-03-29T10:57:31.877432+0000","last_became_peered":"2023-03-29T10:57:31.877432+0000","last_unstale":"2023-03-29T10:57:32.336910+0000","last_undegraded":"2023-03-29T10:57:32.336910+0000","last_fullsized":"2023-03-29T10:57:32.336910+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:19:12.936732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.863218+0000","last_change":"2023-03-29T10:57:22.320330+0000","last_active":"2023-03-29T10:57:31.863218+0000","last_peered":"2023-03-29T10:57:31.863218+0000","last_clean":"2023-03-29T10:57:31.863218+0000","last_became_active":"2023-03-29T10:57:22.320011+0000","last_became_peered":"2023-03-29T10:57:22.320011+0000","last_unstale":"2023-03-29T10:57:31.863218+0000","last_undegraded":"2023-03-29T10:57:31.863218+0000","last_fullsized":"2023-03-29T10:57:31.863218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:47:01.099069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853293+0000","last_change":"2023-03-29T10:57:27.361265+0000","last_active":"2023-03-29T10:57:31.853293+0000","last_peered":"2023-03-29T10:57:31.853293+0000","last_clean":"2023-03-29T10:57:31.853293+0000","last_became_active":"2023-03-29T10:57:27.360017+0000","last_became_peered":"2023-03-29T10:57:27.360017+0000","last_unstale":"2023-03-29T10:57:31.853293+0000","last_undegraded":"2023-03-29T10:57:31.853293+0000","last_fullsized":"2023-03-29T10:57:31.853293+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:12:14.095287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852801+0000","last_change":"2023-03-29T10:57:27.846016+0000","last_active":"2023-03-29T10:57:31.852801+0000","last_peered":"2023-03-29T10:57:31.852801+0000","last_clean":"2023-03-29T10:57:31.852801+0000","last_became_active":"2023-03-29T10:57:27.845677+0000","last_became_peered":"2023-03-29T10:57:27.845677+0000","last_unstale":"2023-03-29T10:57:31.852801+0000","last_undegraded":"2023-03-29T10:57:31.852801+0000","last_fullsized":"2023-03-29T10:57:31.852801+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T10:58:34.425561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605463+0000","last_change":"2023-03-29T10:57:31.863426+0000","last_active":"2023-03-29T10:57:32.605463+0000","last_peered":"2023-03-29T10:57:32.605463+0000","last_clean":"2023-03-29T10:57:32.605463+0000","last_became_active":"2023-03-29T10:57:31.862992+0000","last_became_peered":"2023-03-29T10:57:31.862992+0000","last_unstale":"2023-03-29T10:57:32.605463+0000","last_undegraded":"2023-03-29T10:57:32.605463+0000","last_fullsized":"2023-03-29T10:57:32.605463+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:24:11.757160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.845511+0000","last_change":"2023-03-29T10:57:22.467148+0000","last_active":"2023-03-29T10:57:31.845511+0000","last_peered":"2023-03-29T10:57:31.845511+0000","last_clean":"2023-03-29T10:57:31.845511+0000","last_became_active":"2023-03-29T10:57:22.466822+0000","last_became_peered":"2023-03-29T10:57:22.466822+0000","last_unstale":"2023-03-29T10:57:31.845511+0000","last_undegraded":"2023-03-29T10:57:31.845511+0000","last_fullsized":"2023-03-29T10:57:31.845511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:53:23.007205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851898+0000","last_change":"2023-03-29T10:57:27.364233+0000","last_active":"2023-03-29T10:57:31.851898+0000","last_peered":"2023-03-29T10:57:31.851898+0000","last_clean":"2023-03-29T10:57:31.851898+0000","last_became_active":"2023-03-29T10:57:27.363992+0000","last_became_peered":"2023-03-29T10:57:27.363992+0000","last_unstale":"2023-03-29T10:57:31.851898+0000","last_undegraded":"2023-03-29T10:57:31.851898+0000","last_fullsized":"2023-03-29T10:57:31.851898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:57:18.786945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851648+0000","last_change":"2023-03-29T10:57:27.859088+0000","last_active":"2023-03-29T10:57:31.851648+0000","last_peered":"2023-03-29T10:57:31.851648+0000","last_clean":"2023-03-29T10:57:31.851648+0000","last_became_active":"2023-03-29T10:57:27.858803+0000","last_became_peered":"2023-03-29T10:57:27.858803+0000","last_unstale":"2023-03-29T10:57:31.851648+0000","last_undegraded":"2023-03-29T10:57:31.851648+0000","last_fullsized":"2023-03-29T10:57:31.851648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:44:42.208513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.341079+0000","last_change":"2023-03-29T10:57:31.868378+0000","last_active":"2023-03-29T10:57:32.341079+0000","last_peered":"2023-03-29T10:57:32.341079+0000","last_clean":"2023-03-29T10:57:32.341079+0000","last_became_active":"2023-03-29T10:57:31.867988+0000","last_became_peered":"2023-03-29T10:57:31.867988+0000","last_unstale":"2023-03-29T10:57:32.341079+0000","last_undegraded":"2023-03-29T10:57:32.341079+0000","last_fullsized":"2023-03-29T10:57:32.341079+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:18:14.433000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861576+0000","last_change":"2023-03-29T10:57:22.321523+0000","last_active":"2023-03-29T10:57:31.861576+0000","last_peered":"2023-03-29T10:57:31.861576+0000","last_clean":"2023-03-29T10:57:31.861576+0000","last_became_active":"2023-03-29T10:57:22.321183+0000","last_became_peered":"2023-03-29T10:57:22.321183+0000","last_unstale":"2023-03-29T10:57:31.861576+0000","last_undegraded":"2023-03-29T10:57:31.861576+0000","last_fullsized":"2023-03-29T10:57:31.861576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:35:11.664657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846653+0000","last_change":"2023-03-29T10:57:18.176665+0000","last_active":"2023-03-29T10:57:31.846653+0000","last_peered":"2023-03-29T10:57:31.846653+0000","last_clean":"2023-03-29T10:57:31.846653+0000","last_became_active":"2023-03-29T10:57:18.175739+0000","last_became_peered":"2023-03-29T10:57:18.175739+0000","last_unstale":"2023-03-29T10:57:31.846653+0000","last_undegraded":"2023-03-29T10:57:31.846653+0000","last_fullsized":"2023-03-29T10:57:31.846653+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:16.275989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:16.275989+0000","last_clean_scrub_stamp":"2023-03-29T10:57:16.275989+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:06:32.872344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851023+0000","last_change":"2023-03-29T10:57:27.365796+0000","last_active":"2023-03-29T10:57:31.851023+0000","last_peered":"2023-03-29T10:57:31.851023+0000","last_clean":"2023-03-29T10:57:31.851023+0000","last_became_active":"2023-03-29T10:57:27.365435+0000","last_became_peered":"2023-03-29T10:57:27.365435+0000","last_unstale":"2023-03-29T10:57:31.851023+0000","last_undegraded":"2023-03-29T10:57:31.851023+0000","last_fullsized":"2023-03-29T10:57:31.851023+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:12:14.095287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.855069+0000","last_change":"2023-03-29T10:57:27.866559+0000","last_active":"2023-03-29T10:57:31.855069+0000","last_peered":"2023-03-29T10:57:31.855069+0000","last_clean":"2023-03-29T10:57:31.855069+0000","last_became_active":"2023-03-29T10:57:27.866250+0000","last_became_peered":"2023-03-29T10:57:27.866250+0000","last_unstale":"2023-03-29T10:57:31.855069+0000","last_undegraded":"2023-03-29T10:57:31.855069+0000","last_fullsized":"2023-03-29T10:57:31.855069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:53:00.445130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606048+0000","last_change":"2023-03-29T10:57:31.881080+0000","last_active":"2023-03-29T10:57:32.606048+0000","last_peered":"2023-03-29T10:57:32.606048+0000","last_clean":"2023-03-29T10:57:32.606048+0000","last_became_active":"2023-03-29T10:57:31.880427+0000","last_became_peered":"2023-03-29T10:57:31.880427+0000","last_unstale":"2023-03-29T10:57:32.606048+0000","last_undegraded":"2023-03-29T10:57:32.606048+0000","last_fullsized":"2023-03-29T10:57:32.606048+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:36:05.766940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847505+0000","last_change":"2023-03-29T10:57:22.322607+0000","last_active":"2023-03-29T10:57:31.847505+0000","last_peered":"2023-03-29T10:57:31.847505+0000","last_clean":"2023-03-29T10:57:31.847505+0000","last_became_active":"2023-03-29T10:57:22.321613+0000","last_became_peered":"2023-03-29T10:57:22.321613+0000","last_unstale":"2023-03-29T10:57:31.847505+0000","last_undegraded":"2023-03-29T10:57:31.847505+0000","last_fullsized":"2023-03-29T10:57:31.847505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:35:11.664657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851455+0000","last_change":"2023-03-29T10:57:27.366362+0000","last_active":"2023-03-29T10:57:31.851455+0000","last_peered":"2023-03-29T10:57:31.851455+0000","last_clean":"2023-03-29T10:57:31.851455+0000","last_became_active":"2023-03-29T10:57:27.366048+0000","last_became_peered":"2023-03-29T10:57:27.366048+0000","last_unstale":"2023-03-29T10:57:31.851455+0000","last_undegraded":"2023-03-29T10:57:31.851455+0000","last_fullsized":"2023-03-29T10:57:31.851455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:01:05.024349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851632+0000","last_change":"2023-03-29T10:57:27.847948+0000","last_active":"2023-03-29T10:57:31.851632+0000","last_peered":"2023-03-29T10:57:31.851632+0000","last_clean":"2023-03-29T10:57:31.851632+0000","last_became_active":"2023-03-29T10:57:27.847691+0000","last_became_peered":"2023-03-29T10:57:27.847691+0000","last_unstale":"2023-03-29T10:57:31.851632+0000","last_undegraded":"2023-03-29T10:57:31.851632+0000","last_fullsized":"2023-03-29T10:57:31.851632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:01:58.615750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605974+0000","last_change":"2023-03-29T10:57:31.873406+0000","last_active":"2023-03-29T10:57:32.605974+0000","last_peered":"2023-03-29T10:57:32.605974+0000","last_clean":"2023-03-29T10:57:32.605974+0000","last_became_active":"2023-03-29T10:57:31.873032+0000","last_became_peered":"2023-03-29T10:57:31.873032+0000","last_unstale":"2023-03-29T10:57:32.605974+0000","last_undegraded":"2023-03-29T10:57:32.605974+0000","last_fullsized":"2023-03-29T10:57:32.605974+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:53:41.690520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846972+0000","last_change":"2023-03-29T10:57:22.315905+0000","last_active":"2023-03-29T10:57:31.846972+0000","last_peered":"2023-03-29T10:57:31.846972+0000","last_clean":"2023-03-29T10:57:31.846972+0000","last_became_active":"2023-03-29T10:57:22.315603+0000","last_became_peered":"2023-03-29T10:57:22.315603+0000","last_unstale":"2023-03-29T10:57:31.846972+0000","last_undegraded":"2023-03-29T10:57:31.846972+0000","last_fullsized":"2023-03-29T10:57:31.846972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T10:59:44.030013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853041+0000","last_change":"2023-03-29T10:57:27.363544+0000","last_active":"2023-03-29T10:57:31.853041+0000","last_peered":"2023-03-29T10:57:31.853041+0000","last_clean":"2023-03-29T10:57:31.853041+0000","last_became_active":"2023-03-29T10:57:27.363284+0000","last_became_peered":"2023-03-29T10:57:27.363284+0000","last_unstale":"2023-03-29T10:57:31.853041+0000","last_undegraded":"2023-03-29T10:57:31.853041+0000","last_fullsized":"2023-03-29T10:57:31.853041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:16.312585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853937+0000","last_change":"2023-03-29T10:57:27.844093+0000","last_active":"2023-03-29T10:57:31.853937+0000","last_peered":"2023-03-29T10:57:31.853937+0000","last_clean":"2023-03-29T10:57:31.853937+0000","last_became_active":"2023-03-29T10:57:27.843647+0000","last_became_peered":"2023-03-29T10:57:27.843647+0000","last_unstale":"2023-03-29T10:57:31.853937+0000","last_undegraded":"2023-03-29T10:57:31.853937+0000","last_fullsized":"2023-03-29T10:57:31.853937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:46:29.403965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340198+0000","last_change":"2023-03-29T10:57:31.878229+0000","last_active":"2023-03-29T10:57:32.340198+0000","last_peered":"2023-03-29T10:57:32.340198+0000","last_clean":"2023-03-29T10:57:32.340198+0000","last_became_active":"2023-03-29T10:57:31.877846+0000","last_became_peered":"2023-03-29T10:57:31.877846+0000","last_unstale":"2023-03-29T10:57:32.340198+0000","last_undegraded":"2023-03-29T10:57:32.340198+0000","last_fullsized":"2023-03-29T10:57:32.340198+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:32:15.233141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852732+0000","last_change":"2023-03-29T10:57:27.366984+0000","last_active":"2023-03-29T10:57:31.852732+0000","last_peered":"2023-03-29T10:57:31.852732+0000","last_clean":"2023-03-29T10:57:31.852732+0000","last_became_active":"2023-03-29T10:57:27.366716+0000","last_became_peered":"2023-03-29T10:57:27.366716+0000","last_unstale":"2023-03-29T10:57:31.852732+0000","last_undegraded":"2023-03-29T10:57:31.852732+0000","last_fullsized":"2023-03-29T10:57:31.852732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:00:38.166723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852301+0000","last_change":"2023-03-29T10:57:22.319554+0000","last_active":"2023-03-29T10:57:31.852301+0000","last_peered":"2023-03-29T10:57:31.852301+0000","last_clean":"2023-03-29T10:57:31.852301+0000","last_became_active":"2023-03-29T10:57:22.319198+0000","last_became_peered":"2023-03-29T10:57:22.319198+0000","last_unstale":"2023-03-29T10:57:31.852301+0000","last_undegraded":"2023-03-29T10:57:31.852301+0000","last_fullsized":"2023-03-29T10:57:31.852301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:47:01.099069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849473+0000","last_change":"2023-03-29T10:57:27.839233+0000","last_active":"2023-03-29T10:57:31.849473+0000","last_peered":"2023-03-29T10:57:31.849473+0000","last_clean":"2023-03-29T10:57:31.849473+0000","last_became_active":"2023-03-29T10:57:27.838936+0000","last_became_peered":"2023-03-29T10:57:27.838936+0000","last_unstale":"2023-03-29T10:57:31.849473+0000","last_undegraded":"2023-03-29T10:57:31.849473+0000","last_fullsized":"2023-03-29T10:57:31.849473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:10:11.279944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.607718+0000","last_change":"2023-03-29T10:57:31.876988+0000","last_active":"2023-03-29T10:57:32.607718+0000","last_peered":"2023-03-29T10:57:32.607718+0000","last_clean":"2023-03-29T10:57:32.607718+0000","last_became_active":"2023-03-29T10:57:31.876521+0000","last_became_peered":"2023-03-29T10:57:31.876521+0000","last_unstale":"2023-03-29T10:57:32.607718+0000","last_undegraded":"2023-03-29T10:57:32.607718+0000","last_fullsized":"2023-03-29T10:57:32.607718+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:40:04.971172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.858506+0000","last_change":"2023-03-29T10:57:22.322319+0000","last_active":"2023-03-29T10:57:31.858506+0000","last_peered":"2023-03-29T10:57:31.858506+0000","last_clean":"2023-03-29T10:57:31.858506+0000","last_became_active":"2023-03-29T10:57:22.321943+0000","last_became_peered":"2023-03-29T10:57:22.321943+0000","last_unstale":"2023-03-29T10:57:31.858506+0000","last_undegraded":"2023-03-29T10:57:31.858506+0000","last_fullsized":"2023-03-29T10:57:31.858506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:58:04.969453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"28'1","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.336470+0000","last_change":"2023-03-29T10:57:27.364507+0000","last_active":"2023-03-29T10:57:32.336470+0000","last_peered":"2023-03-29T10:57:32.336470+0000","last_clean":"2023-03-29T10:57:32.336470+0000","last_became_active":"2023-03-29T10:57:27.363959+0000","last_became_peered":"2023-03-29T10:57:27.363959+0000","last_unstale":"2023-03-29T10:57:32.336470+0000","last_undegraded":"2023-03-29T10:57:32.336470+0000","last_fullsized":"2023-03-29T10:57:32.336470+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:16.312585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860854+0000","last_change":"2023-03-29T10:57:27.861040+0000","last_active":"2023-03-29T10:57:31.860854+0000","last_peered":"2023-03-29T10:57:31.860854+0000","last_clean":"2023-03-29T10:57:31.860854+0000","last_became_active":"2023-03-29T10:57:27.860750+0000","last_became_peered":"2023-03-29T10:57:27.860750+0000","last_unstale":"2023-03-29T10:57:31.860854+0000","last_undegraded":"2023-03-29T10:57:31.860854+0000","last_fullsized":"2023-03-29T10:57:31.860854+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:40:51.829430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.543747+0000","last_change":"2023-03-29T10:57:31.882689+0000","last_active":"2023-03-29T10:57:32.543747+0000","last_peered":"2023-03-29T10:57:32.543747+0000","last_clean":"2023-03-29T10:57:32.543747+0000","last_became_active":"2023-03-29T10:57:31.882298+0000","last_became_peered":"2023-03-29T10:57:31.882298+0000","last_unstale":"2023-03-29T10:57:32.543747+0000","last_undegraded":"2023-03-29T10:57:32.543747+0000","last_fullsized":"2023-03-29T10:57:32.543747+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:18:14.433000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853851+0000","last_change":"2023-03-29T10:57:27.363293+0000","last_active":"2023-03-29T10:57:31.853851+0000","last_peered":"2023-03-29T10:57:31.853851+0000","last_clean":"2023-03-29T10:57:31.853851+0000","last_became_active":"2023-03-29T10:57:27.362993+0000","last_became_peered":"2023-03-29T10:57:27.362993+0000","last_unstale":"2023-03-29T10:57:31.853851+0000","last_undegraded":"2023-03-29T10:57:31.853851+0000","last_fullsized":"2023-03-29T10:57:31.853851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:57:18.786945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.845743+0000","last_change":"2023-03-29T10:57:27.849850+0000","last_active":"2023-03-29T10:57:31.845743+0000","last_peered":"2023-03-29T10:57:31.845743+0000","last_clean":"2023-03-29T10:57:31.845743+0000","last_became_active":"2023-03-29T10:57:27.849429+0000","last_became_peered":"2023-03-29T10:57:27.849429+0000","last_unstale":"2023-03-29T10:57:31.845743+0000","last_undegraded":"2023-03-29T10:57:31.845743+0000","last_fullsized":"2023-03-29T10:57:31.845743+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:00:39.175701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.341198+0000","last_change":"2023-03-29T10:57:31.868416+0000","last_active":"2023-03-29T10:57:32.341198+0000","last_peered":"2023-03-29T10:57:32.341198+0000","last_clean":"2023-03-29T10:57:32.341198+0000","last_became_active":"2023-03-29T10:57:31.868042+0000","last_became_peered":"2023-03-29T10:57:31.868042+0000","last_unstale":"2023-03-29T10:57:32.341198+0000","last_undegraded":"2023-03-29T10:57:32.341198+0000","last_fullsized":"2023-03-29T10:57:32.341198+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:28:41.336878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861823+0000","last_change":"2023-03-29T10:57:27.373078+0000","last_active":"2023-03-29T10:57:31.861823+0000","last_peered":"2023-03-29T10:57:31.861823+0000","last_clean":"2023-03-29T10:57:31.861823+0000","last_became_active":"2023-03-29T10:57:27.372747+0000","last_became_peered":"2023-03-29T10:57:27.372747+0000","last_unstale":"2023-03-29T10:57:31.861823+0000","last_undegraded":"2023-03-29T10:57:31.861823+0000","last_fullsized":"2023-03-29T10:57:31.861823+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:23:59.025438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853187+0000","last_change":"2023-03-29T10:57:27.861043+0000","last_active":"2023-03-29T10:57:31.853187+0000","last_peered":"2023-03-29T10:57:31.853187+0000","last_clean":"2023-03-29T10:57:31.853187+0000","last_became_active":"2023-03-29T10:57:27.860522+0000","last_became_peered":"2023-03-29T10:57:27.860522+0000","last_unstale":"2023-03-29T10:57:31.853187+0000","last_undegraded":"2023-03-29T10:57:31.853187+0000","last_fullsized":"2023-03-29T10:57:31.853187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:49:24.886407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606089+0000","last_change":"2023-03-29T10:57:31.879767+0000","last_active":"2023-03-29T10:57:32.606089+0000","last_peered":"2023-03-29T10:57:32.606089+0000","last_clean":"2023-03-29T10:57:32.606089+0000","last_became_active":"2023-03-29T10:57:31.879330+0000","last_became_peered":"2023-03-29T10:57:31.879330+0000","last_unstale":"2023-03-29T10:57:32.606089+0000","last_undegraded":"2023-03-29T10:57:32.606089+0000","last_fullsized":"2023-03-29T10:57:32.606089+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:10:08.859801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.864594+0000","last_change":"2023-03-29T10:57:27.370488+0000","last_active":"2023-03-29T10:57:31.864594+0000","last_peered":"2023-03-29T10:57:31.864594+0000","last_clean":"2023-03-29T10:57:31.864594+0000","last_became_active":"2023-03-29T10:57:27.370181+0000","last_became_peered":"2023-03-29T10:57:27.370181+0000","last_unstale":"2023-03-29T10:57:31.864594+0000","last_undegraded":"2023-03-29T10:57:31.864594+0000","last_fullsized":"2023-03-29T10:57:31.864594+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:10:38.094232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852668+0000","last_change":"2023-03-29T10:57:27.860955+0000","last_active":"2023-03-29T10:57:31.852668+0000","last_peered":"2023-03-29T10:57:31.852668+0000","last_clean":"2023-03-29T10:57:31.852668+0000","last_became_active":"2023-03-29T10:57:27.860396+0000","last_became_peered":"2023-03-29T10:57:27.860396+0000","last_unstale":"2023-03-29T10:57:31.852668+0000","last_undegraded":"2023-03-29T10:57:31.852668+0000","last_fullsized":"2023-03-29T10:57:31.852668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:44:40.017537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.339309+0000","last_change":"2023-03-29T10:57:31.874679+0000","last_active":"2023-03-29T10:57:32.339309+0000","last_peered":"2023-03-29T10:57:32.339309+0000","last_clean":"2023-03-29T10:57:32.339309+0000","last_became_active":"2023-03-29T10:57:31.874124+0000","last_became_peered":"2023-03-29T10:57:31.874124+0000","last_unstale":"2023-03-29T10:57:32.339309+0000","last_undegraded":"2023-03-29T10:57:32.339309+0000","last_fullsized":"2023-03-29T10:57:32.339309+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:36:05.766940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853964+0000","last_change":"2023-03-29T10:57:27.366500+0000","last_active":"2023-03-29T10:57:31.853964+0000","last_peered":"2023-03-29T10:57:31.853964+0000","last_clean":"2023-03-29T10:57:31.853964+0000","last_became_active":"2023-03-29T10:57:27.366215+0000","last_became_peered":"2023-03-29T10:57:27.366215+0000","last_unstale":"2023-03-29T10:57:31.853964+0000","last_undegraded":"2023-03-29T10:57:31.853964+0000","last_fullsized":"2023-03-29T10:57:31.853964+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:45:12.326563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852786+0000","last_change":"2023-03-29T10:57:27.838577+0000","last_active":"2023-03-29T10:57:31.852786+0000","last_peered":"2023-03-29T10:57:31.852786+0000","last_clean":"2023-03-29T10:57:31.852786+0000","last_became_active":"2023-03-29T10:57:27.838301+0000","last_became_peered":"2023-03-29T10:57:27.838301+0000","last_unstale":"2023-03-29T10:57:31.852786+0000","last_undegraded":"2023-03-29T10:57:31.852786+0000","last_fullsized":"2023-03-29T10:57:31.852786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:36:09.200571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.543500+0000","last_change":"2023-03-29T10:57:31.874474+0000","last_active":"2023-03-29T10:57:32.543500+0000","last_peered":"2023-03-29T10:57:32.543500+0000","last_clean":"2023-03-29T10:57:32.543500+0000","last_became_active":"2023-03-29T10:57:31.874137+0000","last_became_peered":"2023-03-29T10:57:31.874137+0000","last_unstale":"2023-03-29T10:57:32.543500+0000","last_undegraded":"2023-03-29T10:57:32.543500+0000","last_fullsized":"2023-03-29T10:57:32.543500+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:10.382014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.857710+0000","last_change":"2023-03-29T10:57:27.365343+0000","last_active":"2023-03-29T10:57:31.857710+0000","last_peered":"2023-03-29T10:57:31.857710+0000","last_clean":"2023-03-29T10:57:31.857710+0000","last_became_active":"2023-03-29T10:57:27.365066+0000","last_became_peered":"2023-03-29T10:57:27.365066+0000","last_unstale":"2023-03-29T10:57:31.857710+0000","last_undegraded":"2023-03-29T10:57:31.857710+0000","last_fullsized":"2023-03-29T10:57:31.857710+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:40:18.602235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860006+0000","last_change":"2023-03-29T10:57:27.860635+0000","last_active":"2023-03-29T10:57:31.860006+0000","last_peered":"2023-03-29T10:57:31.860006+0000","last_clean":"2023-03-29T10:57:31.860006+0000","last_became_active":"2023-03-29T10:57:27.860287+0000","last_became_peered":"2023-03-29T10:57:27.860287+0000","last_unstale":"2023-03-29T10:57:31.860006+0000","last_undegraded":"2023-03-29T10:57:31.860006+0000","last_fullsized":"2023-03-29T10:57:31.860006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:57:16.888544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852270+0000","last_change":"2023-03-29T10:57:27.858882+0000","last_active":"2023-03-29T10:57:31.852270+0000","last_peered":"2023-03-29T10:57:31.852270+0000","last_clean":"2023-03-29T10:57:31.852270+0000","last_became_active":"2023-03-29T10:57:27.858471+0000","last_became_peered":"2023-03-29T10:57:27.858471+0000","last_unstale":"2023-03-29T10:57:31.852270+0000","last_undegraded":"2023-03-29T10:57:31.852270+0000","last_fullsized":"2023-03-29T10:57:31.852270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:10:39.103210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852924+0000","last_change":"2023-03-29T10:57:27.848104+0000","last_active":"2023-03-29T10:57:31.852924+0000","last_peered":"2023-03-29T10:57:31.852924+0000","last_clean":"2023-03-29T10:57:31.852924+0000","last_became_active":"2023-03-29T10:57:27.847812+0000","last_became_peered":"2023-03-29T10:57:27.847812+0000","last_unstale":"2023-03-29T10:57:31.852924+0000","last_undegraded":"2023-03-29T10:57:31.852924+0000","last_fullsized":"2023-03-29T10:57:31.852924+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:50:00.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852955+0000","last_change":"2023-03-29T10:57:27.868878+0000","last_active":"2023-03-29T10:57:31.852955+0000","last_peered":"2023-03-29T10:57:31.852955+0000","last_clean":"2023-03-29T10:57:31.852955+0000","last_became_active":"2023-03-29T10:57:27.868603+0000","last_became_peered":"2023-03-29T10:57:27.868603+0000","last_unstale":"2023-03-29T10:57:31.852955+0000","last_undegraded":"2023-03-29T10:57:31.852955+0000","last_fullsized":"2023-03-29T10:57:31.852955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:53:38.186916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851718+0000","last_change":"2023-03-29T10:57:27.853683+0000","last_active":"2023-03-29T10:57:31.851718+0000","last_peered":"2023-03-29T10:57:31.851718+0000","last_clean":"2023-03-29T10:57:31.851718+0000","last_became_active":"2023-03-29T10:57:27.853396+0000","last_became_peered":"2023-03-29T10:57:27.853396+0000","last_unstale":"2023-03-29T10:57:31.851718+0000","last_undegraded":"2023-03-29T10:57:31.851718+0000","last_fullsized":"2023-03-29T10:57:31.851718+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:57:16.888544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852962+0000","last_change":"2023-03-29T10:57:27.856369+0000","last_active":"2023-03-29T10:57:31.852962+0000","last_peered":"2023-03-29T10:57:31.852962+0000","last_clean":"2023-03-29T10:57:31.852962+0000","last_became_active":"2023-03-29T10:57:27.856118+0000","last_became_peered":"2023-03-29T10:57:27.856118+0000","last_unstale":"2023-03-29T10:57:31.852962+0000","last_undegraded":"2023-03-29T10:57:31.852962+0000","last_fullsized":"2023-03-29T10:57:31.852962+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:33:37.107795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4598,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9296,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362544,"statfs":{"total":96636764160,"available":96627245056,"internally_reserved":0,"allocated":925696,"data_stored":718956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8112,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363728,"statfs":{"total":96636764160,"available":96628457472,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]}]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8176,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363664,"statfs":{"total":96636764160,"available":96628391936,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9268,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362572,"statfs":{"total":96636764160,"available":96627273728,"internally_reserved":0,"allocated":897024,"data_stored":714903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8108,"kb_used_data":292,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363732,"statfs":{"total":96636764160,"available":96628461568,"internally_reserved":0,"allocated":299008,"data_stored":123953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8180,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363660,"statfs":{"total":96636764160,"available":96628387840,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8260,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363580,"statfs":{"total":96636764160,"available":96628305920,"internally_reserved":0,"allocated":323584,"data_stored":128428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8184,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363656,"statfs":{"total":96636764160,"available":96628383744,"internally_reserved":0,"allocated":311296,"data_stored":125224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-03-29T10:58:00.167 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T10:58:00.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.372+0000 7fe4c83ff700 1 Processor -- start 2023-03-29T10:58:00.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.372+0000 7fe4c83ff700 1 -- start start 2023-03-29T10:58:00.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.373+0000 7fe4c83ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe4c015dce0 0x7fe4c00cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:00.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.373+0000 7fe4c83ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c00cd9f0 0x7fe4c00cde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:00.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.373+0000 7fe4c83ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c00ce350 con 0x7fe4c01594c0 2023-03-29T10:58:00.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.373+0000 7fe4c83ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c00b0dd0 con 0x7fe4c015dce0 2023-03-29T10:58:00.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.373+0000 7fe4c83ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c00b0fc0 con 0x7fe4c00cd9f0 2023-03-29T10:58:00.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.373+0000 7fe4c619b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe4c01594c0 0x7fe4c00b06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36952/0 (socket says 172.21.15.111:36952) 2023-03-29T10:58:00.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.373+0000 7fe4c619b700 1 -- 172.21.15.111:0/3607028429 learned_addr learned my addr 172.21.15.111:0/3607028429 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:00.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.374+0000 7fe4c699c700 1 --2- 172.21.15.111:0/3607028429 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c00cd9f0 0x7fe4c00cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:00.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.374+0000 7fe4c599a700 1 --2- 172.21.15.111:0/3607028429 >> v2:172.21.15.119:3300/0 conn(0x7fe4c015dce0 0x7fe4c00cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:00.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.374+0000 7fe4c5199700 1 -- 172.21.15.111:0/3607028429 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635723348 0 0) 0x7fe4c00ce350 con 0x7fe4c01594c0 2023-03-29T10:58:00.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.374+0000 7fe4c5199700 1 -- 172.21.15.111:0/3607028429 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4ac0057c0 con 0x7fe4c01594c0 2023-03-29T10:58:00.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.374+0000 7fe4c599a700 1 -- 172.21.15.111:0/3607028429 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c00cd9f0 msgr2=0x7fe4c00cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:00.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.374+0000 7fe4c599a700 1 --2- 172.21.15.111:0/3607028429 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c00cd9f0 0x7fe4c00cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:00.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.374+0000 7fe4c599a700 1 -- 172.21.15.111:0/3607028429 >> v1:172.21.15.111:6789/0 conn(0x7fe4c01594c0 legacy=0x7fe4c00b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:00.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.374+0000 7fe4c5199700 1 -- 172.21.15.111:0/3607028429 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1341876065 0 0) 0x7fe4ac0057c0 con 0x7fe4c01594c0 2023-03-29T10:58:00.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.374+0000 7fe4c599a700 1 -- 172.21.15.111:0/3607028429 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c00cc9b0 con 0x7fe4c015dce0 2023-03-29T10:58:00.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.374+0000 7fe4c699c700 1 --2- 172.21.15.111:0/3607028429 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c00cd9f0 0x7fe4c00cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:58:00.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.374+0000 7fe4c599a700 1 --2- 172.21.15.111:0/3607028429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c015dce0 0x7fe4c00cd470 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009840 tx=0x7fe4b0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ad4624b2fb1cbf8 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:00.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.375+0000 7fe4c5199700 1 -- 172.21.15.111:0/3607028429 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc001670 con 0x7fe4c015dce0 2023-03-29T10:58:00.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.375+0000 7fe4c5199700 1 -- 172.21.15.111:0/3607028429 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b0004030 con 0x7fe4c015dce0 2023-03-29T10:58:00.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.375+0000 7fe4c5199700 1 -- 172.21.15.111:0/3607028429 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b00043c0 con 0x7fe4c015dce0 2023-03-29T10:58:00.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.375+0000 7fe4c83ff700 1 -- 172.21.15.111:0/3607028429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c015dce0 msgr2=0x7fe4c00cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:00.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.375+0000 7fe4c83ff700 1 --2- 172.21.15.111:0/3607028429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c015dce0 0x7fe4c00cd470 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009840 tx=0x7fe4b0009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:00.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.375+0000 7fe4c83ff700 1 -- 172.21.15.111:0/3607028429 shutdown_connections 2023-03-29T10:58:00.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.375+0000 7fe4c83ff700 1 --2- 172.21.15.111:0/3607028429 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c00cd9f0 0x7fe4c00cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:00.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.375+0000 7fe4c83ff700 1 --2- 172.21.15.111:0/3607028429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c015dce0 0x7fe4c00cd470 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:00.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.375+0000 7fe4c83ff700 1 -- 172.21.15.111:0/3607028429 >> 172.21.15.111:0/3607028429 conn(0x7fe4c00cb6c0 msgr2=0x7fe4c00cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:00.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.375+0000 7fe4c83ff700 1 -- 172.21.15.111:0/3607028429 shutdown_connections 2023-03-29T10:58:00.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.375+0000 7fe4c83ff700 1 -- 172.21.15.111:0/3607028429 wait complete. 2023-03-29T10:58:00.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.377+0000 7fe4c83ff700 1 Processor -- start 2023-03-29T10:58:00.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.377+0000 7fe4c83ff700 1 -- start start 2023-03-29T10:58:00.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.378+0000 7fe4c83ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c00cd9f0 0x7fe4c0145c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:00.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.378+0000 7fe4c83ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c01594c0 0x7fe4c00c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:00.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.378+0000 7fe4c83ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c015dce0 0x7fe4c01f1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:00.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.378+0000 7fe4c83ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c01ed930 con 0x7fe4c00cd9f0 2023-03-29T10:58:00.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.378+0000 7fe4c83ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c00c6250 con 0x7fe4c01594c0 2023-03-29T10:58:00.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.378+0000 7fe4c83ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c01f1b90 con 0x7fe4c015dce0 2023-03-29T10:58:00.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.378+0000 7fe4c619b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c00cd9f0 0x7fe4c0145c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:00.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.378+0000 7fe4c619b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c00cd9f0 0x7fe4c0145c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50374/0 (socket says 172.21.15.111:50374) 2023-03-29T10:58:00.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.378+0000 7fe4c619b700 1 -- 172.21.15.111:0/2039102274 learned_addr learned my addr 172.21.15.111:0/2039102274 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:00.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.378+0000 7fe4c599a700 1 --2- 172.21.15.111:0/2039102274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c01594c0 0x7fe4c00c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:00.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.378+0000 7fe4c699c700 1 --2- 172.21.15.111:0/2039102274 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c015dce0 0x7fe4c01f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:00.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.379+0000 7fe4c619b700 1 -- 172.21.15.111:0/2039102274 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c015dce0 msgr2=0x7fe4c01f1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:00.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.379+0000 7fe4c619b700 1 --2- 172.21.15.111:0/2039102274 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c015dce0 0x7fe4c01f1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:00.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.379+0000 7fe4c619b700 1 -- 172.21.15.111:0/2039102274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c01594c0 msgr2=0x7fe4c00c5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:00.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.379+0000 7fe4c619b700 1 --2- 172.21.15.111:0/2039102274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c01594c0 0x7fe4c00c5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:00.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.379+0000 7fe4c619b700 1 -- 172.21.15.111:0/2039102274 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b8009a70 con 0x7fe4c00cd9f0 2023-03-29T10:58:00.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.379+0000 7fe4c619b700 1 --2- 172.21.15.111:0/2039102274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c00cd9f0 0x7fe4c0145c00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00c780 tx=0x7fe4bc00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:00.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.379+0000 7fe4b6ffd700 1 -- 172.21.15.111:0/2039102274 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc004c80 con 0x7fe4c00cd9f0 2023-03-29T10:58:00.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.379+0000 7fe4b6ffd700 1 -- 172.21.15.111:0/2039102274 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4bc004de0 con 0x7fe4c00cd9f0 2023-03-29T10:58:00.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.379+0000 7fe4b6ffd700 1 -- 172.21.15.111:0/2039102274 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4bc0072f0 con 0x7fe4c00cd9f0 2023-03-29T10:58:00.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.379+0000 7fe4c83ff700 1 -- 172.21.15.111:0/2039102274 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b00094a0 con 0x7fe4c00cd9f0 2023-03-29T10:58:00.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.379+0000 7fe4c83ff700 1 -- 172.21.15.111:0/2039102274 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c00ccf70 con 0x7fe4c00cd9f0 2023-03-29T10:58:00.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.380+0000 7fe4c83ff700 1 -- 172.21.15.111:0/2039102274 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c00ccc90 con 0x7fe4c00cd9f0 2023-03-29T10:58:00.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.381+0000 7fe4b6ffd700 1 -- 172.21.15.111:0/2039102274 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fe4bc007450 con 0x7fe4c00cd9f0 2023-03-29T10:58:00.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.382+0000 7fe4b6ffd700 1 --2- 172.21.15.111:0/2039102274 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe4a4069e10 0x7fe4a406c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:00.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.382+0000 7fe4b6ffd700 1 -- 172.21.15.111:0/2039102274 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe4bc0872d0 con 0x7fe4c00cd9f0 2023-03-29T10:58:00.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.382+0000 7fe4c599a700 1 --2- 172.21.15.111:0/2039102274 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe4a4069e10 0x7fe4a406c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:00.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.383+0000 7fe4c599a700 1 --2- 172.21.15.111:0/2039102274 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe4a4069e10 0x7fe4a406c2c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe4c015e120 tx=0x7fe4b0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:00.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.386+0000 7fe4b6ffd700 1 -- 172.21.15.111:0/2039102274 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fe4bc052880 con 0x7fe4c00cd9f0 2023-03-29T10:58:00.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.575+0000 7fe4c83ff700 1 -- 172.21.15.111:0/2039102274 --> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4c0159890 con 0x7fe4a4069e10 2023-03-29T10:58:00.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.585+0000 7fe4b6ffd700 1 -- 172.21.15.111:0/2039102274 <== mgr.4103 v2:172.21.15.111:6832/110141 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367293 (secure 0 0 0) 0x7fe4c0159890 con 0x7fe4a4069e10 2023-03-29T10:58:00.590 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:00.592 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.590+0000 7fe4c83ff700 1 -- 172.21.15.111:0/2039102274 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe4a4069e10 msgr2=0x7fe4a406c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:00.592 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.590+0000 7fe4c83ff700 1 --2- 172.21.15.111:0/2039102274 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe4a4069e10 0x7fe4a406c2c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe4c015e120 tx=0x7fe4b0008030 comp rx=0 tx=0).stop 2023-03-29T10:58:00.592 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.590+0000 7fe4c83ff700 1 -- 172.21.15.111:0/2039102274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c00cd9f0 msgr2=0x7fe4c0145c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:00.592 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.590+0000 7fe4c83ff700 1 --2- 172.21.15.111:0/2039102274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c00cd9f0 0x7fe4c0145c00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc00c780 tx=0x7fe4bc00ca90 comp rx=0 tx=0).stop 2023-03-29T10:58:00.592 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.590+0000 7fe4c83ff700 1 -- 172.21.15.111:0/2039102274 shutdown_connections 2023-03-29T10:58:00.592 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.590+0000 7fe4c83ff700 1 --2- 172.21.15.111:0/2039102274 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe4a4069e10 0x7fe4a406c2c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:00.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.590+0000 7fe4c83ff700 1 --2- 172.21.15.111:0/2039102274 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c00cd9f0 0x7fe4c0145c00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:00.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.590+0000 7fe4c83ff700 1 --2- 172.21.15.111:0/2039102274 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c015dce0 0x7fe4c01f1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:00.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.590+0000 7fe4c83ff700 1 --2- 172.21.15.111:0/2039102274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c01594c0 0x7fe4c00c5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:00.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.590+0000 7fe4c83ff700 1 -- 172.21.15.111:0/2039102274 >> 172.21.15.111:0/2039102274 conn(0x7fe4c00cb6c0 msgr2=0x7fe4c00ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:00.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.590+0000 7fe4c83ff700 1 -- 172.21.15.111:0/2039102274 shutdown_connections 2023-03-29T10:58:00.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.590+0000 7fe4c83ff700 1 -- 172.21.15.111:0/2039102274 wait complete. 2023-03-29T10:58:00.593 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2023-03-29T10:58:00.607 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":44,"stamp":"2023-03-29T10:58:00.133162+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594966,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67584,"kb_used_data":3584,"kb_used_omap":0,"kb_used_meta":63936,"kb_avail":754907136,"statfs":{"total":773094113280,"available":773024907264,"internally_reserved":0,"allocated":3670016,"data_stored":2185270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65470464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003055"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.845704+0000","last_change":"2023-03-29T10:57:27.854795+0000","last_active":"2023-03-29T10:57:31.845704+0000","last_peered":"2023-03-29T10:57:31.845704+0000","last_clean":"2023-03-29T10:57:31.845704+0000","last_became_active":"2023-03-29T10:57:27.854521+0000","last_became_peered":"2023-03-29T10:57:27.854521+0000","last_unstale":"2023-03-29T10:57:31.845704+0000","last_undegraded":"2023-03-29T10:57:31.845704+0000","last_fullsized":"2023-03-29T10:57:31.845704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:40:19.611213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.862071+0000","last_change":"2023-03-29T10:57:27.850091+0000","last_active":"2023-03-29T10:57:31.862071+0000","last_peered":"2023-03-29T10:57:31.862071+0000","last_clean":"2023-03-29T10:57:31.862071+0000","last_became_active":"2023-03-29T10:57:27.849795+0000","last_became_peered":"2023-03-29T10:57:27.849795+0000","last_unstale":"2023-03-29T10:57:31.862071+0000","last_undegraded":"2023-03-29T10:57:31.862071+0000","last_fullsized":"2023-03-29T10:57:31.862071+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:36:09.200571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853118+0000","last_change":"2023-03-29T10:57:27.864238+0000","last_active":"2023-03-29T10:57:31.853118+0000","last_peered":"2023-03-29T10:57:31.853118+0000","last_clean":"2023-03-29T10:57:31.853118+0000","last_became_active":"2023-03-29T10:57:27.863878+0000","last_became_peered":"2023-03-29T10:57:27.863878+0000","last_unstale":"2023-03-29T10:57:31.853118+0000","last_undegraded":"2023-03-29T10:57:31.853118+0000","last_fullsized":"2023-03-29T10:57:31.853118+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:01:58.615750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854397+0000","last_change":"2023-03-29T10:57:27.861054+0000","last_active":"2023-03-29T10:57:31.854397+0000","last_peered":"2023-03-29T10:57:31.854397+0000","last_clean":"2023-03-29T10:57:31.854397+0000","last_became_active":"2023-03-29T10:57:27.859547+0000","last_became_peered":"2023-03-29T10:57:27.859547+0000","last_unstale":"2023-03-29T10:57:31.854397+0000","last_undegraded":"2023-03-29T10:57:31.854397+0000","last_fullsized":"2023-03-29T10:57:31.854397+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:24:00.034416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854082+0000","last_change":"2023-03-29T10:57:27.855728+0000","last_active":"2023-03-29T10:57:31.854082+0000","last_peered":"2023-03-29T10:57:31.854082+0000","last_clean":"2023-03-29T10:57:31.854082+0000","last_became_active":"2023-03-29T10:57:27.855438+0000","last_became_peered":"2023-03-29T10:57:27.855438+0000","last_unstale":"2023-03-29T10:57:31.854082+0000","last_undegraded":"2023-03-29T10:57:31.854082+0000","last_fullsized":"2023-03-29T10:57:31.854082+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:33:37.107795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852905+0000","last_change":"2023-03-29T10:57:27.861795+0000","last_active":"2023-03-29T10:57:31.852905+0000","last_peered":"2023-03-29T10:57:31.852905+0000","last_clean":"2023-03-29T10:57:31.852905+0000","last_became_active":"2023-03-29T10:57:27.860751+0000","last_became_peered":"2023-03-29T10:57:27.860751+0000","last_unstale":"2023-03-29T10:57:31.852905+0000","last_undegraded":"2023-03-29T10:57:31.852905+0000","last_fullsized":"2023-03-29T10:57:31.852905+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:41.882164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847170+0000","last_change":"2023-03-29T10:57:27.855143+0000","last_active":"2023-03-29T10:57:31.847170+0000","last_peered":"2023-03-29T10:57:31.847170+0000","last_clean":"2023-03-29T10:57:31.847170+0000","last_became_active":"2023-03-29T10:57:27.854804+0000","last_became_peered":"2023-03-29T10:57:27.854804+0000","last_unstale":"2023-03-29T10:57:31.847170+0000","last_undegraded":"2023-03-29T10:57:31.847170+0000","last_fullsized":"2023-03-29T10:57:31.847170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:24:08.253556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861416+0000","last_change":"2023-03-29T10:57:27.864339+0000","last_active":"2023-03-29T10:57:31.861416+0000","last_peered":"2023-03-29T10:57:31.861416+0000","last_clean":"2023-03-29T10:57:31.861416+0000","last_became_active":"2023-03-29T10:57:27.864034+0000","last_became_peered":"2023-03-29T10:57:27.864034+0000","last_unstale":"2023-03-29T10:57:31.861416+0000","last_undegraded":"2023-03-29T10:57:31.861416+0000","last_fullsized":"2023-03-29T10:57:31.861416+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:10:11.279944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851971+0000","last_change":"2023-03-29T10:57:27.849118+0000","last_active":"2023-03-29T10:57:31.851971+0000","last_peered":"2023-03-29T10:57:31.851971+0000","last_clean":"2023-03-29T10:57:31.851971+0000","last_became_active":"2023-03-29T10:57:27.848724+0000","last_became_peered":"2023-03-29T10:57:27.848724+0000","last_unstale":"2023-03-29T10:57:31.851971+0000","last_undegraded":"2023-03-29T10:57:31.851971+0000","last_fullsized":"2023-03-29T10:57:31.851971+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:36:09.200571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.862264+0000","last_change":"2023-03-29T10:57:27.848373+0000","last_active":"2023-03-29T10:57:31.862264+0000","last_peered":"2023-03-29T10:57:31.862264+0000","last_clean":"2023-03-29T10:57:31.862264+0000","last_became_active":"2023-03-29T10:57:27.847792+0000","last_became_peered":"2023-03-29T10:57:27.847792+0000","last_unstale":"2023-03-29T10:57:31.862264+0000","last_undegraded":"2023-03-29T10:57:31.862264+0000","last_fullsized":"2023-03-29T10:57:31.862264+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:46:29.403965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853207+0000","last_change":"2023-03-29T10:57:27.864182+0000","last_active":"2023-03-29T10:57:31.853207+0000","last_peered":"2023-03-29T10:57:31.853207+0000","last_clean":"2023-03-29T10:57:31.853207+0000","last_became_active":"2023-03-29T10:57:27.863567+0000","last_became_peered":"2023-03-29T10:57:27.863567+0000","last_unstale":"2023-03-29T10:57:31.853207+0000","last_undegraded":"2023-03-29T10:57:31.853207+0000","last_fullsized":"2023-03-29T10:57:31.853207+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:52:36.730410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.845640+0000","last_change":"2023-03-29T10:57:27.863745+0000","last_active":"2023-03-29T10:57:31.845640+0000","last_peered":"2023-03-29T10:57:31.845640+0000","last_clean":"2023-03-29T10:57:31.845640+0000","last_became_active":"2023-03-29T10:57:27.863377+0000","last_became_peered":"2023-03-29T10:57:27.863377+0000","last_unstale":"2023-03-29T10:57:31.845640+0000","last_undegraded":"2023-03-29T10:57:31.845640+0000","last_fullsized":"2023-03-29T10:57:31.845640+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:53:39.143611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846141+0000","last_change":"2023-03-29T10:57:27.856601+0000","last_active":"2023-03-29T10:57:31.846141+0000","last_peered":"2023-03-29T10:57:31.846141+0000","last_clean":"2023-03-29T10:57:31.846141+0000","last_became_active":"2023-03-29T10:57:27.856323+0000","last_became_peered":"2023-03-29T10:57:27.856323+0000","last_unstale":"2023-03-29T10:57:31.846141+0000","last_undegraded":"2023-03-29T10:57:31.846141+0000","last_fullsized":"2023-03-29T10:57:31.846141+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:12:15.104266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.863135+0000","last_change":"2023-03-29T10:57:27.850723+0000","last_active":"2023-03-29T10:57:31.863135+0000","last_peered":"2023-03-29T10:57:31.863135+0000","last_clean":"2023-03-29T10:57:31.863135+0000","last_became_active":"2023-03-29T10:57:27.844480+0000","last_became_peered":"2023-03-29T10:57:27.844480+0000","last_unstale":"2023-03-29T10:57:31.863135+0000","last_undegraded":"2023-03-29T10:57:31.863135+0000","last_fullsized":"2023-03-29T10:57:31.863135+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:07:19.154573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847070+0000","last_change":"2023-03-29T10:57:27.855233+0000","last_active":"2023-03-29T10:57:31.847070+0000","last_peered":"2023-03-29T10:57:31.847070+0000","last_clean":"2023-03-29T10:57:31.847070+0000","last_became_active":"2023-03-29T10:57:27.854970+0000","last_became_peered":"2023-03-29T10:57:27.854970+0000","last_unstale":"2023-03-29T10:57:31.847070+0000","last_undegraded":"2023-03-29T10:57:31.847070+0000","last_fullsized":"2023-03-29T10:57:31.847070+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:46:29.403965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860076+0000","last_change":"2023-03-29T10:57:27.860037+0000","last_active":"2023-03-29T10:57:31.860076+0000","last_peered":"2023-03-29T10:57:31.860076+0000","last_clean":"2023-03-29T10:57:31.860076+0000","last_became_active":"2023-03-29T10:57:27.859764+0000","last_became_peered":"2023-03-29T10:57:27.859764+0000","last_unstale":"2023-03-29T10:57:31.860076+0000","last_undegraded":"2023-03-29T10:57:31.860076+0000","last_fullsized":"2023-03-29T10:57:31.860076+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:25:48.508072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851770+0000","last_change":"2023-03-29T10:57:27.848952+0000","last_active":"2023-03-29T10:57:31.851770+0000","last_peered":"2023-03-29T10:57:31.851770+0000","last_clean":"2023-03-29T10:57:31.851770+0000","last_became_active":"2023-03-29T10:57:27.848390+0000","last_became_peered":"2023-03-29T10:57:27.848390+0000","last_unstale":"2023-03-29T10:57:31.851770+0000","last_undegraded":"2023-03-29T10:57:31.851770+0000","last_fullsized":"2023-03-29T10:57:31.851770+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:07:19.154573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854647+0000","last_change":"2023-03-29T10:57:27.862653+0000","last_active":"2023-03-29T10:57:31.854647+0000","last_peered":"2023-03-29T10:57:31.854647+0000","last_clean":"2023-03-29T10:57:31.854647+0000","last_became_active":"2023-03-29T10:57:27.862108+0000","last_became_peered":"2023-03-29T10:57:27.862108+0000","last_unstale":"2023-03-29T10:57:31.854647+0000","last_undegraded":"2023-03-29T10:57:31.854647+0000","last_fullsized":"2023-03-29T10:57:31.854647+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:06.878410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846508+0000","last_change":"2023-03-29T10:57:27.859887+0000","last_active":"2023-03-29T10:57:31.846508+0000","last_peered":"2023-03-29T10:57:31.846508+0000","last_clean":"2023-03-29T10:57:31.846508+0000","last_became_active":"2023-03-29T10:57:27.859632+0000","last_became_peered":"2023-03-29T10:57:27.859632+0000","last_unstale":"2023-03-29T10:57:31.846508+0000","last_undegraded":"2023-03-29T10:57:31.846508+0000","last_fullsized":"2023-03-29T10:57:31.846508+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:17.321564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854180+0000","last_change":"2023-03-29T10:57:27.865228+0000","last_active":"2023-03-29T10:57:31.854180+0000","last_peered":"2023-03-29T10:57:31.854180+0000","last_clean":"2023-03-29T10:57:31.854180+0000","last_became_active":"2023-03-29T10:57:27.864919+0000","last_became_peered":"2023-03-29T10:57:27.864919+0000","last_unstale":"2023-03-29T10:57:31.854180+0000","last_undegraded":"2023-03-29T10:57:31.854180+0000","last_fullsized":"2023-03-29T10:57:31.854180+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:44:47.553332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846978+0000","last_change":"2023-03-29T10:57:27.853377+0000","last_active":"2023-03-29T10:57:31.846978+0000","last_peered":"2023-03-29T10:57:31.846978+0000","last_clean":"2023-03-29T10:57:31.846978+0000","last_became_active":"2023-03-29T10:57:27.853039+0000","last_became_peered":"2023-03-29T10:57:27.853039+0000","last_unstale":"2023-03-29T10:57:31.846978+0000","last_undegraded":"2023-03-29T10:57:31.846978+0000","last_fullsized":"2023-03-29T10:57:31.846978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:23:24.586546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851362+0000","last_change":"2023-03-29T10:57:27.849931+0000","last_active":"2023-03-29T10:57:31.851362+0000","last_peered":"2023-03-29T10:57:31.851362+0000","last_clean":"2023-03-29T10:57:31.851362+0000","last_became_active":"2023-03-29T10:57:27.849662+0000","last_became_peered":"2023-03-29T10:57:27.849662+0000","last_unstale":"2023-03-29T10:57:31.851362+0000","last_undegraded":"2023-03-29T10:57:31.851362+0000","last_fullsized":"2023-03-29T10:57:31.851362+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:25:48.508072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852622+0000","last_change":"2023-03-29T10:57:27.850427+0000","last_active":"2023-03-29T10:57:31.852622+0000","last_peered":"2023-03-29T10:57:31.852622+0000","last_clean":"2023-03-29T10:57:31.852622+0000","last_became_active":"2023-03-29T10:57:27.850140+0000","last_became_peered":"2023-03-29T10:57:27.850140+0000","last_unstale":"2023-03-29T10:57:31.852622+0000","last_undegraded":"2023-03-29T10:57:31.852622+0000","last_fullsized":"2023-03-29T10:57:31.852622+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:01:58.615750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847710+0000","last_change":"2023-03-29T10:57:27.854437+0000","last_active":"2023-03-29T10:57:31.847710+0000","last_peered":"2023-03-29T10:57:31.847710+0000","last_clean":"2023-03-29T10:57:31.847710+0000","last_became_active":"2023-03-29T10:57:27.853876+0000","last_became_peered":"2023-03-29T10:57:27.853876+0000","last_unstale":"2023-03-29T10:57:31.847710+0000","last_undegraded":"2023-03-29T10:57:31.847710+0000","last_fullsized":"2023-03-29T10:57:31.847710+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:33:37.107795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851569+0000","last_change":"2023-03-29T10:57:27.856343+0000","last_active":"2023-03-29T10:57:31.851569+0000","last_peered":"2023-03-29T10:57:31.851569+0000","last_clean":"2023-03-29T10:57:31.851569+0000","last_became_active":"2023-03-29T10:57:27.855843+0000","last_became_peered":"2023-03-29T10:57:27.855843+0000","last_unstale":"2023-03-29T10:57:31.851569+0000","last_undegraded":"2023-03-29T10:57:31.851569+0000","last_fullsized":"2023-03-29T10:57:31.851569+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:46:29.403965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852481+0000","last_change":"2023-03-29T10:57:27.846853+0000","last_active":"2023-03-29T10:57:31.852481+0000","last_peered":"2023-03-29T10:57:31.852481+0000","last_clean":"2023-03-29T10:57:31.852481+0000","last_became_active":"2023-03-29T10:57:27.846503+0000","last_became_peered":"2023-03-29T10:57:27.846503+0000","last_unstale":"2023-03-29T10:57:31.852481+0000","last_undegraded":"2023-03-29T10:57:31.852481+0000","last_fullsized":"2023-03-29T10:57:31.852481+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:37:35.403032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.864802+0000","last_change":"2023-03-29T10:57:27.850277+0000","last_active":"2023-03-29T10:57:31.864802+0000","last_peered":"2023-03-29T10:57:31.864802+0000","last_clean":"2023-03-29T10:57:31.864802+0000","last_became_active":"2023-03-29T10:57:27.849726+0000","last_became_peered":"2023-03-29T10:57:27.849726+0000","last_unstale":"2023-03-29T10:57:31.864802+0000","last_undegraded":"2023-03-29T10:57:31.864802+0000","last_fullsized":"2023-03-29T10:57:31.864802+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:25:48.508072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340375+0000","last_change":"2023-03-29T10:57:31.874510+0000","last_active":"2023-03-29T10:57:32.340375+0000","last_peered":"2023-03-29T10:57:32.340375+0000","last_clean":"2023-03-29T10:57:32.340375+0000","last_became_active":"2023-03-29T10:57:31.874055+0000","last_became_peered":"2023-03-29T10:57:31.874055+0000","last_unstale":"2023-03-29T10:57:32.340375+0000","last_undegraded":"2023-03-29T10:57:32.340375+0000","last_fullsized":"2023-03-29T10:57:32.340375+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:53:20.999237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852831+0000","last_change":"2023-03-29T10:57:27.366688+0000","last_active":"2023-03-29T10:57:31.852831+0000","last_peered":"2023-03-29T10:57:31.852831+0000","last_clean":"2023-03-29T10:57:31.852831+0000","last_became_active":"2023-03-29T10:57:27.366396+0000","last_became_peered":"2023-03-29T10:57:27.366396+0000","last_unstale":"2023-03-29T10:57:31.852831+0000","last_undegraded":"2023-03-29T10:57:31.852831+0000","last_fullsized":"2023-03-29T10:57:31.852831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:16.312585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853217+0000","last_change":"2023-03-29T10:57:27.851695+0000","last_active":"2023-03-29T10:57:31.853217+0000","last_peered":"2023-03-29T10:57:31.853217+0000","last_clean":"2023-03-29T10:57:31.853217+0000","last_became_active":"2023-03-29T10:57:27.851344+0000","last_became_peered":"2023-03-29T10:57:27.851344+0000","last_unstale":"2023-03-29T10:57:31.853217+0000","last_undegraded":"2023-03-29T10:57:31.853217+0000","last_fullsized":"2023-03-29T10:57:31.853217+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:53:38.186916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606792+0000","last_change":"2023-03-29T10:57:31.878011+0000","last_active":"2023-03-29T10:57:32.606792+0000","last_peered":"2023-03-29T10:57:32.606792+0000","last_clean":"2023-03-29T10:57:32.606792+0000","last_became_active":"2023-03-29T10:57:31.877524+0000","last_became_peered":"2023-03-29T10:57:31.877524+0000","last_unstale":"2023-03-29T10:57:32.606792+0000","last_undegraded":"2023-03-29T10:57:32.606792+0000","last_fullsized":"2023-03-29T10:57:32.606792+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:44:51.056937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.850681+0000","last_change":"2023-03-29T10:57:27.368907+0000","last_active":"2023-03-29T10:57:31.850681+0000","last_peered":"2023-03-29T10:57:31.850681+0000","last_clean":"2023-03-29T10:57:31.850681+0000","last_became_active":"2023-03-29T10:57:27.367137+0000","last_became_peered":"2023-03-29T10:57:27.367137+0000","last_unstale":"2023-03-29T10:57:31.850681+0000","last_undegraded":"2023-03-29T10:57:31.850681+0000","last_fullsized":"2023-03-29T10:57:31.850681+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:12:14.095287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853885+0000","last_change":"2023-03-29T10:57:27.844281+0000","last_active":"2023-03-29T10:57:31.853885+0000","last_peered":"2023-03-29T10:57:31.853885+0000","last_clean":"2023-03-29T10:57:31.853885+0000","last_became_active":"2023-03-29T10:57:27.844012+0000","last_became_peered":"2023-03-29T10:57:27.844012+0000","last_unstale":"2023-03-29T10:57:31.853885+0000","last_undegraded":"2023-03-29T10:57:31.853885+0000","last_fullsized":"2023-03-29T10:57:31.853885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:23:24.586546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.336612+0000","last_change":"2023-03-29T10:57:31.872397+0000","last_active":"2023-03-29T10:57:32.336612+0000","last_peered":"2023-03-29T10:57:32.336612+0000","last_clean":"2023-03-29T10:57:32.336612+0000","last_became_active":"2023-03-29T10:57:31.872073+0000","last_became_peered":"2023-03-29T10:57:31.872073+0000","last_unstale":"2023-03-29T10:57:32.336612+0000","last_undegraded":"2023-03-29T10:57:32.336612+0000","last_fullsized":"2023-03-29T10:57:32.336612+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:24:14.450310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849730+0000","last_change":"2023-03-29T10:57:27.362811+0000","last_active":"2023-03-29T10:57:31.849730+0000","last_peered":"2023-03-29T10:57:31.849730+0000","last_clean":"2023-03-29T10:57:31.849730+0000","last_became_active":"2023-03-29T10:57:27.362313+0000","last_became_peered":"2023-03-29T10:57:27.362313+0000","last_unstale":"2023-03-29T10:57:31.849730+0000","last_undegraded":"2023-03-29T10:57:31.849730+0000","last_fullsized":"2023-03-29T10:57:31.849730+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:57:18.786945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.857696+0000","last_change":"2023-03-29T10:57:27.853328+0000","last_active":"2023-03-29T10:57:31.857696+0000","last_peered":"2023-03-29T10:57:31.857696+0000","last_clean":"2023-03-29T10:57:31.857696+0000","last_became_active":"2023-03-29T10:57:27.852980+0000","last_became_peered":"2023-03-29T10:57:27.852980+0000","last_unstale":"2023-03-29T10:57:31.857696+0000","last_undegraded":"2023-03-29T10:57:31.857696+0000","last_fullsized":"2023-03-29T10:57:31.857696+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:35:25.612159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606385+0000","last_change":"2023-03-29T10:57:31.880406+0000","last_active":"2023-03-29T10:57:32.606385+0000","last_peered":"2023-03-29T10:57:32.606385+0000","last_clean":"2023-03-29T10:57:32.606385+0000","last_became_active":"2023-03-29T10:57:31.879910+0000","last_became_peered":"2023-03-29T10:57:31.879910+0000","last_unstale":"2023-03-29T10:57:32.606385+0000","last_undegraded":"2023-03-29T10:57:32.606385+0000","last_fullsized":"2023-03-29T10:57:32.606385+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:50:03.793655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852066+0000","last_change":"2023-03-29T10:57:27.361203+0000","last_active":"2023-03-29T10:57:31.852066+0000","last_peered":"2023-03-29T10:57:31.852066+0000","last_clean":"2023-03-29T10:57:31.852066+0000","last_became_active":"2023-03-29T10:57:27.360905+0000","last_became_peered":"2023-03-29T10:57:27.360905+0000","last_unstale":"2023-03-29T10:57:31.852066+0000","last_undegraded":"2023-03-29T10:57:31.852066+0000","last_fullsized":"2023-03-29T10:57:31.852066+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:53:38.134633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849652+0000","last_change":"2023-03-29T10:57:27.848599+0000","last_active":"2023-03-29T10:57:31.849652+0000","last_peered":"2023-03-29T10:57:31.849652+0000","last_clean":"2023-03-29T10:57:31.849652+0000","last_became_active":"2023-03-29T10:57:27.848300+0000","last_became_peered":"2023-03-29T10:57:27.848300+0000","last_unstale":"2023-03-29T10:57:31.849652+0000","last_undegraded":"2023-03-29T10:57:31.849652+0000","last_fullsized":"2023-03-29T10:57:31.849652+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:44:47.553332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340886+0000","last_change":"2023-03-29T10:57:31.875820+0000","last_active":"2023-03-29T10:57:32.340886+0000","last_peered":"2023-03-29T10:57:32.340886+0000","last_clean":"2023-03-29T10:57:32.340886+0000","last_became_active":"2023-03-29T10:57:31.875285+0000","last_became_peered":"2023-03-29T10:57:31.875285+0000","last_unstale":"2023-03-29T10:57:32.340886+0000","last_undegraded":"2023-03-29T10:57:32.340886+0000","last_fullsized":"2023-03-29T10:57:32.340886+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:05:02.661508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.862158+0000","last_change":"2023-03-29T10:57:27.369635+0000","last_active":"2023-03-29T10:57:31.862158+0000","last_peered":"2023-03-29T10:57:31.862158+0000","last_clean":"2023-03-29T10:57:31.862158+0000","last_became_active":"2023-03-29T10:57:27.369232+0000","last_became_peered":"2023-03-29T10:57:27.369232+0000","last_unstale":"2023-03-29T10:57:31.862158+0000","last_undegraded":"2023-03-29T10:57:31.862158+0000","last_fullsized":"2023-03-29T10:57:31.862158+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:45:12.326563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852705+0000","last_change":"2023-03-29T10:57:27.841973+0000","last_active":"2023-03-29T10:57:31.852705+0000","last_peered":"2023-03-29T10:57:31.852705+0000","last_clean":"2023-03-29T10:57:31.852705+0000","last_became_active":"2023-03-29T10:57:27.841668+0000","last_became_peered":"2023-03-29T10:57:27.841668+0000","last_unstale":"2023-03-29T10:57:31.852705+0000","last_undegraded":"2023-03-29T10:57:31.852705+0000","last_fullsized":"2023-03-29T10:57:31.852705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:25:48.508072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.608208+0000","last_change":"2023-03-29T10:57:31.867435+0000","last_active":"2023-03-29T10:57:32.608208+0000","last_peered":"2023-03-29T10:57:32.608208+0000","last_clean":"2023-03-29T10:57:32.608208+0000","last_became_active":"2023-03-29T10:57:31.867024+0000","last_became_peered":"2023-03-29T10:57:31.867024+0000","last_unstale":"2023-03-29T10:57:32.608208+0000","last_undegraded":"2023-03-29T10:57:32.608208+0000","last_fullsized":"2023-03-29T10:57:32.608208+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:33:59.083704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846915+0000","last_change":"2023-03-29T10:57:27.367580+0000","last_active":"2023-03-29T10:57:31.846915+0000","last_peered":"2023-03-29T10:57:31.846915+0000","last_clean":"2023-03-29T10:57:31.846915+0000","last_became_active":"2023-03-29T10:57:27.367297+0000","last_became_peered":"2023-03-29T10:57:27.367297+0000","last_unstale":"2023-03-29T10:57:31.846915+0000","last_undegraded":"2023-03-29T10:57:31.846915+0000","last_fullsized":"2023-03-29T10:57:31.846915+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:35:24.603180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851381+0000","last_change":"2023-03-29T10:57:27.868674+0000","last_active":"2023-03-29T10:57:31.851381+0000","last_peered":"2023-03-29T10:57:31.851381+0000","last_clean":"2023-03-29T10:57:31.851381+0000","last_became_active":"2023-03-29T10:57:27.868480+0000","last_became_peered":"2023-03-29T10:57:27.868480+0000","last_unstale":"2023-03-29T10:57:31.851381+0000","last_undegraded":"2023-03-29T10:57:31.851381+0000","last_fullsized":"2023-03-29T10:57:31.851381+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:24:08.253556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.543504+0000","last_change":"2023-03-29T10:57:31.865358+0000","last_active":"2023-03-29T10:57:32.543504+0000","last_peered":"2023-03-29T10:57:32.543504+0000","last_clean":"2023-03-29T10:57:32.543504+0000","last_became_active":"2023-03-29T10:57:31.864834+0000","last_became_peered":"2023-03-29T10:57:31.864834+0000","last_unstale":"2023-03-29T10:57:32.543504+0000","last_undegraded":"2023-03-29T10:57:32.543504+0000","last_fullsized":"2023-03-29T10:57:32.543504+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:40:55.333034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"28'10","reported_seq":25,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:44.820956+0000","last_change":"2023-03-29T10:57:27.361696+0000","last_active":"2023-03-29T10:57:44.820956+0000","last_peered":"2023-03-29T10:57:44.820956+0000","last_clean":"2023-03-29T10:57:44.820956+0000","last_became_active":"2023-03-29T10:57:27.360021+0000","last_became_peered":"2023-03-29T10:57:27.360021+0000","last_unstale":"2023-03-29T10:57:44.820956+0000","last_undegraded":"2023-03-29T10:57:44.820956+0000","last_fullsized":"2023-03-29T10:57:44.820956+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:57:18.786945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3796,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853468+0000","last_change":"2023-03-29T10:57:27.860824+0000","last_active":"2023-03-29T10:57:31.853468+0000","last_peered":"2023-03-29T10:57:31.853468+0000","last_clean":"2023-03-29T10:57:31.853468+0000","last_became_active":"2023-03-29T10:57:27.860139+0000","last_became_peered":"2023-03-29T10:57:27.860139+0000","last_unstale":"2023-03-29T10:57:31.853468+0000","last_undegraded":"2023-03-29T10:57:31.853468+0000","last_fullsized":"2023-03-29T10:57:31.853468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:10:11.279944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606214+0000","last_change":"2023-03-29T10:57:31.869234+0000","last_active":"2023-03-29T10:57:32.606214+0000","last_peered":"2023-03-29T10:57:32.606214+0000","last_clean":"2023-03-29T10:57:32.606214+0000","last_became_active":"2023-03-29T10:57:31.868479+0000","last_became_peered":"2023-03-29T10:57:31.868479+0000","last_unstale":"2023-03-29T10:57:32.606214+0000","last_undegraded":"2023-03-29T10:57:32.606214+0000","last_fullsized":"2023-03-29T10:57:32.606214+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:29:07.043419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846875+0000","last_change":"2023-03-29T10:57:27.363811+0000","last_active":"2023-03-29T10:57:31.846875+0000","last_peered":"2023-03-29T10:57:31.846875+0000","last_clean":"2023-03-29T10:57:31.846875+0000","last_became_active":"2023-03-29T10:57:27.362755+0000","last_became_peered":"2023-03-29T10:57:27.362755+0000","last_unstale":"2023-03-29T10:57:31.846875+0000","last_undegraded":"2023-03-29T10:57:31.846875+0000","last_fullsized":"2023-03-29T10:57:31.846875+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:45:12.326563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849249+0000","last_change":"2023-03-29T10:57:27.848024+0000","last_active":"2023-03-29T10:57:31.849249+0000","last_peered":"2023-03-29T10:57:31.849249+0000","last_clean":"2023-03-29T10:57:31.849249+0000","last_became_active":"2023-03-29T10:57:27.847742+0000","last_became_peered":"2023-03-29T10:57:27.847742+0000","last_unstale":"2023-03-29T10:57:31.849249+0000","last_undegraded":"2023-03-29T10:57:31.849249+0000","last_fullsized":"2023-03-29T10:57:31.849249+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:07:19.154573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.608195+0000","last_change":"2023-03-29T10:57:31.874176+0000","last_active":"2023-03-29T10:57:32.608195+0000","last_peered":"2023-03-29T10:57:32.608195+0000","last_clean":"2023-03-29T10:57:32.608195+0000","last_became_active":"2023-03-29T10:57:31.873748+0000","last_became_peered":"2023-03-29T10:57:31.873748+0000","last_unstale":"2023-03-29T10:57:32.608195+0000","last_undegraded":"2023-03-29T10:57:32.608195+0000","last_fullsized":"2023-03-29T10:57:32.608195+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:29:07.043419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851278+0000","last_change":"2023-03-29T10:57:27.353170+0000","last_active":"2023-03-29T10:57:31.851278+0000","last_peered":"2023-03-29T10:57:31.851278+0000","last_clean":"2023-03-29T10:57:31.851278+0000","last_became_active":"2023-03-29T10:57:27.352891+0000","last_became_peered":"2023-03-29T10:57:27.352891+0000","last_unstale":"2023-03-29T10:57:31.851278+0000","last_undegraded":"2023-03-29T10:57:31.851278+0000","last_fullsized":"2023-03-29T10:57:31.851278+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:42:03.146529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.864725+0000","last_change":"2023-03-29T10:57:27.850083+0000","last_active":"2023-03-29T10:57:31.864725+0000","last_peered":"2023-03-29T10:57:31.864725+0000","last_clean":"2023-03-29T10:57:31.864725+0000","last_became_active":"2023-03-29T10:57:27.849341+0000","last_became_peered":"2023-03-29T10:57:27.849341+0000","last_unstale":"2023-03-29T10:57:31.864725+0000","last_undegraded":"2023-03-29T10:57:31.864725+0000","last_fullsized":"2023-03-29T10:57:31.864725+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:41.882164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.608429+0000","last_change":"2023-03-29T10:57:31.874626+0000","last_active":"2023-03-29T10:57:32.608429+0000","last_peered":"2023-03-29T10:57:32.608429+0000","last_clean":"2023-03-29T10:57:32.608429+0000","last_became_active":"2023-03-29T10:57:31.874288+0000","last_became_peered":"2023-03-29T10:57:31.874288+0000","last_unstale":"2023-03-29T10:57:32.608429+0000","last_undegraded":"2023-03-29T10:57:32.608429+0000","last_fullsized":"2023-03-29T10:57:32.608429+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:10:08.859801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847314+0000","last_change":"2023-03-29T10:57:27.371046+0000","last_active":"2023-03-29T10:57:31.847314+0000","last_peered":"2023-03-29T10:57:31.847314+0000","last_clean":"2023-03-29T10:57:31.847314+0000","last_became_active":"2023-03-29T10:57:27.370703+0000","last_became_peered":"2023-03-29T10:57:27.370703+0000","last_unstale":"2023-03-29T10:57:31.847314+0000","last_undegraded":"2023-03-29T10:57:31.847314+0000","last_fullsized":"2023-03-29T10:57:31.847314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:00:38.166723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853955+0000","last_change":"2023-03-29T10:57:27.857805+0000","last_active":"2023-03-29T10:57:31.853955+0000","last_peered":"2023-03-29T10:57:31.853955+0000","last_clean":"2023-03-29T10:57:31.853955+0000","last_became_active":"2023-03-29T10:57:27.857520+0000","last_became_peered":"2023-03-29T10:57:27.857520+0000","last_unstale":"2023-03-29T10:57:31.853955+0000","last_undegraded":"2023-03-29T10:57:31.853955+0000","last_fullsized":"2023-03-29T10:57:31.853955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:24:08.253556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605664+0000","last_change":"2023-03-29T10:57:31.868093+0000","last_active":"2023-03-29T10:57:32.605664+0000","last_peered":"2023-03-29T10:57:32.605664+0000","last_clean":"2023-03-29T10:57:32.605664+0000","last_became_active":"2023-03-29T10:57:31.867516+0000","last_became_peered":"2023-03-29T10:57:31.867516+0000","last_unstale":"2023-03-29T10:57:32.605664+0000","last_undegraded":"2023-03-29T10:57:32.605664+0000","last_fullsized":"2023-03-29T10:57:32.605664+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:56:12.713461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852187+0000","last_change":"2023-03-29T10:57:27.366626+0000","last_active":"2023-03-29T10:57:31.852187+0000","last_peered":"2023-03-29T10:57:31.852187+0000","last_clean":"2023-03-29T10:57:31.852187+0000","last_became_active":"2023-03-29T10:57:27.366259+0000","last_became_peered":"2023-03-29T10:57:27.366259+0000","last_unstale":"2023-03-29T10:57:31.852187+0000","last_undegraded":"2023-03-29T10:57:31.852187+0000","last_fullsized":"2023-03-29T10:57:31.852187+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:01:05.024349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852636+0000","last_change":"2023-03-29T10:57:27.842305+0000","last_active":"2023-03-29T10:57:31.852636+0000","last_peered":"2023-03-29T10:57:31.852636+0000","last_clean":"2023-03-29T10:57:31.852636+0000","last_became_active":"2023-03-29T10:57:27.841993+0000","last_became_peered":"2023-03-29T10:57:27.841993+0000","last_unstale":"2023-03-29T10:57:31.852636+0000","last_undegraded":"2023-03-29T10:57:31.852636+0000","last_fullsized":"2023-03-29T10:57:31.852636+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:41.882164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340179+0000","last_change":"2023-03-29T10:57:31.874901+0000","last_active":"2023-03-29T10:57:32.340179+0000","last_peered":"2023-03-29T10:57:32.340179+0000","last_clean":"2023-03-29T10:57:32.340179+0000","last_became_active":"2023-03-29T10:57:31.874412+0000","last_became_peered":"2023-03-29T10:57:31.874412+0000","last_unstale":"2023-03-29T10:57:32.340179+0000","last_undegraded":"2023-03-29T10:57:32.340179+0000","last_fullsized":"2023-03-29T10:57:32.340179+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:19:12.936732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847695+0000","last_change":"2023-03-29T10:57:27.363942+0000","last_active":"2023-03-29T10:57:31.847695+0000","last_peered":"2023-03-29T10:57:31.847695+0000","last_clean":"2023-03-29T10:57:31.847695+0000","last_became_active":"2023-03-29T10:57:27.363509+0000","last_became_peered":"2023-03-29T10:57:27.363509+0000","last_unstale":"2023-03-29T10:57:31.847695+0000","last_undegraded":"2023-03-29T10:57:31.847695+0000","last_fullsized":"2023-03-29T10:57:31.847695+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:23:59.025438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861310+0000","last_change":"2023-03-29T10:57:27.849632+0000","last_active":"2023-03-29T10:57:31.861310+0000","last_peered":"2023-03-29T10:57:31.861310+0000","last_clean":"2023-03-29T10:57:31.861310+0000","last_became_active":"2023-03-29T10:57:27.849341+0000","last_became_peered":"2023-03-29T10:57:27.849341+0000","last_unstale":"2023-03-29T10:57:31.861310+0000","last_undegraded":"2023-03-29T10:57:31.861310+0000","last_fullsized":"2023-03-29T10:57:31.861310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:01:58.615750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.604803+0000","last_change":"2023-03-29T10:57:31.864416+0000","last_active":"2023-03-29T10:57:32.604803+0000","last_peered":"2023-03-29T10:57:32.604803+0000","last_clean":"2023-03-29T10:57:32.604803+0000","last_became_active":"2023-03-29T10:57:31.864021+0000","last_became_peered":"2023-03-29T10:57:31.864021+0000","last_unstale":"2023-03-29T10:57:32.604803+0000","last_undegraded":"2023-03-29T10:57:32.604803+0000","last_fullsized":"2023-03-29T10:57:32.604803+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:44:51.056937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847268+0000","last_change":"2023-03-29T10:57:27.365113+0000","last_active":"2023-03-29T10:57:31.847268+0000","last_peered":"2023-03-29T10:57:31.847268+0000","last_clean":"2023-03-29T10:57:31.847268+0000","last_became_active":"2023-03-29T10:57:27.363593+0000","last_became_peered":"2023-03-29T10:57:27.363593+0000","last_unstale":"2023-03-29T10:57:31.847268+0000","last_undegraded":"2023-03-29T10:57:31.847268+0000","last_fullsized":"2023-03-29T10:57:31.847268+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:10:38.094232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860900+0000","last_change":"2023-03-29T10:57:27.861487+0000","last_active":"2023-03-29T10:57:31.860900+0000","last_peered":"2023-03-29T10:57:31.860900+0000","last_clean":"2023-03-29T10:57:31.860900+0000","last_became_active":"2023-03-29T10:57:27.861218+0000","last_became_peered":"2023-03-29T10:57:27.861218+0000","last_unstale":"2023-03-29T10:57:31.860900+0000","last_undegraded":"2023-03-29T10:57:31.860900+0000","last_fullsized":"2023-03-29T10:57:31.860900+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:04:59.157904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.604937+0000","last_change":"2023-03-29T10:57:31.871267+0000","last_active":"2023-03-29T10:57:32.604937+0000","last_peered":"2023-03-29T10:57:32.604937+0000","last_clean":"2023-03-29T10:57:32.604937+0000","last_became_active":"2023-03-29T10:57:31.870814+0000","last_became_peered":"2023-03-29T10:57:31.870814+0000","last_unstale":"2023-03-29T10:57:32.604937+0000","last_undegraded":"2023-03-29T10:57:32.604937+0000","last_fullsized":"2023-03-29T10:57:32.604937+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:57:20.392149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847436+0000","last_change":"2023-03-29T10:57:27.367428+0000","last_active":"2023-03-29T10:57:31.847436+0000","last_peered":"2023-03-29T10:57:31.847436+0000","last_clean":"2023-03-29T10:57:31.847436+0000","last_became_active":"2023-03-29T10:57:27.367100+0000","last_became_peered":"2023-03-29T10:57:27.367100+0000","last_unstale":"2023-03-29T10:57:31.847436+0000","last_undegraded":"2023-03-29T10:57:31.847436+0000","last_fullsized":"2023-03-29T10:57:31.847436+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:52:59.436152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.858073+0000","last_change":"2023-03-29T10:57:27.863156+0000","last_active":"2023-03-29T10:57:31.858073+0000","last_peered":"2023-03-29T10:57:31.858073+0000","last_clean":"2023-03-29T10:57:31.858073+0000","last_became_active":"2023-03-29T10:57:27.862805+0000","last_became_peered":"2023-03-29T10:57:27.862805+0000","last_unstale":"2023-03-29T10:57:31.858073+0000","last_undegraded":"2023-03-29T10:57:31.858073+0000","last_fullsized":"2023-03-29T10:57:31.858073+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:32:05.407357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605019+0000","last_change":"2023-03-29T10:57:31.878616+0000","last_active":"2023-03-29T10:57:32.605019+0000","last_peered":"2023-03-29T10:57:32.605019+0000","last_clean":"2023-03-29T10:57:32.605019+0000","last_became_active":"2023-03-29T10:57:31.878248+0000","last_became_peered":"2023-03-29T10:57:31.878248+0000","last_unstale":"2023-03-29T10:57:32.605019+0000","last_undegraded":"2023-03-29T10:57:32.605019+0000","last_fullsized":"2023-03-29T10:57:32.605019+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:53:41.690520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861723+0000","last_change":"2023-03-29T10:57:27.370499+0000","last_active":"2023-03-29T10:57:31.861723+0000","last_peered":"2023-03-29T10:57:31.861723+0000","last_clean":"2023-03-29T10:57:31.861723+0000","last_became_active":"2023-03-29T10:57:27.370048+0000","last_became_peered":"2023-03-29T10:57:27.370048+0000","last_unstale":"2023-03-29T10:57:31.861723+0000","last_undegraded":"2023-03-29T10:57:31.861723+0000","last_fullsized":"2023-03-29T10:57:31.861723+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:52:59.436152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854114+0000","last_change":"2023-03-29T10:57:27.848004+0000","last_active":"2023-03-29T10:57:31.854114+0000","last_peered":"2023-03-29T10:57:31.854114+0000","last_clean":"2023-03-29T10:57:31.854114+0000","last_became_active":"2023-03-29T10:57:27.847731+0000","last_became_peered":"2023-03-29T10:57:27.847731+0000","last_unstale":"2023-03-29T10:57:31.854114+0000","last_undegraded":"2023-03-29T10:57:31.854114+0000","last_fullsized":"2023-03-29T10:57:31.854114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:41.882164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605202+0000","last_change":"2023-03-29T10:57:31.868289+0000","last_active":"2023-03-29T10:57:32.605202+0000","last_peered":"2023-03-29T10:57:32.605202+0000","last_clean":"2023-03-29T10:57:32.605202+0000","last_became_active":"2023-03-29T10:57:31.867905+0000","last_became_peered":"2023-03-29T10:57:31.867905+0000","last_unstale":"2023-03-29T10:57:32.605202+0000","last_undegraded":"2023-03-29T10:57:32.605202+0000","last_fullsized":"2023-03-29T10:57:32.605202+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:23:28.090151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851809+0000","last_change":"2023-03-29T10:57:27.368350+0000","last_active":"2023-03-29T10:57:31.851809+0000","last_peered":"2023-03-29T10:57:31.851809+0000","last_clean":"2023-03-29T10:57:31.851809+0000","last_became_active":"2023-03-29T10:57:27.367743+0000","last_became_peered":"2023-03-29T10:57:27.367743+0000","last_unstale":"2023-03-29T10:57:31.851809+0000","last_undegraded":"2023-03-29T10:57:31.851809+0000","last_fullsized":"2023-03-29T10:57:31.851809+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:16.312585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853838+0000","last_change":"2023-03-29T10:57:27.862273+0000","last_active":"2023-03-29T10:57:31.853838+0000","last_peered":"2023-03-29T10:57:31.853838+0000","last_clean":"2023-03-29T10:57:31.853838+0000","last_became_active":"2023-03-29T10:57:27.861893+0000","last_became_peered":"2023-03-29T10:57:27.861893+0000","last_unstale":"2023-03-29T10:57:31.853838+0000","last_undegraded":"2023-03-29T10:57:31.853838+0000","last_fullsized":"2023-03-29T10:57:31.853838+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:34:37.113840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.543869+0000","last_change":"2023-03-29T10:57:31.877007+0000","last_active":"2023-03-29T10:57:32.543869+0000","last_peered":"2023-03-29T10:57:32.543869+0000","last_clean":"2023-03-29T10:57:32.543869+0000","last_became_active":"2023-03-29T10:57:31.876649+0000","last_became_peered":"2023-03-29T10:57:31.876649+0000","last_unstale":"2023-03-29T10:57:32.543869+0000","last_undegraded":"2023-03-29T10:57:32.543869+0000","last_fullsized":"2023-03-29T10:57:32.543869+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:19:12.936732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.858178+0000","last_change":"2023-03-29T10:57:27.370469+0000","last_active":"2023-03-29T10:57:31.858178+0000","last_peered":"2023-03-29T10:57:31.858178+0000","last_clean":"2023-03-29T10:57:31.858178+0000","last_became_active":"2023-03-29T10:57:27.370194+0000","last_became_peered":"2023-03-29T10:57:27.370194+0000","last_unstale":"2023-03-29T10:57:31.858178+0000","last_undegraded":"2023-03-29T10:57:31.858178+0000","last_fullsized":"2023-03-29T10:57:31.858178+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:42:03.146529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853028+0000","last_change":"2023-03-29T10:57:27.863980+0000","last_active":"2023-03-29T10:57:31.853028+0000","last_peered":"2023-03-29T10:57:31.853028+0000","last_clean":"2023-03-29T10:57:31.853028+0000","last_became_active":"2023-03-29T10:57:27.863503+0000","last_became_peered":"2023-03-29T10:57:27.863503+0000","last_unstale":"2023-03-29T10:57:31.853028+0000","last_undegraded":"2023-03-29T10:57:31.853028+0000","last_fullsized":"2023-03-29T10:57:31.853028+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:36:09.200571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606340+0000","last_change":"2023-03-29T10:57:31.880859+0000","last_active":"2023-03-29T10:57:32.606340+0000","last_peered":"2023-03-29T10:57:32.606340+0000","last_clean":"2023-03-29T10:57:32.606340+0000","last_became_active":"2023-03-29T10:57:31.879872+0000","last_became_peered":"2023-03-29T10:57:31.879872+0000","last_unstale":"2023-03-29T10:57:32.606340+0000","last_undegraded":"2023-03-29T10:57:32.606340+0000","last_fullsized":"2023-03-29T10:57:32.606340+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:40:04.971172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860797+0000","last_change":"2023-03-29T10:57:27.359363+0000","last_active":"2023-03-29T10:57:31.860797+0000","last_peered":"2023-03-29T10:57:31.860797+0000","last_clean":"2023-03-29T10:57:31.860797+0000","last_became_active":"2023-03-29T10:57:27.359054+0000","last_became_peered":"2023-03-29T10:57:27.359054+0000","last_unstale":"2023-03-29T10:57:31.860797+0000","last_undegraded":"2023-03-29T10:57:31.860797+0000","last_fullsized":"2023-03-29T10:57:31.860797+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:42:03.146529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.854683+0000","last_change":"2023-03-29T10:57:27.862443+0000","last_active":"2023-03-29T10:57:31.854683+0000","last_peered":"2023-03-29T10:57:31.854683+0000","last_clean":"2023-03-29T10:57:31.854683+0000","last_became_active":"2023-03-29T10:57:27.861796+0000","last_became_peered":"2023-03-29T10:57:27.861796+0000","last_unstale":"2023-03-29T10:57:31.854683+0000","last_undegraded":"2023-03-29T10:57:31.854683+0000","last_fullsized":"2023-03-29T10:57:31.854683+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:50:00.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340645+0000","last_change":"2023-03-29T10:57:31.875674+0000","last_active":"2023-03-29T10:57:32.340645+0000","last_peered":"2023-03-29T10:57:32.340645+0000","last_clean":"2023-03-29T10:57:32.340645+0000","last_became_active":"2023-03-29T10:57:31.875240+0000","last_became_peered":"2023-03-29T10:57:31.875240+0000","last_unstale":"2023-03-29T10:57:32.340645+0000","last_undegraded":"2023-03-29T10:57:32.340645+0000","last_fullsized":"2023-03-29T10:57:32.340645+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:36:10.088665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849570+0000","last_change":"2023-03-29T10:57:27.366426+0000","last_active":"2023-03-29T10:57:31.849570+0000","last_peered":"2023-03-29T10:57:31.849570+0000","last_clean":"2023-03-29T10:57:31.849570+0000","last_became_active":"2023-03-29T10:57:27.366129+0000","last_became_peered":"2023-03-29T10:57:27.366129+0000","last_unstale":"2023-03-29T10:57:31.849570+0000","last_undegraded":"2023-03-29T10:57:31.849570+0000","last_fullsized":"2023-03-29T10:57:31.849570+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:42:03.146529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851728+0000","last_change":"2023-03-29T10:57:27.856138+0000","last_active":"2023-03-29T10:57:31.851728+0000","last_peered":"2023-03-29T10:57:31.851728+0000","last_clean":"2023-03-29T10:57:31.851728+0000","last_became_active":"2023-03-29T10:57:27.855392+0000","last_became_peered":"2023-03-29T10:57:27.855392+0000","last_unstale":"2023-03-29T10:57:31.851728+0000","last_undegraded":"2023-03-29T10:57:31.851728+0000","last_fullsized":"2023-03-29T10:57:31.851728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:12:10.758718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.336574+0000","last_change":"2023-03-29T10:57:31.872511+0000","last_active":"2023-03-29T10:57:32.336574+0000","last_peered":"2023-03-29T10:57:32.336574+0000","last_clean":"2023-03-29T10:57:32.336574+0000","last_became_active":"2023-03-29T10:57:31.872088+0000","last_became_peered":"2023-03-29T10:57:31.872088+0000","last_unstale":"2023-03-29T10:57:32.336574+0000","last_undegraded":"2023-03-29T10:57:32.336574+0000","last_fullsized":"2023-03-29T10:57:32.336574+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:28:41.336878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852880+0000","last_change":"2023-03-29T10:57:22.318583+0000","last_active":"2023-03-29T10:57:31.852880+0000","last_peered":"2023-03-29T10:57:31.852880+0000","last_clean":"2023-03-29T10:57:31.852880+0000","last_became_active":"2023-03-29T10:57:22.318223+0000","last_became_peered":"2023-03-29T10:57:22.318223+0000","last_unstale":"2023-03-29T10:57:31.852880+0000","last_undegraded":"2023-03-29T10:57:31.852880+0000","last_fullsized":"2023-03-29T10:57:31.852880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:31:03.349241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.864963+0000","last_change":"2023-03-29T10:57:27.369314+0000","last_active":"2023-03-29T10:57:31.864963+0000","last_peered":"2023-03-29T10:57:31.864963+0000","last_clean":"2023-03-29T10:57:31.864963+0000","last_became_active":"2023-03-29T10:57:27.368989+0000","last_became_peered":"2023-03-29T10:57:27.368989+0000","last_unstale":"2023-03-29T10:57:31.864963+0000","last_undegraded":"2023-03-29T10:57:31.864963+0000","last_fullsized":"2023-03-29T10:57:31.864963+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:35:24.603180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851286+0000","last_change":"2023-03-29T10:57:27.860136+0000","last_active":"2023-03-29T10:57:31.851286+0000","last_peered":"2023-03-29T10:57:31.851286+0000","last_clean":"2023-03-29T10:57:31.851286+0000","last_became_active":"2023-03-29T10:57:27.859888+0000","last_became_peered":"2023-03-29T10:57:27.859888+0000","last_unstale":"2023-03-29T10:57:31.851286+0000","last_undegraded":"2023-03-29T10:57:31.851286+0000","last_fullsized":"2023-03-29T10:57:31.851286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:23:24.586546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.336910+0000","last_change":"2023-03-29T10:57:31.877812+0000","last_active":"2023-03-29T10:57:32.336910+0000","last_peered":"2023-03-29T10:57:32.336910+0000","last_clean":"2023-03-29T10:57:32.336910+0000","last_became_active":"2023-03-29T10:57:31.877432+0000","last_became_peered":"2023-03-29T10:57:31.877432+0000","last_unstale":"2023-03-29T10:57:32.336910+0000","last_undegraded":"2023-03-29T10:57:32.336910+0000","last_fullsized":"2023-03-29T10:57:32.336910+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:19:12.936732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.863218+0000","last_change":"2023-03-29T10:57:22.320330+0000","last_active":"2023-03-29T10:57:31.863218+0000","last_peered":"2023-03-29T10:57:31.863218+0000","last_clean":"2023-03-29T10:57:31.863218+0000","last_became_active":"2023-03-29T10:57:22.320011+0000","last_became_peered":"2023-03-29T10:57:22.320011+0000","last_unstale":"2023-03-29T10:57:31.863218+0000","last_undegraded":"2023-03-29T10:57:31.863218+0000","last_fullsized":"2023-03-29T10:57:31.863218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:47:01.099069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853293+0000","last_change":"2023-03-29T10:57:27.361265+0000","last_active":"2023-03-29T10:57:31.853293+0000","last_peered":"2023-03-29T10:57:31.853293+0000","last_clean":"2023-03-29T10:57:31.853293+0000","last_became_active":"2023-03-29T10:57:27.360017+0000","last_became_peered":"2023-03-29T10:57:27.360017+0000","last_unstale":"2023-03-29T10:57:31.853293+0000","last_undegraded":"2023-03-29T10:57:31.853293+0000","last_fullsized":"2023-03-29T10:57:31.853293+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:12:14.095287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852801+0000","last_change":"2023-03-29T10:57:27.846016+0000","last_active":"2023-03-29T10:57:31.852801+0000","last_peered":"2023-03-29T10:57:31.852801+0000","last_clean":"2023-03-29T10:57:31.852801+0000","last_became_active":"2023-03-29T10:57:27.845677+0000","last_became_peered":"2023-03-29T10:57:27.845677+0000","last_unstale":"2023-03-29T10:57:31.852801+0000","last_undegraded":"2023-03-29T10:57:31.852801+0000","last_fullsized":"2023-03-29T10:57:31.852801+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T10:58:34.425561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605463+0000","last_change":"2023-03-29T10:57:31.863426+0000","last_active":"2023-03-29T10:57:32.605463+0000","last_peered":"2023-03-29T10:57:32.605463+0000","last_clean":"2023-03-29T10:57:32.605463+0000","last_became_active":"2023-03-29T10:57:31.862992+0000","last_became_peered":"2023-03-29T10:57:31.862992+0000","last_unstale":"2023-03-29T10:57:32.605463+0000","last_undegraded":"2023-03-29T10:57:32.605463+0000","last_fullsized":"2023-03-29T10:57:32.605463+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:24:11.757160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.845511+0000","last_change":"2023-03-29T10:57:22.467148+0000","last_active":"2023-03-29T10:57:31.845511+0000","last_peered":"2023-03-29T10:57:31.845511+0000","last_clean":"2023-03-29T10:57:31.845511+0000","last_became_active":"2023-03-29T10:57:22.466822+0000","last_became_peered":"2023-03-29T10:57:22.466822+0000","last_unstale":"2023-03-29T10:57:31.845511+0000","last_undegraded":"2023-03-29T10:57:31.845511+0000","last_fullsized":"2023-03-29T10:57:31.845511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:53:23.007205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851898+0000","last_change":"2023-03-29T10:57:27.364233+0000","last_active":"2023-03-29T10:57:31.851898+0000","last_peered":"2023-03-29T10:57:31.851898+0000","last_clean":"2023-03-29T10:57:31.851898+0000","last_became_active":"2023-03-29T10:57:27.363992+0000","last_became_peered":"2023-03-29T10:57:27.363992+0000","last_unstale":"2023-03-29T10:57:31.851898+0000","last_undegraded":"2023-03-29T10:57:31.851898+0000","last_fullsized":"2023-03-29T10:57:31.851898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:57:18.786945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851648+0000","last_change":"2023-03-29T10:57:27.859088+0000","last_active":"2023-03-29T10:57:31.851648+0000","last_peered":"2023-03-29T10:57:31.851648+0000","last_clean":"2023-03-29T10:57:31.851648+0000","last_became_active":"2023-03-29T10:57:27.858803+0000","last_became_peered":"2023-03-29T10:57:27.858803+0000","last_unstale":"2023-03-29T10:57:31.851648+0000","last_undegraded":"2023-03-29T10:57:31.851648+0000","last_fullsized":"2023-03-29T10:57:31.851648+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:44:42.208513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.341079+0000","last_change":"2023-03-29T10:57:31.868378+0000","last_active":"2023-03-29T10:57:32.341079+0000","last_peered":"2023-03-29T10:57:32.341079+0000","last_clean":"2023-03-29T10:57:32.341079+0000","last_became_active":"2023-03-29T10:57:31.867988+0000","last_became_peered":"2023-03-29T10:57:31.867988+0000","last_unstale":"2023-03-29T10:57:32.341079+0000","last_undegraded":"2023-03-29T10:57:32.341079+0000","last_fullsized":"2023-03-29T10:57:32.341079+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:18:14.433000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861576+0000","last_change":"2023-03-29T10:57:22.321523+0000","last_active":"2023-03-29T10:57:31.861576+0000","last_peered":"2023-03-29T10:57:31.861576+0000","last_clean":"2023-03-29T10:57:31.861576+0000","last_became_active":"2023-03-29T10:57:22.321183+0000","last_became_peered":"2023-03-29T10:57:22.321183+0000","last_unstale":"2023-03-29T10:57:31.861576+0000","last_undegraded":"2023-03-29T10:57:31.861576+0000","last_fullsized":"2023-03-29T10:57:31.861576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:35:11.664657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846653+0000","last_change":"2023-03-29T10:57:18.176665+0000","last_active":"2023-03-29T10:57:31.846653+0000","last_peered":"2023-03-29T10:57:31.846653+0000","last_clean":"2023-03-29T10:57:31.846653+0000","last_became_active":"2023-03-29T10:57:18.175739+0000","last_became_peered":"2023-03-29T10:57:18.175739+0000","last_unstale":"2023-03-29T10:57:31.846653+0000","last_undegraded":"2023-03-29T10:57:31.846653+0000","last_fullsized":"2023-03-29T10:57:31.846653+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:16.275989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:16.275989+0000","last_clean_scrub_stamp":"2023-03-29T10:57:16.275989+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:06:32.872344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851023+0000","last_change":"2023-03-29T10:57:27.365796+0000","last_active":"2023-03-29T10:57:31.851023+0000","last_peered":"2023-03-29T10:57:31.851023+0000","last_clean":"2023-03-29T10:57:31.851023+0000","last_became_active":"2023-03-29T10:57:27.365435+0000","last_became_peered":"2023-03-29T10:57:27.365435+0000","last_unstale":"2023-03-29T10:57:31.851023+0000","last_undegraded":"2023-03-29T10:57:31.851023+0000","last_fullsized":"2023-03-29T10:57:31.851023+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:12:14.095287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.855069+0000","last_change":"2023-03-29T10:57:27.866559+0000","last_active":"2023-03-29T10:57:31.855069+0000","last_peered":"2023-03-29T10:57:31.855069+0000","last_clean":"2023-03-29T10:57:31.855069+0000","last_became_active":"2023-03-29T10:57:27.866250+0000","last_became_peered":"2023-03-29T10:57:27.866250+0000","last_unstale":"2023-03-29T10:57:31.855069+0000","last_undegraded":"2023-03-29T10:57:31.855069+0000","last_fullsized":"2023-03-29T10:57:31.855069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:53:00.445130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606048+0000","last_change":"2023-03-29T10:57:31.881080+0000","last_active":"2023-03-29T10:57:32.606048+0000","last_peered":"2023-03-29T10:57:32.606048+0000","last_clean":"2023-03-29T10:57:32.606048+0000","last_became_active":"2023-03-29T10:57:31.880427+0000","last_became_peered":"2023-03-29T10:57:31.880427+0000","last_unstale":"2023-03-29T10:57:32.606048+0000","last_undegraded":"2023-03-29T10:57:32.606048+0000","last_fullsized":"2023-03-29T10:57:32.606048+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:36:05.766940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.847505+0000","last_change":"2023-03-29T10:57:22.322607+0000","last_active":"2023-03-29T10:57:31.847505+0000","last_peered":"2023-03-29T10:57:31.847505+0000","last_clean":"2023-03-29T10:57:31.847505+0000","last_became_active":"2023-03-29T10:57:22.321613+0000","last_became_peered":"2023-03-29T10:57:22.321613+0000","last_unstale":"2023-03-29T10:57:31.847505+0000","last_undegraded":"2023-03-29T10:57:31.847505+0000","last_fullsized":"2023-03-29T10:57:31.847505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:35:11.664657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851455+0000","last_change":"2023-03-29T10:57:27.366362+0000","last_active":"2023-03-29T10:57:31.851455+0000","last_peered":"2023-03-29T10:57:31.851455+0000","last_clean":"2023-03-29T10:57:31.851455+0000","last_became_active":"2023-03-29T10:57:27.366048+0000","last_became_peered":"2023-03-29T10:57:27.366048+0000","last_unstale":"2023-03-29T10:57:31.851455+0000","last_undegraded":"2023-03-29T10:57:31.851455+0000","last_fullsized":"2023-03-29T10:57:31.851455+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:01:05.024349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851632+0000","last_change":"2023-03-29T10:57:27.847948+0000","last_active":"2023-03-29T10:57:31.851632+0000","last_peered":"2023-03-29T10:57:31.851632+0000","last_clean":"2023-03-29T10:57:31.851632+0000","last_became_active":"2023-03-29T10:57:27.847691+0000","last_became_peered":"2023-03-29T10:57:27.847691+0000","last_unstale":"2023-03-29T10:57:31.851632+0000","last_undegraded":"2023-03-29T10:57:31.851632+0000","last_fullsized":"2023-03-29T10:57:31.851632+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:01:58.615750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.605974+0000","last_change":"2023-03-29T10:57:31.873406+0000","last_active":"2023-03-29T10:57:32.605974+0000","last_peered":"2023-03-29T10:57:32.605974+0000","last_clean":"2023-03-29T10:57:32.605974+0000","last_became_active":"2023-03-29T10:57:31.873032+0000","last_became_peered":"2023-03-29T10:57:31.873032+0000","last_unstale":"2023-03-29T10:57:32.605974+0000","last_undegraded":"2023-03-29T10:57:32.605974+0000","last_fullsized":"2023-03-29T10:57:32.605974+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:53:41.690520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.846972+0000","last_change":"2023-03-29T10:57:22.315905+0000","last_active":"2023-03-29T10:57:31.846972+0000","last_peered":"2023-03-29T10:57:31.846972+0000","last_clean":"2023-03-29T10:57:31.846972+0000","last_became_active":"2023-03-29T10:57:22.315603+0000","last_became_peered":"2023-03-29T10:57:22.315603+0000","last_unstale":"2023-03-29T10:57:31.846972+0000","last_undegraded":"2023-03-29T10:57:31.846972+0000","last_fullsized":"2023-03-29T10:57:31.846972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T10:59:44.030013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853041+0000","last_change":"2023-03-29T10:57:27.363544+0000","last_active":"2023-03-29T10:57:31.853041+0000","last_peered":"2023-03-29T10:57:31.853041+0000","last_clean":"2023-03-29T10:57:31.853041+0000","last_became_active":"2023-03-29T10:57:27.363284+0000","last_became_peered":"2023-03-29T10:57:27.363284+0000","last_unstale":"2023-03-29T10:57:31.853041+0000","last_undegraded":"2023-03-29T10:57:31.853041+0000","last_fullsized":"2023-03-29T10:57:31.853041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:16.312585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853937+0000","last_change":"2023-03-29T10:57:27.844093+0000","last_active":"2023-03-29T10:57:31.853937+0000","last_peered":"2023-03-29T10:57:31.853937+0000","last_clean":"2023-03-29T10:57:31.853937+0000","last_became_active":"2023-03-29T10:57:27.843647+0000","last_became_peered":"2023-03-29T10:57:27.843647+0000","last_unstale":"2023-03-29T10:57:31.853937+0000","last_undegraded":"2023-03-29T10:57:31.853937+0000","last_fullsized":"2023-03-29T10:57:31.853937+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:46:29.403965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.340198+0000","last_change":"2023-03-29T10:57:31.878229+0000","last_active":"2023-03-29T10:57:32.340198+0000","last_peered":"2023-03-29T10:57:32.340198+0000","last_clean":"2023-03-29T10:57:32.340198+0000","last_became_active":"2023-03-29T10:57:31.877846+0000","last_became_peered":"2023-03-29T10:57:31.877846+0000","last_unstale":"2023-03-29T10:57:32.340198+0000","last_undegraded":"2023-03-29T10:57:32.340198+0000","last_fullsized":"2023-03-29T10:57:32.340198+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:32:15.233141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852732+0000","last_change":"2023-03-29T10:57:27.366984+0000","last_active":"2023-03-29T10:57:31.852732+0000","last_peered":"2023-03-29T10:57:31.852732+0000","last_clean":"2023-03-29T10:57:31.852732+0000","last_became_active":"2023-03-29T10:57:27.366716+0000","last_became_peered":"2023-03-29T10:57:27.366716+0000","last_unstale":"2023-03-29T10:57:31.852732+0000","last_undegraded":"2023-03-29T10:57:31.852732+0000","last_fullsized":"2023-03-29T10:57:31.852732+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:00:38.166723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852301+0000","last_change":"2023-03-29T10:57:22.319554+0000","last_active":"2023-03-29T10:57:31.852301+0000","last_peered":"2023-03-29T10:57:31.852301+0000","last_clean":"2023-03-29T10:57:31.852301+0000","last_became_active":"2023-03-29T10:57:22.319198+0000","last_became_peered":"2023-03-29T10:57:22.319198+0000","last_unstale":"2023-03-29T10:57:31.852301+0000","last_undegraded":"2023-03-29T10:57:31.852301+0000","last_fullsized":"2023-03-29T10:57:31.852301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:47:01.099069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.849473+0000","last_change":"2023-03-29T10:57:27.839233+0000","last_active":"2023-03-29T10:57:31.849473+0000","last_peered":"2023-03-29T10:57:31.849473+0000","last_clean":"2023-03-29T10:57:31.849473+0000","last_became_active":"2023-03-29T10:57:27.838936+0000","last_became_peered":"2023-03-29T10:57:27.838936+0000","last_unstale":"2023-03-29T10:57:31.849473+0000","last_undegraded":"2023-03-29T10:57:31.849473+0000","last_fullsized":"2023-03-29T10:57:31.849473+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:10:11.279944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.607718+0000","last_change":"2023-03-29T10:57:31.876988+0000","last_active":"2023-03-29T10:57:32.607718+0000","last_peered":"2023-03-29T10:57:32.607718+0000","last_clean":"2023-03-29T10:57:32.607718+0000","last_became_active":"2023-03-29T10:57:31.876521+0000","last_became_peered":"2023-03-29T10:57:31.876521+0000","last_unstale":"2023-03-29T10:57:32.607718+0000","last_undegraded":"2023-03-29T10:57:32.607718+0000","last_fullsized":"2023-03-29T10:57:32.607718+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:40:04.971172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.858506+0000","last_change":"2023-03-29T10:57:22.322319+0000","last_active":"2023-03-29T10:57:31.858506+0000","last_peered":"2023-03-29T10:57:31.858506+0000","last_clean":"2023-03-29T10:57:31.858506+0000","last_became_active":"2023-03-29T10:57:22.321943+0000","last_became_peered":"2023-03-29T10:57:22.321943+0000","last_unstale":"2023-03-29T10:57:31.858506+0000","last_undegraded":"2023-03-29T10:57:31.858506+0000","last_fullsized":"2023-03-29T10:57:31.858506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:58:04.969453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"28'1","reported_seq":16,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.336470+0000","last_change":"2023-03-29T10:57:27.364507+0000","last_active":"2023-03-29T10:57:32.336470+0000","last_peered":"2023-03-29T10:57:32.336470+0000","last_clean":"2023-03-29T10:57:32.336470+0000","last_became_active":"2023-03-29T10:57:27.363959+0000","last_became_peered":"2023-03-29T10:57:27.363959+0000","last_unstale":"2023-03-29T10:57:32.336470+0000","last_undegraded":"2023-03-29T10:57:32.336470+0000","last_fullsized":"2023-03-29T10:57:32.336470+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:16.312585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860854+0000","last_change":"2023-03-29T10:57:27.861040+0000","last_active":"2023-03-29T10:57:31.860854+0000","last_peered":"2023-03-29T10:57:31.860854+0000","last_clean":"2023-03-29T10:57:31.860854+0000","last_became_active":"2023-03-29T10:57:27.860750+0000","last_became_peered":"2023-03-29T10:57:27.860750+0000","last_unstale":"2023-03-29T10:57:31.860854+0000","last_undegraded":"2023-03-29T10:57:31.860854+0000","last_fullsized":"2023-03-29T10:57:31.860854+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:40:51.829430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.543747+0000","last_change":"2023-03-29T10:57:31.882689+0000","last_active":"2023-03-29T10:57:32.543747+0000","last_peered":"2023-03-29T10:57:32.543747+0000","last_clean":"2023-03-29T10:57:32.543747+0000","last_became_active":"2023-03-29T10:57:31.882298+0000","last_became_peered":"2023-03-29T10:57:31.882298+0000","last_unstale":"2023-03-29T10:57:32.543747+0000","last_undegraded":"2023-03-29T10:57:32.543747+0000","last_fullsized":"2023-03-29T10:57:32.543747+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:18:14.433000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853851+0000","last_change":"2023-03-29T10:57:27.363293+0000","last_active":"2023-03-29T10:57:31.853851+0000","last_peered":"2023-03-29T10:57:31.853851+0000","last_clean":"2023-03-29T10:57:31.853851+0000","last_became_active":"2023-03-29T10:57:27.362993+0000","last_became_peered":"2023-03-29T10:57:27.362993+0000","last_unstale":"2023-03-29T10:57:31.853851+0000","last_undegraded":"2023-03-29T10:57:31.853851+0000","last_fullsized":"2023-03-29T10:57:31.853851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:57:18.786945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.845743+0000","last_change":"2023-03-29T10:57:27.849850+0000","last_active":"2023-03-29T10:57:31.845743+0000","last_peered":"2023-03-29T10:57:31.845743+0000","last_clean":"2023-03-29T10:57:31.845743+0000","last_became_active":"2023-03-29T10:57:27.849429+0000","last_became_peered":"2023-03-29T10:57:27.849429+0000","last_unstale":"2023-03-29T10:57:31.845743+0000","last_undegraded":"2023-03-29T10:57:31.845743+0000","last_fullsized":"2023-03-29T10:57:31.845743+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:00:39.175701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.341198+0000","last_change":"2023-03-29T10:57:31.868416+0000","last_active":"2023-03-29T10:57:32.341198+0000","last_peered":"2023-03-29T10:57:32.341198+0000","last_clean":"2023-03-29T10:57:32.341198+0000","last_became_active":"2023-03-29T10:57:31.868042+0000","last_became_peered":"2023-03-29T10:57:31.868042+0000","last_unstale":"2023-03-29T10:57:32.341198+0000","last_undegraded":"2023-03-29T10:57:32.341198+0000","last_fullsized":"2023-03-29T10:57:32.341198+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:28:41.336878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.861823+0000","last_change":"2023-03-29T10:57:27.373078+0000","last_active":"2023-03-29T10:57:31.861823+0000","last_peered":"2023-03-29T10:57:31.861823+0000","last_clean":"2023-03-29T10:57:31.861823+0000","last_became_active":"2023-03-29T10:57:27.372747+0000","last_became_peered":"2023-03-29T10:57:27.372747+0000","last_unstale":"2023-03-29T10:57:31.861823+0000","last_undegraded":"2023-03-29T10:57:31.861823+0000","last_fullsized":"2023-03-29T10:57:31.861823+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:23:59.025438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853187+0000","last_change":"2023-03-29T10:57:27.861043+0000","last_active":"2023-03-29T10:57:31.853187+0000","last_peered":"2023-03-29T10:57:31.853187+0000","last_clean":"2023-03-29T10:57:31.853187+0000","last_became_active":"2023-03-29T10:57:27.860522+0000","last_became_peered":"2023-03-29T10:57:27.860522+0000","last_unstale":"2023-03-29T10:57:31.853187+0000","last_undegraded":"2023-03-29T10:57:31.853187+0000","last_fullsized":"2023-03-29T10:57:31.853187+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:49:24.886407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.606089+0000","last_change":"2023-03-29T10:57:31.879767+0000","last_active":"2023-03-29T10:57:32.606089+0000","last_peered":"2023-03-29T10:57:32.606089+0000","last_clean":"2023-03-29T10:57:32.606089+0000","last_became_active":"2023-03-29T10:57:31.879330+0000","last_became_peered":"2023-03-29T10:57:31.879330+0000","last_unstale":"2023-03-29T10:57:32.606089+0000","last_undegraded":"2023-03-29T10:57:32.606089+0000","last_fullsized":"2023-03-29T10:57:32.606089+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:10:08.859801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.864594+0000","last_change":"2023-03-29T10:57:27.370488+0000","last_active":"2023-03-29T10:57:31.864594+0000","last_peered":"2023-03-29T10:57:31.864594+0000","last_clean":"2023-03-29T10:57:31.864594+0000","last_became_active":"2023-03-29T10:57:27.370181+0000","last_became_peered":"2023-03-29T10:57:27.370181+0000","last_unstale":"2023-03-29T10:57:31.864594+0000","last_undegraded":"2023-03-29T10:57:31.864594+0000","last_fullsized":"2023-03-29T10:57:31.864594+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:10:38.094232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852668+0000","last_change":"2023-03-29T10:57:27.860955+0000","last_active":"2023-03-29T10:57:31.852668+0000","last_peered":"2023-03-29T10:57:31.852668+0000","last_clean":"2023-03-29T10:57:31.852668+0000","last_became_active":"2023-03-29T10:57:27.860396+0000","last_became_peered":"2023-03-29T10:57:27.860396+0000","last_unstale":"2023-03-29T10:57:31.852668+0000","last_undegraded":"2023-03-29T10:57:31.852668+0000","last_fullsized":"2023-03-29T10:57:31.852668+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:44:40.017537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.339309+0000","last_change":"2023-03-29T10:57:31.874679+0000","last_active":"2023-03-29T10:57:32.339309+0000","last_peered":"2023-03-29T10:57:32.339309+0000","last_clean":"2023-03-29T10:57:32.339309+0000","last_became_active":"2023-03-29T10:57:31.874124+0000","last_became_peered":"2023-03-29T10:57:31.874124+0000","last_unstale":"2023-03-29T10:57:32.339309+0000","last_undegraded":"2023-03-29T10:57:32.339309+0000","last_fullsized":"2023-03-29T10:57:32.339309+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:36:05.766940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":16,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.853964+0000","last_change":"2023-03-29T10:57:27.366500+0000","last_active":"2023-03-29T10:57:31.853964+0000","last_peered":"2023-03-29T10:57:31.853964+0000","last_clean":"2023-03-29T10:57:31.853964+0000","last_became_active":"2023-03-29T10:57:27.366215+0000","last_became_peered":"2023-03-29T10:57:27.366215+0000","last_unstale":"2023-03-29T10:57:31.853964+0000","last_undegraded":"2023-03-29T10:57:31.853964+0000","last_fullsized":"2023-03-29T10:57:31.853964+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:45:12.326563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852786+0000","last_change":"2023-03-29T10:57:27.838577+0000","last_active":"2023-03-29T10:57:31.852786+0000","last_peered":"2023-03-29T10:57:31.852786+0000","last_clean":"2023-03-29T10:57:31.852786+0000","last_became_active":"2023-03-29T10:57:27.838301+0000","last_became_peered":"2023-03-29T10:57:27.838301+0000","last_unstale":"2023-03-29T10:57:31.852786+0000","last_undegraded":"2023-03-29T10:57:31.852786+0000","last_fullsized":"2023-03-29T10:57:31.852786+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:36:09.200571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-03-29T10:57:32.543500+0000","last_change":"2023-03-29T10:57:31.874474+0000","last_active":"2023-03-29T10:57:32.543500+0000","last_peered":"2023-03-29T10:57:32.543500+0000","last_clean":"2023-03-29T10:57:32.543500+0000","last_became_active":"2023-03-29T10:57:31.874137+0000","last_became_peered":"2023-03-29T10:57:31.874137+0000","last_unstale":"2023-03-29T10:57:32.543500+0000","last_undegraded":"2023-03-29T10:57:32.543500+0000","last_fullsized":"2023-03-29T10:57:32.543500+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:30.835303+0000","last_clean_scrub_stamp":"2023-03-29T10:57:30.835303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:22:10.382014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.857710+0000","last_change":"2023-03-29T10:57:27.365343+0000","last_active":"2023-03-29T10:57:31.857710+0000","last_peered":"2023-03-29T10:57:31.857710+0000","last_clean":"2023-03-29T10:57:31.857710+0000","last_became_active":"2023-03-29T10:57:27.365066+0000","last_became_peered":"2023-03-29T10:57:27.365066+0000","last_unstale":"2023-03-29T10:57:31.857710+0000","last_undegraded":"2023-03-29T10:57:31.857710+0000","last_fullsized":"2023-03-29T10:57:31.857710+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:26.322720+0000","last_clean_scrub_stamp":"2023-03-29T10:57:26.322720+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:40:18.602235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.860006+0000","last_change":"2023-03-29T10:57:27.860635+0000","last_active":"2023-03-29T10:57:31.860006+0000","last_peered":"2023-03-29T10:57:31.860006+0000","last_clean":"2023-03-29T10:57:31.860006+0000","last_became_active":"2023-03-29T10:57:27.860287+0000","last_became_peered":"2023-03-29T10:57:27.860287+0000","last_unstale":"2023-03-29T10:57:31.860006+0000","last_undegraded":"2023-03-29T10:57:31.860006+0000","last_fullsized":"2023-03-29T10:57:31.860006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:57:16.888544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852270+0000","last_change":"2023-03-29T10:57:27.858882+0000","last_active":"2023-03-29T10:57:31.852270+0000","last_peered":"2023-03-29T10:57:31.852270+0000","last_clean":"2023-03-29T10:57:31.852270+0000","last_became_active":"2023-03-29T10:57:27.858471+0000","last_became_peered":"2023-03-29T10:57:27.858471+0000","last_unstale":"2023-03-29T10:57:31.852270+0000","last_undegraded":"2023-03-29T10:57:31.852270+0000","last_fullsized":"2023-03-29T10:57:31.852270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:10:39.103210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852924+0000","last_change":"2023-03-29T10:57:27.848104+0000","last_active":"2023-03-29T10:57:31.852924+0000","last_peered":"2023-03-29T10:57:31.852924+0000","last_clean":"2023-03-29T10:57:31.852924+0000","last_became_active":"2023-03-29T10:57:27.847812+0000","last_became_peered":"2023-03-29T10:57:27.847812+0000","last_unstale":"2023-03-29T10:57:31.852924+0000","last_undegraded":"2023-03-29T10:57:31.852924+0000","last_fullsized":"2023-03-29T10:57:31.852924+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:50:00.290051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852955+0000","last_change":"2023-03-29T10:57:27.868878+0000","last_active":"2023-03-29T10:57:31.852955+0000","last_peered":"2023-03-29T10:57:31.852955+0000","last_clean":"2023-03-29T10:57:31.852955+0000","last_became_active":"2023-03-29T10:57:27.868603+0000","last_became_peered":"2023-03-29T10:57:27.868603+0000","last_unstale":"2023-03-29T10:57:31.852955+0000","last_undegraded":"2023-03-29T10:57:31.852955+0000","last_fullsized":"2023-03-29T10:57:31.852955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:53:38.186916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.851718+0000","last_change":"2023-03-29T10:57:27.853683+0000","last_active":"2023-03-29T10:57:31.851718+0000","last_peered":"2023-03-29T10:57:31.851718+0000","last_clean":"2023-03-29T10:57:31.851718+0000","last_became_active":"2023-03-29T10:57:27.853396+0000","last_became_peered":"2023-03-29T10:57:27.853396+0000","last_unstale":"2023-03-29T10:57:31.851718+0000","last_undegraded":"2023-03-29T10:57:31.851718+0000","last_fullsized":"2023-03-29T10:57:31.851718+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:57:16.888544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2023-03-29T10:57:31.852962+0000","last_change":"2023-03-29T10:57:27.856369+0000","last_active":"2023-03-29T10:57:31.852962+0000","last_peered":"2023-03-29T10:57:31.852962+0000","last_clean":"2023-03-29T10:57:31.852962+0000","last_became_active":"2023-03-29T10:57:27.856118+0000","last_became_peered":"2023-03-29T10:57:27.856118+0000","last_unstale":"2023-03-29T10:57:31.852962+0000","last_undegraded":"2023-03-29T10:57:31.852962+0000","last_fullsized":"2023-03-29T10:57:31.852962+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:27.331698+0000","last_clean_scrub_stamp":"2023-03-29T10:57:27.331698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:33:37.107795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4598,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9296,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362544,"statfs":{"total":96636764160,"available":96627245056,"internally_reserved":0,"allocated":925696,"data_stored":718956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8112,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363728,"statfs":{"total":96636764160,"available":96628457472,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]}]},{"osd":1,"up_from":13,"seq":55834574860,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8176,"kb_used_data":296,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363664,"statfs":{"total":96636764160,"available":96628391936,"internally_reserved":0,"allocated":303104,"data_stored":124572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9268,"kb_used_data":876,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362572,"statfs":{"total":96636764160,"available":96627273728,"internally_reserved":0,"allocated":897024,"data_stored":714903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.317}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8108,"kb_used_data":292,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363732,"statfs":{"total":96636764160,"available":96628461568,"internally_reserved":0,"allocated":299008,"data_stored":123953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8180,"kb_used_data":300,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363660,"statfs":{"total":96636764160,"available":96628387840,"internally_reserved":0,"allocated":307200,"data_stored":124662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8260,"kb_used_data":316,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363580,"statfs":{"total":96636764160,"available":96628305920,"internally_reserved":0,"allocated":323584,"data_stored":128428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]}]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8184,"kb_used_data":304,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363656,"statfs":{"total":96636764160,"available":96628383744,"internally_reserved":0,"allocated":311296,"data_stored":125224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-03-29T10:58:00.627 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-03-29T10:58:00.627 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-03-29T10:58:00.627 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-03-29T10:58:00.627 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-03-29T10:58:00.829 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.829+0000 7f4af53ff700 1 Processor -- start 2023-03-29T10:58:00.829 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.829+0000 7f4af53ff700 1 -- start start 2023-03-29T10:58:00.830 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.830+0000 7f4af53ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4af00d3a30 0x7f4af00b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:00.830 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.830+0000 7f4af53ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af00cdc80 con 0x7f4af015c950 2023-03-29T10:58:00.830 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.831+0000 7f4af53ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af00cd6c0 con 0x7f4af00cf950 2023-03-29T10:58:00.830 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.831+0000 7f4af53ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af00d3e00 con 0x7f4af00d3a30 2023-03-29T10:58:00.830 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.831+0000 7f4ae7fff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f4af015c950 0x7f4af00cf0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36960/0 (socket says 172.21.15.111:36960) 2023-03-29T10:58:00.830 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.831+0000 7f4ae7fff700 1 -- 172.21.15.111:0/2230039069 learned_addr learned my addr 172.21.15.111:0/2230039069 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:00.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.831+0000 7f4aef7fe700 1 --2- 172.21.15.111:0/2230039069 >> v2:172.21.15.119:3300/0 conn(0x7f4af00d3a30 0x7f4af00b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:00.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.831+0000 7f4aee7fc700 1 -- 172.21.15.111:0/2230039069 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 47128160 0 0) 0x7f4af00cdc80 con 0x7f4af015c950 2023-03-29T10:58:00.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.831+0000 7f4aee7fc700 1 -- 172.21.15.111:0/2230039069 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4adc0035b0 con 0x7f4af015c950 2023-03-29T10:58:00.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.831+0000 7f4aee7fc700 1 -- 172.21.15.111:0/2230039069 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 831340957 0 0) 0x7f4af00cd6c0 con 0x7f4af00cf950 2023-03-29T10:58:00.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.831+0000 7f4aee7fc700 1 -- 172.21.15.111:0/2230039069 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4af00cdc80 con 0x7f4af00cf950 2023-03-29T10:58:00.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.831+0000 7f4aef7fe700 1 -- 172.21.15.111:0/2230039069 >> v1:172.21.15.119:6789/0 conn(0x7f4af00cf950 legacy=0x7f4af00cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:00.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.831+0000 7f4aef7fe700 1 -- 172.21.15.111:0/2230039069 >> v1:172.21.15.111:6789/0 conn(0x7f4af015c950 legacy=0x7f4af00cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:00.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.831+0000 7f4aef7fe700 1 -- 172.21.15.111:0/2230039069 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af00cd9a0 con 0x7f4af00d3a30 2023-03-29T10:58:00.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.832+0000 7f4aef7fe700 1 --2- 172.21.15.111:0/2230039069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4af00d3a30 0x7f4af00b5d90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009d90 tx=0x7f4ae800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf65166fadb6eff4 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:00.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.832+0000 7f4aee7fc700 1 -- 172.21.15.111:0/2230039069 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae800e070 con 0x7f4af00d3a30 2023-03-29T10:58:00.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.832+0000 7f4aee7fc700 1 -- 172.21.15.111:0/2230039069 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae800bb20 con 0x7f4af00d3a30 2023-03-29T10:58:00.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.832+0000 7f4aee7fc700 1 -- 172.21.15.111:0/2230039069 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8012430 con 0x7f4af00d3a30 2023-03-29T10:58:00.833 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.832+0000 7f4af53ff700 1 -- 172.21.15.111:0/2230039069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4af00d3a30 msgr2=0x7f4af00b5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:00.833 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.832+0000 7f4af53ff700 1 --2- 172.21.15.111:0/2230039069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4af00d3a30 0x7f4af00b5d90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009d90 tx=0x7f4ae800b330 comp rx=0 tx=0).stop 2023-03-29T10:58:00.833 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.833+0000 7f4af53ff700 1 -- 172.21.15.111:0/2230039069 shutdown_connections 2023-03-29T10:58:00.833 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.833+0000 7f4af53ff700 1 --2- 172.21.15.111:0/2230039069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4af00d3a30 0x7f4af00b5d90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:00.833 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.833+0000 7f4af53ff700 1 -- 172.21.15.111:0/2230039069 >> 172.21.15.111:0/2230039069 conn(0x7f4af0071b10 msgr2=0x7f4af0071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:00.833 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.833+0000 7f4af53ff700 1 -- 172.21.15.111:0/2230039069 shutdown_connections 2023-03-29T10:58:00.833 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.833+0000 7f4af53ff700 1 -- 172.21.15.111:0/2230039069 wait complete. 2023-03-29T10:58:00.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.834+0000 7f4af53ff700 1 Processor -- start 2023-03-29T10:58:00.834 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.834+0000 7f4af53ff700 1 -- start start 2023-03-29T10:58:00.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.835+0000 7f4af53ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4af00cf950 0x7f4af01547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:00.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.835+0000 7f4af53ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4af00d3a30 0x7f4af00c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:00.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.835+0000 7f4af53ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4af015c950 0x7f4af00b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:00.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.835+0000 7f4af53ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af015e6b0 con 0x7f4af00d3a30 2023-03-29T10:58:00.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.835+0000 7f4af53ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af015e990 con 0x7f4af015c950 2023-03-29T10:58:00.835 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.835+0000 7f4af53ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af015ec70 con 0x7f4af00cf950 2023-03-29T10:58:00.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.835+0000 7f4ae7fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4af00d3a30 0x7f4af00c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:00.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.836+0000 7f4ae7fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4af00d3a30 0x7f4af00c71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50380/0 (socket says 172.21.15.111:50380) 2023-03-29T10:58:00.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.836+0000 7f4ae7fff700 1 -- 172.21.15.111:0/3662801805 learned_addr learned my addr 172.21.15.111:0/3662801805 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:00.836 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.836+0000 7f4aeeffd700 1 --2- 172.21.15.111:0/3662801805 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4af00cf950 0x7f4af01547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:00.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.836+0000 7f4aef7fe700 1 --2- 172.21.15.111:0/3662801805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4af015c950 0x7f4af00b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:00.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.836+0000 7f4ae7fff700 1 -- 172.21.15.111:0/3662801805 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4af00cf950 msgr2=0x7f4af01547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:00.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.836+0000 7f4ae7fff700 1 --2- 172.21.15.111:0/3662801805 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4af00cf950 0x7f4af01547a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:00.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.836+0000 7f4ae7fff700 1 -- 172.21.15.111:0/3662801805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4af015c950 msgr2=0x7f4af00b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:00.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.836+0000 7f4ae7fff700 1 --2- 172.21.15.111:0/3662801805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4af015c950 0x7f4af00b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:00.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.836+0000 7f4ae7fff700 1 -- 172.21.15.111:0/3662801805 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae8009a70 con 0x7f4af00d3a30 2023-03-29T10:58:00.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.836+0000 7f4ae7fff700 1 --2- 172.21.15.111:0/3662801805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4af00d3a30 0x7f4af00c71d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800c700 tx=0x7f4ad800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:00.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.836+0000 7f4ae77fe700 1 -- 172.21.15.111:0/3662801805 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8005d70 con 0x7f4af00d3a30 2023-03-29T10:58:00.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.836+0000 7f4ae77fe700 1 -- 172.21.15.111:0/3662801805 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad80070d0 con 0x7f4af00d3a30 2023-03-29T10:58:00.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.836+0000 7f4af53ff700 1 -- 172.21.15.111:0/3662801805 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af00d19a0 con 0x7f4af00d3a30 2023-03-29T10:58:00.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.837+0000 7f4ae77fe700 1 -- 172.21.15.111:0/3662801805 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8007440 con 0x7f4af00d3a30 2023-03-29T10:58:00.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.837+0000 7f4af53ff700 1 -- 172.21.15.111:0/3662801805 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af00d16c0 con 0x7f4af00d3a30 2023-03-29T10:58:00.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.838+0000 7f4af53ff700 1 -- 172.21.15.111:0/3662801805 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af00b62d0 con 0x7f4af00d3a30 2023-03-29T10:58:00.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.838+0000 7f4ae77fe700 1 -- 172.21.15.111:0/3662801805 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f4ad80075a0 con 0x7f4af00d3a30 2023-03-29T10:58:00.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.839+0000 7f4ae77fe700 1 --2- 172.21.15.111:0/3662801805 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4ad0069de0 0x7f4ad006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:00.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.839+0000 7f4ae77fe700 1 -- 172.21.15.111:0/3662801805 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4ad80875a0 con 0x7f4af00d3a30 2023-03-29T10:58:00.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.839+0000 7f4aeeffd700 1 --2- 172.21.15.111:0/3662801805 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4ad0069de0 0x7f4ad006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:00.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.840+0000 7f4aeeffd700 1 --2- 172.21.15.111:0/3662801805 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4ad0069de0 0x7f4ad006c290 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00011e0 tx=0x7f4ae0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:00.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:00.843+0000 7f4ae77fe700 1 -- 172.21.15.111:0/3662801805 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f4ad804c160 con 0x7f4af00d3a30 2023-03-29T10:58:01.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.078+0000 7f4af53ff700 1 -- 172.21.15.111:0/3662801805 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4af015f160 con 0x7f4af00d3a30 2023-03-29T10:58:01.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.078+0000 7f4ae77fe700 1 -- 172.21.15.111:0/3662801805 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f4ad8017080 con 0x7f4af00d3a30 2023-03-29T10:58:01.078 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:01.078 INFO:teuthology.orchestra.run.smithi111.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-03-29T10:58:01.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.080+0000 7f4af53ff700 1 -- 172.21.15.111:0/3662801805 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4ad0069de0 msgr2=0x7f4ad006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:01.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.080+0000 7f4af53ff700 1 --2- 172.21.15.111:0/3662801805 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4ad0069de0 0x7f4ad006c290 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00011e0 tx=0x7f4ae0009040 comp rx=0 tx=0).stop 2023-03-29T10:58:01.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.080+0000 7f4af53ff700 1 -- 172.21.15.111:0/3662801805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4af00d3a30 msgr2=0x7f4af00c71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:01.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.080+0000 7f4af53ff700 1 --2- 172.21.15.111:0/3662801805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4af00d3a30 0x7f4af00c71d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800c700 tx=0x7f4ad800ca10 comp rx=0 tx=0).stop 2023-03-29T10:58:01.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.080+0000 7f4af53ff700 1 -- 172.21.15.111:0/3662801805 shutdown_connections 2023-03-29T10:58:01.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.080+0000 7f4af53ff700 1 --2- 172.21.15.111:0/3662801805 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4ad0069de0 0x7f4ad006c290 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.080+0000 7f4af53ff700 1 --2- 172.21.15.111:0/3662801805 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4af015c950 0x7f4af00b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.080+0000 7f4af53ff700 1 --2- 172.21.15.111:0/3662801805 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4af00cf950 0x7f4af01547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.080+0000 7f4af53ff700 1 --2- 172.21.15.111:0/3662801805 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4af00d3a30 0x7f4af00c71d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.080+0000 7f4af53ff700 1 -- 172.21.15.111:0/3662801805 >> 172.21.15.111:0/3662801805 conn(0x7f4af0071b10 msgr2=0x7f4af00d1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:01.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.081+0000 7f4af53ff700 1 -- 172.21.15.111:0/3662801805 shutdown_connections 2023-03-29T10:58:01.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.081+0000 7f4af53ff700 1 -- 172.21.15.111:0/3662801805 wait complete. 2023-03-29T10:58:01.097 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-03-29T10:58:01.098 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:58:01.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.281+0000 7f10a95ff700 1 Processor -- start 2023-03-29T10:58:01.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.281+0000 7f10a95ff700 1 -- start start 2023-03-29T10:58:01.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.282+0000 7f10a95ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f10a40b2440 0x7f10a40cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:01.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.282+0000 7f10a95ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f10a40d2ea0 0x7f10a40d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:01.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.282+0000 7f10a95ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a40d3860 con 0x7f10a415db70 2023-03-29T10:58:01.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.282+0000 7f10a95ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a40b2810 con 0x7f10a40b2440 2023-03-29T10:58:01.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.282+0000 7f10a95ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a415ce30 con 0x7f10a40d2ea0 2023-03-29T10:58:01.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.282+0000 7f10a37fe700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f10a40d2ea0 0x7f10a40d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:01.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.282+0000 7f10a2ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f10a40b2440 0x7f10a40cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:01.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.282+0000 7f10a37fe700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f10a40d2ea0 0x7f10a40d3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:38810/0 (socket says 172.21.15.111:38810) 2023-03-29T10:58:01.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.282+0000 7f10a37fe700 1 -- 172.21.15.111:0/147698790 learned_addr learned my addr 172.21.15.111:0/147698790 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:01.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.282+0000 7f10a27fc700 1 --1- 172.21.15.111:0/147698790 >> v1:172.21.15.119:6789/0 conn(0x7f10a415db70 0x7f10a4160e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.111:58880/0 (socket says 172.21.15.111:58880) 2023-03-29T10:58:01.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.283+0000 7f10a37fe700 1 -- 172.21.15.111:0/147698790 >> v2:172.21.15.119:3300/0 conn(0x7f10a40b2440 msgr2=0x7f10a40cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:01.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.283+0000 7f10a37fe700 1 --2- 172.21.15.111:0/147698790 >> v2:172.21.15.119:3300/0 conn(0x7f10a40b2440 0x7f10a40cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.283+0000 7f10a37fe700 1 -- 172.21.15.111:0/147698790 >> v1:172.21.15.119:6789/0 conn(0x7f10a415db70 legacy=0x7f10a4160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:01.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.283+0000 7f10a37fe700 1 -- 172.21.15.111:0/147698790 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a40cdee0 con 0x7f10a40d2ea0 2023-03-29T10:58:01.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.283+0000 7f10a37fe700 1 --2- 172.21.15.111:0/147698790 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f10a40d2ea0 0x7f10a40d3320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f109c009e50 tx=0x7f109c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecade1116472c878 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:01.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.283+0000 7f10a1ffb700 1 -- 172.21.15.111:0/147698790 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c00e070 con 0x7f10a40d2ea0 2023-03-29T10:58:01.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.283+0000 7f10a1ffb700 1 -- 172.21.15.111:0/147698790 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109c00b910 con 0x7f10a40d2ea0 2023-03-29T10:58:01.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.283+0000 7f10a1ffb700 1 -- 172.21.15.111:0/147698790 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c00bcc0 con 0x7f10a40d2ea0 2023-03-29T10:58:01.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.284+0000 7f10a95ff700 1 -- 172.21.15.111:0/147698790 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f10a40d2ea0 msgr2=0x7f10a40d3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:01.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.284+0000 7f10a95ff700 1 --2- 172.21.15.111:0/147698790 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f10a40d2ea0 0x7f10a40d3320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f109c009e50 tx=0x7f109c00b330 comp rx=0 tx=0).stop 2023-03-29T10:58:01.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.284+0000 7f10a95ff700 1 -- 172.21.15.111:0/147698790 shutdown_connections 2023-03-29T10:58:01.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.284+0000 7f10a95ff700 1 --2- 172.21.15.111:0/147698790 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f10a40d2ea0 0x7f10a40d3320 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.284+0000 7f10a95ff700 1 --2- 172.21.15.111:0/147698790 >> v2:172.21.15.119:3300/0 conn(0x7f10a40b2440 0x7f10a40cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.284+0000 7f10a95ff700 1 -- 172.21.15.111:0/147698790 >> 172.21.15.111:0/147698790 conn(0x7f10a4071b10 msgr2=0x7f10a4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:01.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.284+0000 7f10a95ff700 1 -- 172.21.15.111:0/147698790 shutdown_connections 2023-03-29T10:58:01.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.284+0000 7f10a95ff700 1 -- 172.21.15.111:0/147698790 wait complete. 2023-03-29T10:58:01.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.285+0000 7f10a95ff700 1 Processor -- start 2023-03-29T10:58:01.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.286+0000 7f10a95ff700 1 -- start start 2023-03-29T10:58:01.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.286+0000 7f10a95ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10a40b2440 0x7f10a414fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:01.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.286+0000 7f10a95ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f10a40d2ea0 0x7f10a41e6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:01.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.286+0000 7f10a95ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10a415db70 0x7f10a415a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:01.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.287+0000 7f10a95ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40b6510 con 0x7f10a40b2440 2023-03-29T10:58:01.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.287+0000 7f10a95ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a4150200 con 0x7f10a415db70 2023-03-29T10:58:01.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.287+0000 7f10a95ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a4156e00 con 0x7f10a40d2ea0 2023-03-29T10:58:01.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.287+0000 7f10a2ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10a40b2440 0x7f10a414fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:01.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.287+0000 7f10a2ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10a40b2440 0x7f10a414fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50386/0 (socket says 172.21.15.111:50386) 2023-03-29T10:58:01.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.287+0000 7f10a2ffd700 1 -- 172.21.15.111:0/2731888293 learned_addr learned my addr 172.21.15.111:0/2731888293 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:01.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.287+0000 7f10a27fc700 1 --2- 172.21.15.111:0/2731888293 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f10a40d2ea0 0x7f10a41e6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:01.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.287+0000 7f10a37fe700 1 --2- 172.21.15.111:0/2731888293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10a415db70 0x7f10a415a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:01.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.287+0000 7f10a2ffd700 1 -- 172.21.15.111:0/2731888293 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f10a40d2ea0 msgr2=0x7f10a41e6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:01.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.287+0000 7f10a2ffd700 1 --2- 172.21.15.111:0/2731888293 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f10a40d2ea0 0x7f10a41e6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.287+0000 7f10a2ffd700 1 -- 172.21.15.111:0/2731888293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10a415db70 msgr2=0x7f10a415a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:01.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.287+0000 7f10a2ffd700 1 --2- 172.21.15.111:0/2731888293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10a415db70 0x7f10a415a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.287+0000 7f10a2ffd700 1 -- 172.21.15.111:0/2731888293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c009b30 con 0x7f10a40b2440 2023-03-29T10:58:01.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.287+0000 7f10a37fe700 1 --2- 172.21.15.111:0/2731888293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10a415db70 0x7f10a415a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:01.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.287+0000 7f10a2ffd700 1 --2- 172.21.15.111:0/2731888293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10a40b2440 0x7f10a414fcc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f109400e9b0 tx=0x7f109400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:01.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.288+0000 7f108b7fe700 1 -- 172.21.15.111:0/2731888293 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1094009710 con 0x7f10a40b2440 2023-03-29T10:58:01.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.288+0000 7f108b7fe700 1 -- 172.21.15.111:0/2731888293 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109400c850 con 0x7f10a40b2440 2023-03-29T10:58:01.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.288+0000 7f108b7fe700 1 -- 172.21.15.111:0/2731888293 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109400cbc0 con 0x7f10a40b2440 2023-03-29T10:58:01.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.288+0000 7f10a95ff700 1 -- 172.21.15.111:0/2731888293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a415d090 con 0x7f10a40b2440 2023-03-29T10:58:01.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.288+0000 7f10a95ff700 1 -- 172.21.15.111:0/2731888293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a40cdc00 con 0x7f10a40b2440 2023-03-29T10:58:01.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.289+0000 7f10a95ff700 1 -- 172.21.15.111:0/2731888293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a415d370 con 0x7f10a40b2440 2023-03-29T10:58:01.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.290+0000 7f108b7fe700 1 -- 172.21.15.111:0/2731888293 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f109400cd20 con 0x7f10a40b2440 2023-03-29T10:58:01.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.291+0000 7f108b7fe700 1 --2- 172.21.15.111:0/2731888293 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1090069de0 0x7f109006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:01.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.291+0000 7f108b7fe700 1 -- 172.21.15.111:0/2731888293 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1094088360 con 0x7f10a40b2440 2023-03-29T10:58:01.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.291+0000 7f10a27fc700 1 --2- 172.21.15.111:0/2731888293 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1090069de0 0x7f109006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:01.291 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.291+0000 7f10a27fc700 1 --2- 172.21.15.111:0/2731888293 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1090069de0 0x7f109006c290 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f10a415a340 tx=0x7f1098007ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:01.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.295+0000 7f108b7fe700 1 -- 172.21.15.111:0/2731888293 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f1094053910 con 0x7f10a40b2440 2023-03-29T10:58:01.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.492+0000 7f10a95ff700 1 -- 172.21.15.111:0/2731888293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10a4074430 con 0x7f10a40b2440 2023-03-29T10:58:01.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.493+0000 7f108b7fe700 1 -- 172.21.15.111:0/2731888293 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+19796 (secure 0 0 0) 0x7f1094056f30 con 0x7f10a40b2440 2023-03-29T10:58:01.493 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:01.494 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":28,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:57:32.331880+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:58:01.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.496+0000 7f10a95ff700 1 -- 172.21.15.111:0/2731888293 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1090069de0 msgr2=0x7f109006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:01.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.496+0000 7f10a95ff700 1 --2- 172.21.15.111:0/2731888293 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1090069de0 0x7f109006c290 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f10a415a340 tx=0x7f1098007ed0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.496+0000 7f10a95ff700 1 -- 172.21.15.111:0/2731888293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10a40b2440 msgr2=0x7f10a414fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:01.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.496+0000 7f10a95ff700 1 --2- 172.21.15.111:0/2731888293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10a40b2440 0x7f10a414fcc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f109400e9b0 tx=0x7f109400ecc0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.496+0000 7f10a95ff700 1 -- 172.21.15.111:0/2731888293 shutdown_connections 2023-03-29T10:58:01.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.496+0000 7f10a95ff700 1 --2- 172.21.15.111:0/2731888293 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1090069de0 0x7f109006c290 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.496+0000 7f10a95ff700 1 --2- 172.21.15.111:0/2731888293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f10a40b2440 0x7f10a414fcc0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.496+0000 7f10a95ff700 1 --2- 172.21.15.111:0/2731888293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10a415db70 0x7f10a415a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.496+0000 7f10a95ff700 1 --2- 172.21.15.111:0/2731888293 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f10a40d2ea0 0x7f10a41e6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.496+0000 7f10a95ff700 1 -- 172.21.15.111:0/2731888293 >> 172.21.15.111:0/2731888293 conn(0x7f10a4071b10 msgr2=0x7f10a4158690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:01.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.496+0000 7f10a95ff700 1 -- 172.21.15.111:0/2731888293 shutdown_connections 2023-03-29T10:58:01.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.496+0000 7f10a95ff700 1 -- 172.21.15.111:0/2731888293 wait complete. 2023-03-29T10:58:01.513 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-29T10:57:16.183979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-29T10:57:21.239809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-29T10:57:26.192124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-29T10:57:26.796777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-29T10:57:29.874795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-29T10:58:01.513 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-29T10:58:01.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.699+0000 7f82bd5ff700 1 Processor -- start 2023-03-29T10:58:01.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.699+0000 7f82bd5ff700 1 -- start start 2023-03-29T10:58:01.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.700+0000 7f82bd5ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f82b80b87a0 0x7f82b80cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:01.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.700+0000 7f82bd5ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b80cdc60 con 0x7f82b8160d40 2023-03-29T10:58:01.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.700+0000 7f82bd5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b80cd6a0 con 0x7f82b81601e0 2023-03-29T10:58:01.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.700+0000 7f82bd5ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b80d02d0 con 0x7f82b80b87a0 2023-03-29T10:58:01.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.701+0000 7f82b6ffd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f82b80b87a0 0x7f82b80cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:01.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.701+0000 7f82b77fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f82b8160d40 0x7f82b80d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36968/0 (socket says 172.21.15.111:36968) 2023-03-29T10:58:01.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.701+0000 7f82b77fe700 1 -- 172.21.15.111:0/768965907 learned_addr learned my addr 172.21.15.111:0/768965907 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:01.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.701+0000 7f82b6ffd700 1 -- 172.21.15.111:0/768965907 >> v1:172.21.15.119:6789/0 conn(0x7f82b81601e0 legacy=0x7f82b8160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:01.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.701+0000 7f82b6ffd700 1 -- 172.21.15.111:0/768965907 >> v1:172.21.15.111:6789/0 conn(0x7f82b8160d40 legacy=0x7f82b80d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:01.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.701+0000 7f82b6ffd700 1 -- 172.21.15.111:0/768965907 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b80cd980 con 0x7f82b80b87a0 2023-03-29T10:58:01.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.701+0000 7f82b67fc700 1 -- 172.21.15.111:0/768965907 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 783371871 0 0) 0x7f82b80cdc60 con 0x7f82b8160d40 2023-03-29T10:58:01.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.701+0000 7f82b6ffd700 1 --2- 172.21.15.111:0/768965907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82b80b87a0 0x7f82b80cfbf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009840 tx=0x7f82a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1dd6b9638548e server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:01.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.702+0000 7f82b67fc700 1 -- 172.21.15.111:0/768965907 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8011830 con 0x7f82b80b87a0 2023-03-29T10:58:01.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.702+0000 7f82b67fc700 1 -- 172.21.15.111:0/768965907 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a8011990 con 0x7f82b80b87a0 2023-03-29T10:58:01.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.702+0000 7f82b67fc700 1 -- 172.21.15.111:0/768965907 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a8011d20 con 0x7f82b80b87a0 2023-03-29T10:58:01.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.702+0000 7f82bd5ff700 1 -- 172.21.15.111:0/768965907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82b80b87a0 msgr2=0x7f82b80cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:01.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.702+0000 7f82bd5ff700 1 --2- 172.21.15.111:0/768965907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82b80b87a0 0x7f82b80cfbf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f82a8009840 tx=0x7f82a8009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:01.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.702+0000 7f82bd5ff700 1 -- 172.21.15.111:0/768965907 shutdown_connections 2023-03-29T10:58:01.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.702+0000 7f82bd5ff700 1 --2- 172.21.15.111:0/768965907 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82b80b87a0 0x7f82b80cfbf0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.702+0000 7f82bd5ff700 1 -- 172.21.15.111:0/768965907 >> 172.21.15.111:0/768965907 conn(0x7f82b8071b10 msgr2=0x7f82b8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:01.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.702+0000 7f82bd5ff700 1 -- 172.21.15.111:0/768965907 shutdown_connections 2023-03-29T10:58:01.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.702+0000 7f82bd5ff700 1 -- 172.21.15.111:0/768965907 wait complete. 2023-03-29T10:58:01.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.704+0000 7f82bd5ff700 1 Processor -- start 2023-03-29T10:58:01.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.704+0000 7f82bd5ff700 1 -- start start 2023-03-29T10:58:01.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.705+0000 7f82bd5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f82b80b87a0 0x7f82b815a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:01.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.705+0000 7f82bd5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b81601e0 0x7f82b80ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:01.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.705+0000 7f82bd5ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82b8160d40 0x7f82b815d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:01.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.705+0000 7f82bd5ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b815aa00 con 0x7f82b8160d40 2023-03-29T10:58:01.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.705+0000 7f82bd5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b81f6590 con 0x7f82b81601e0 2023-03-29T10:58:01.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.705+0000 7f82bd5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b80cb170 con 0x7f82b80b87a0 2023-03-29T10:58:01.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.705+0000 7f82b77fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82b8160d40 0x7f82b815d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:01.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.705+0000 7f82b77fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82b8160d40 0x7f82b815d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50406/0 (socket says 172.21.15.111:50406) 2023-03-29T10:58:01.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.705+0000 7f82b77fe700 1 -- 172.21.15.111:0/772402055 learned_addr learned my addr 172.21.15.111:0/772402055 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:01.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.705+0000 7f82b6ffd700 1 --2- 172.21.15.111:0/772402055 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f82b80b87a0 0x7f82b815a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:01.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.705+0000 7f82ae7fc700 1 --2- 172.21.15.111:0/772402055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b81601e0 0x7f82b80ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:01.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.706+0000 7f82b77fe700 1 -- 172.21.15.111:0/772402055 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f82b80b87a0 msgr2=0x7f82b815a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:01.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.706+0000 7f82b77fe700 1 --2- 172.21.15.111:0/772402055 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f82b80b87a0 0x7f82b815a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.706+0000 7f82b77fe700 1 -- 172.21.15.111:0/772402055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b81601e0 msgr2=0x7f82b80ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:01.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.706+0000 7f82b77fe700 1 --2- 172.21.15.111:0/772402055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b81601e0 0x7f82b80ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.706+0000 7f82b77fe700 1 -- 172.21.15.111:0/772402055 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a80094a0 con 0x7f82b8160d40 2023-03-29T10:58:01.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.706+0000 7f82ae7fc700 1 --2- 172.21.15.111:0/772402055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b81601e0 0x7f82b80ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:01.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.706+0000 7f82b77fe700 1 --2- 172.21.15.111:0/772402055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82b8160d40 0x7f82b815d830 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f82b000cbd0 tx=0x7f82b000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:01.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.706+0000 7f82affff700 1 -- 172.21.15.111:0/772402055 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b000aa60 con 0x7f82b8160d40 2023-03-29T10:58:01.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.706+0000 7f82affff700 1 -- 172.21.15.111:0/772402055 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82b000abe0 con 0x7f82b8160d40 2023-03-29T10:58:01.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.706+0000 7f82affff700 1 -- 172.21.15.111:0/772402055 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82b00070d0 con 0x7f82b8160d40 2023-03-29T10:58:01.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.706+0000 7f82bd5ff700 1 -- 172.21.15.111:0/772402055 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b80d19a0 con 0x7f82b8160d40 2023-03-29T10:58:01.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.706+0000 7f82bd5ff700 1 -- 172.21.15.111:0/772402055 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b80d1810 con 0x7f82b8160d40 2023-03-29T10:58:01.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.707+0000 7f82bd5ff700 1 -- 172.21.15.111:0/772402055 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b80d1b30 con 0x7f82b8160d40 2023-03-29T10:58:01.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.708+0000 7f82affff700 1 -- 172.21.15.111:0/772402055 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f82b0007230 con 0x7f82b8160d40 2023-03-29T10:58:01.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.709+0000 7f82affff700 1 --2- 172.21.15.111:0/772402055 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8298069de0 0x7f829806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:01.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.709+0000 7f82affff700 1 -- 172.21.15.111:0/772402055 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f82b0087c20 con 0x7f82b8160d40 2023-03-29T10:58:01.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.709+0000 7f82b6ffd700 1 --2- 172.21.15.111:0/772402055 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8298069de0 0x7f829806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:01.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.710+0000 7f82b6ffd700 1 --2- 172.21.15.111:0/772402055 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8298069de0 0x7f829806c290 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f82a80090b0 tx=0x7f82a801c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:01.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.714+0000 7f82affff700 1 -- 172.21.15.111:0/772402055 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f82b0053270 con 0x7f82b8160d40 2023-03-29T10:58:01.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.905+0000 7f82bd5ff700 1 -- 172.21.15.111:0/772402055 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82b8072060 con 0x7f82b8160d40 2023-03-29T10:58:01.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.906+0000 7f82affff700 1 -- 172.21.15.111:0/772402055 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v28) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82b00180a0 con 0x7f82b8160d40 2023-03-29T10:58:01.905 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-03-29T10:58:01.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.908+0000 7f82bd5ff700 1 -- 172.21.15.111:0/772402055 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8298069de0 msgr2=0x7f829806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:01.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.908+0000 7f82bd5ff700 1 --2- 172.21.15.111:0/772402055 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8298069de0 0x7f829806c290 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f82a80090b0 tx=0x7f82a801c040 comp rx=0 tx=0).stop 2023-03-29T10:58:01.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.908+0000 7f82bd5ff700 1 -- 172.21.15.111:0/772402055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82b8160d40 msgr2=0x7f82b815d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:01.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.908+0000 7f82bd5ff700 1 --2- 172.21.15.111:0/772402055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82b8160d40 0x7f82b815d830 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f82b000cbd0 tx=0x7f82b000cee0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.908+0000 7f82bd5ff700 1 -- 172.21.15.111:0/772402055 shutdown_connections 2023-03-29T10:58:01.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.908+0000 7f82bd5ff700 1 --2- 172.21.15.111:0/772402055 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8298069de0 0x7f829806c290 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.908+0000 7f82bd5ff700 1 --2- 172.21.15.111:0/772402055 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f82b8160d40 0x7f82b815d830 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.908+0000 7f82bd5ff700 1 --2- 172.21.15.111:0/772402055 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f82b80b87a0 0x7f82b815a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.908+0000 7f82bd5ff700 1 --2- 172.21.15.111:0/772402055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b81601e0 0x7f82b80ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:01.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.908+0000 7f82bd5ff700 1 -- 172.21.15.111:0/772402055 >> 172.21.15.111:0/772402055 conn(0x7f82b8071b10 msgr2=0x7f82b80d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:01.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.908+0000 7f82bd5ff700 1 -- 172.21.15.111:0/772402055 shutdown_connections 2023-03-29T10:58:01.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:01.908+0000 7f82bd5ff700 1 -- 172.21.15.111:0/772402055 wait complete. 2023-03-29T10:58:01.925 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-29T10:58:02.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.110+0000 7ff13b5ff700 1 Processor -- start 2023-03-29T10:58:02.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.110+0000 7ff13b5ff700 1 -- start start 2023-03-29T10:58:02.111 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.112+0000 7ff13b5ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff1340b87a0 0x7ff1340cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:02.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.112+0000 7ff13b5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1341601e0 0x7ff134160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:02.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.112+0000 7ff13b5ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff134160ff0 con 0x7ff134160b70 2023-03-29T10:58:02.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.112+0000 7ff13b5ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1340d02d0 con 0x7ff1340b87a0 2023-03-29T10:58:02.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.112+0000 7ff13b5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1340d0cb0 con 0x7ff1341601e0 2023-03-29T10:58:02.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.112+0000 7ff13939b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff1340b87a0 0x7ff1340cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:02.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.112+0000 7ff139b9c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff134160b70 0x7ff1340d2de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36984/0 (socket says 172.21.15.111:36984) 2023-03-29T10:58:02.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.112+0000 7ff139b9c700 1 -- 172.21.15.111:0/3088173663 learned_addr learned my addr 172.21.15.111:0/3088173663 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.112+0000 7ff138b9a700 1 --2- 172.21.15.111:0/3088173663 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1341601e0 0x7ff134160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.113+0000 7ff13939b700 1 -- 172.21.15.111:0/3088173663 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1341601e0 msgr2=0x7ff134160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.113+0000 7ff13939b700 1 --2- 172.21.15.111:0/3088173663 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1341601e0 0x7ff134160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.113+0000 7ff13939b700 1 -- 172.21.15.111:0/3088173663 >> v1:172.21.15.111:6789/0 conn(0x7ff134160b70 legacy=0x7ff1340d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.113+0000 7ff13939b700 1 -- 172.21.15.111:0/3088173663 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1340cdc60 con 0x7ff1340b87a0 2023-03-29T10:58:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.113+0000 7ff13939b700 1 --2- 172.21.15.111:0/3088173663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1340b87a0 0x7ff1340cfbf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff130009840 tx=0x7ff130009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a8505021c38570 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.113+0000 7ff12bfff700 1 -- 172.21.15.111:0/3088173663 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff130011830 con 0x7ff1340b87a0 2023-03-29T10:58:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.113+0000 7ff12bfff700 1 -- 172.21.15.111:0/3088173663 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff130011990 con 0x7ff1340b87a0 2023-03-29T10:58:02.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.113+0000 7ff12bfff700 1 -- 172.21.15.111:0/3088173663 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff130011d20 con 0x7ff1340b87a0 2023-03-29T10:58:02.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.114+0000 7ff13b5ff700 1 -- 172.21.15.111:0/3088173663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1340b87a0 msgr2=0x7ff1340cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:02.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.114+0000 7ff13b5ff700 1 --2- 172.21.15.111:0/3088173663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1340b87a0 0x7ff1340cfbf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff130009840 tx=0x7ff130009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:02.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.114+0000 7ff13b5ff700 1 -- 172.21.15.111:0/3088173663 shutdown_connections 2023-03-29T10:58:02.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.114+0000 7ff13b5ff700 1 --2- 172.21.15.111:0/3088173663 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff1340b87a0 0x7ff1340cfbf0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.114+0000 7ff13b5ff700 1 --2- 172.21.15.111:0/3088173663 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1341601e0 0x7ff134160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.114+0000 7ff13b5ff700 1 -- 172.21.15.111:0/3088173663 >> 172.21.15.111:0/3088173663 conn(0x7ff134071b10 msgr2=0x7ff134071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:02.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.114+0000 7ff13b5ff700 1 -- 172.21.15.111:0/3088173663 shutdown_connections 2023-03-29T10:58:02.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.114+0000 7ff13b5ff700 1 -- 172.21.15.111:0/3088173663 wait complete. 2023-03-29T10:58:02.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.115+0000 7ff13b5ff700 1 Processor -- start 2023-03-29T10:58:02.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.115+0000 7ff13b5ff700 1 -- start start 2023-03-29T10:58:02.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.116+0000 7ff13b5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1340b87a0 0x7ff1340cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:02.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.116+0000 7ff13b5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1341601e0 0x7ff1340c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:02.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.116+0000 7ff13b5ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff134160b70 0x7ff134150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:02.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.116+0000 7ff13b5ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13415d370 con 0x7ff134160b70 2023-03-29T10:58:02.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.116+0000 7ff13b5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1340cb310 con 0x7ff1340b87a0 2023-03-29T10:58:02.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.116+0000 7ff13b5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1340ccc90 con 0x7ff1341601e0 2023-03-29T10:58:02.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.117+0000 7ff139b9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff134160b70 0x7ff134150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:02.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.117+0000 7ff139b9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff134160b70 0x7ff134150320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50426/0 (socket says 172.21.15.111:50426) 2023-03-29T10:58:02.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.117+0000 7ff139b9c700 1 -- 172.21.15.111:0/2917388743 learned_addr learned my addr 172.21.15.111:0/2917388743 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:02.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.117+0000 7ff13939b700 1 --2- 172.21.15.111:0/2917388743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1340b87a0 0x7ff1340cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:02.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.117+0000 7ff138b9a700 1 --2- 172.21.15.111:0/2917388743 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1341601e0 0x7ff1340c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:02.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.117+0000 7ff139b9c700 1 -- 172.21.15.111:0/2917388743 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1341601e0 msgr2=0x7ff1340c69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:02.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.117+0000 7ff139b9c700 1 --2- 172.21.15.111:0/2917388743 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1341601e0 0x7ff1340c69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.117+0000 7ff139b9c700 1 -- 172.21.15.111:0/2917388743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1340b87a0 msgr2=0x7ff1340cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:02.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.117+0000 7ff139b9c700 1 --2- 172.21.15.111:0/2917388743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1340b87a0 0x7ff1340cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.117+0000 7ff139b9c700 1 -- 172.21.15.111:0/2917388743 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1300094a0 con 0x7ff134160b70 2023-03-29T10:58:02.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.117+0000 7ff13939b700 1 --2- 172.21.15.111:0/2917388743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1340b87a0 0x7ff1340cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:02.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.117+0000 7ff139b9c700 1 --2- 172.21.15.111:0/2917388743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff134160b70 0x7ff134150320 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00ca80 tx=0x7ff12c00ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:02.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.117+0000 7ff129ffb700 1 -- 172.21.15.111:0/2917388743 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00aaa0 con 0x7ff134160b70 2023-03-29T10:58:02.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.118+0000 7ff129ffb700 1 -- 172.21.15.111:0/2917388743 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c00ac00 con 0x7ff134160b70 2023-03-29T10:58:02.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.118+0000 7ff13b5ff700 1 -- 172.21.15.111:0/2917388743 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1340cd980 con 0x7ff134160b70 2023-03-29T10:58:02.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.118+0000 7ff129ffb700 1 -- 172.21.15.111:0/2917388743 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c012400 con 0x7ff134160b70 2023-03-29T10:58:02.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.118+0000 7ff13b5ff700 1 -- 172.21.15.111:0/2917388743 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1340cd6a0 con 0x7ff134160b70 2023-03-29T10:58:02.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.119+0000 7ff13b5ff700 1 -- 172.21.15.111:0/2917388743 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1340d2110 con 0x7ff134160b70 2023-03-29T10:58:02.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.120+0000 7ff129ffb700 1 -- 172.21.15.111:0/2917388743 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7ff12c01b3b0 con 0x7ff134160b70 2023-03-29T10:58:02.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.120+0000 7ff129ffb700 1 --2- 172.21.15.111:0/2917388743 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff118069de0 0x7ff11806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:02.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.120+0000 7ff129ffb700 1 -- 172.21.15.111:0/2917388743 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff12c087e00 con 0x7ff134160b70 2023-03-29T10:58:02.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.121+0000 7ff13939b700 1 --2- 172.21.15.111:0/2917388743 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff118069de0 0x7ff11806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:02.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.121+0000 7ff13939b700 1 --2- 172.21.15.111:0/2917388743 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff118069de0 0x7ff11806c290 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff1300090b0 tx=0x7ff130008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:02.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.125+0000 7ff129ffb700 1 -- 172.21.15.111:0/2917388743 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7ff12c04c160 con 0x7ff134160b70 2023-03-29T10:58:02.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.316+0000 7ff13b5ff700 1 -- 172.21.15.111:0/2917388743 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff13415d4b0 con 0x7ff134160b70 2023-03-29T10:58:02.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.316+0000 7ff129ffb700 1 -- 172.21.15.111:0/2917388743 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v28) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff13415d4b0 con 0x7ff134160b70 2023-03-29T10:58:02.316 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 8 2023-03-29T10:58:02.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.318+0000 7ff13b5ff700 1 -- 172.21.15.111:0/2917388743 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff118069de0 msgr2=0x7ff11806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:02.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.319+0000 7ff13b5ff700 1 --2- 172.21.15.111:0/2917388743 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff118069de0 0x7ff11806c290 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff1300090b0 tx=0x7ff130008030 comp rx=0 tx=0).stop 2023-03-29T10:58:02.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.319+0000 7ff13b5ff700 1 -- 172.21.15.111:0/2917388743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff134160b70 msgr2=0x7ff134150320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:02.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.319+0000 7ff13b5ff700 1 --2- 172.21.15.111:0/2917388743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff134160b70 0x7ff134150320 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00ca80 tx=0x7ff12c00ce40 comp rx=0 tx=0).stop 2023-03-29T10:58:02.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.319+0000 7ff13b5ff700 1 -- 172.21.15.111:0/2917388743 shutdown_connections 2023-03-29T10:58:02.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.319+0000 7ff13b5ff700 1 --2- 172.21.15.111:0/2917388743 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff118069de0 0x7ff11806c290 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.319+0000 7ff13b5ff700 1 --2- 172.21.15.111:0/2917388743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff134160b70 0x7ff134150320 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.319+0000 7ff13b5ff700 1 --2- 172.21.15.111:0/2917388743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1340b87a0 0x7ff1340cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.319+0000 7ff13b5ff700 1 --2- 172.21.15.111:0/2917388743 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff1341601e0 0x7ff1340c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.319+0000 7ff13b5ff700 1 -- 172.21.15.111:0/2917388743 >> 172.21.15.111:0/2917388743 conn(0x7ff134071b10 msgr2=0x7ff134071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:02.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.319+0000 7ff13b5ff700 1 -- 172.21.15.111:0/2917388743 shutdown_connections 2023-03-29T10:58:02.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.319+0000 7ff13b5ff700 1 -- 172.21.15.111:0/2917388743 wait complete. 2023-03-29T10:58:02.336 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-29T10:58:02.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.522+0000 7f87af7ff700 1 Processor -- start 2023-03-29T10:58:02.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.522+0000 7f87af7ff700 1 -- start start 2023-03-29T10:58:02.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.523+0000 7f87af7ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87a80cfbf0 0x7f87a81608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:02.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.523+0000 7f87af7ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f87a80d1100 0x7f87a80d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:02.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.523+0000 7f87af7ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a80d0020 con 0x7f87a80b87a0 2023-03-29T10:58:02.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.523+0000 7f87af7ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a80d02d0 con 0x7f87a80d1100 2023-03-29T10:58:02.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.523+0000 7f87af7ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a80d0cb0 con 0x7f87a80cfbf0 2023-03-29T10:58:02.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.524+0000 7f87ad59b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f87a80b87a0 0x7f87a81601e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:36992/0 (socket says 172.21.15.111:36992) 2023-03-29T10:58:02.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.524+0000 7f87ad59b700 1 -- 172.21.15.111:0/2472485440 learned_addr learned my addr 172.21.15.111:0/2472485440 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:02.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.524+0000 7f87add9c700 1 --2- 172.21.15.111:0/2472485440 >> v2:172.21.15.119:3300/0 conn(0x7f87a80d1100 0x7f87a80d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:02.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.524+0000 7f87acd9a700 1 --2- 172.21.15.111:0/2472485440 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87a80cfbf0 0x7f87a81608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:02.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.524+0000 7f879ffff700 1 -- 172.21.15.111:0/2472485440 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1755592541 0 0) 0x7f87a80d0020 con 0x7f87a80b87a0 2023-03-29T10:58:02.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.524+0000 7f879ffff700 1 -- 172.21.15.111:0/2472485440 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87940035b0 con 0x7f87a80b87a0 2023-03-29T10:58:02.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.524+0000 7f87add9c700 1 -- 172.21.15.111:0/2472485440 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87a80cfbf0 msgr2=0x7f87a81608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:02.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.524+0000 7f87add9c700 1 --2- 172.21.15.111:0/2472485440 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87a80cfbf0 0x7f87a81608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.524+0000 7f87add9c700 1 -- 172.21.15.111:0/2472485440 >> v1:172.21.15.111:6789/0 conn(0x7f87a80b87a0 legacy=0x7f87a81601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:02.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.524+0000 7f87add9c700 1 -- 172.21.15.111:0/2472485440 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a80cdc60 con 0x7f87a80d1100 2023-03-29T10:58:02.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.524+0000 7f87acd9a700 1 --2- 172.21.15.111:0/2472485440 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87a80cfbf0 0x7f87a81608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:58:02.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.525+0000 7f87add9c700 1 --2- 172.21.15.111:0/2472485440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87a80d1100 0x7f87a80d1580 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009d90 tx=0x7f87a000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60147947bffc4ff7 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:02.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.525+0000 7f879ffff700 1 -- 172.21.15.111:0/2472485440 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a000e070 con 0x7f87a80d1100 2023-03-29T10:58:02.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.525+0000 7f879ffff700 1 -- 172.21.15.111:0/2472485440 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a000bb20 con 0x7f87a80d1100 2023-03-29T10:58:02.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.525+0000 7f879ffff700 1 -- 172.21.15.111:0/2472485440 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a0012430 con 0x7f87a80d1100 2023-03-29T10:58:02.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.525+0000 7f87af7ff700 1 -- 172.21.15.111:0/2472485440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87a80d1100 msgr2=0x7f87a80d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:02.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.525+0000 7f87af7ff700 1 --2- 172.21.15.111:0/2472485440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87a80d1100 0x7f87a80d1580 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009d90 tx=0x7f87a000b330 comp rx=0 tx=0).stop 2023-03-29T10:58:02.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.526+0000 7f87af7ff700 1 -- 172.21.15.111:0/2472485440 shutdown_connections 2023-03-29T10:58:02.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.526+0000 7f87af7ff700 1 --2- 172.21.15.111:0/2472485440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87a80d1100 0x7f87a80d1580 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.526+0000 7f87af7ff700 1 --2- 172.21.15.111:0/2472485440 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87a80cfbf0 0x7f87a81608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.526+0000 7f87af7ff700 1 -- 172.21.15.111:0/2472485440 >> 172.21.15.111:0/2472485440 conn(0x7f87a8071b10 msgr2=0x7f87a8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:02.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.526+0000 7f87af7ff700 1 -- 172.21.15.111:0/2472485440 shutdown_connections 2023-03-29T10:58:02.527 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.526+0000 7f87af7ff700 1 -- 172.21.15.111:0/2472485440 wait complete. 2023-03-29T10:58:02.527 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.527+0000 7f87af7ff700 1 Processor -- start 2023-03-29T10:58:02.527 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.527+0000 7f87af7ff700 1 -- start start 2023-03-29T10:58:02.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.528+0000 7f87af7ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87a80b87a0 0x7f87a80caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:02.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.528+0000 7f87af7ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87a80cfbf0 0x7f87a815f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:02.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.528+0000 7f87af7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87a80d1100 0x7f87a814a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:02.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.528+0000 7f87af7ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a81585e0 con 0x7f87a80b87a0 2023-03-29T10:58:02.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.528+0000 7f87af7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a815fa30 con 0x7f87a80d1100 2023-03-29T10:58:02.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.528+0000 7f87af7ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a81587f0 con 0x7f87a80cfbf0 2023-03-29T10:58:02.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.529+0000 7f87ad59b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87a80b87a0 0x7f87a80caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:02.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.529+0000 7f87ad59b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87a80b87a0 0x7f87a80caf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50438/0 (socket says 172.21.15.111:50438) 2023-03-29T10:58:02.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.529+0000 7f87ad59b700 1 -- 172.21.15.111:0/1029870925 learned_addr learned my addr 172.21.15.111:0/1029870925 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:02.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.529+0000 7f87acd9a700 1 --2- 172.21.15.111:0/1029870925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87a80cfbf0 0x7f87a815f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:02.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.529+0000 7f87add9c700 1 --2- 172.21.15.111:0/1029870925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87a80d1100 0x7f87a814a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:02.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.529+0000 7f87ad59b700 1 -- 172.21.15.111:0/1029870925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87a80cfbf0 msgr2=0x7f87a815f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:02.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.529+0000 7f87ad59b700 1 --2- 172.21.15.111:0/1029870925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87a80cfbf0 0x7f87a815f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.529+0000 7f87ad59b700 1 -- 172.21.15.111:0/1029870925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87a80d1100 msgr2=0x7f87a814a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:02.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.529+0000 7f87ad59b700 1 --2- 172.21.15.111:0/1029870925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87a80d1100 0x7f87a814a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.529+0000 7f87ad59b700 1 -- 172.21.15.111:0/1029870925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87980094a0 con 0x7f87a80b87a0 2023-03-29T10:58:02.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.529+0000 7f87ad59b700 1 --2- 172.21.15.111:0/1029870925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87a80b87a0 0x7f87a80caf00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f87a4001eb0 tx=0x7f87a400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:02.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.529+0000 7f879dffb700 1 -- 172.21.15.111:0/1029870925 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4005d70 con 0x7f87a80b87a0 2023-03-29T10:58:02.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.529+0000 7f879dffb700 1 -- 172.21.15.111:0/1029870925 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a40070d0 con 0x7f87a80b87a0 2023-03-29T10:58:02.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.529+0000 7f879dffb700 1 -- 172.21.15.111:0/1029870925 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4007460 con 0x7f87a80b87a0 2023-03-29T10:58:02.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.529+0000 7f87af7ff700 1 -- 172.21.15.111:0/1029870925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a0009a70 con 0x7f87a80b87a0 2023-03-29T10:58:02.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.530+0000 7f87af7ff700 1 -- 172.21.15.111:0/1029870925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a80cd980 con 0x7f87a80b87a0 2023-03-29T10:58:02.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.531+0000 7f87af7ff700 1 -- 172.21.15.111:0/1029870925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a80cd6a0 con 0x7f87a80b87a0 2023-03-29T10:58:02.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.531+0000 7f879dffb700 1 -- 172.21.15.111:0/1029870925 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f87a40075c0 con 0x7f87a80b87a0 2023-03-29T10:58:02.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.532+0000 7f879dffb700 1 --2- 172.21.15.111:0/1029870925 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f878c069de0 0x7f878c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:02.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.532+0000 7f879dffb700 1 -- 172.21.15.111:0/1029870925 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f87a4087750 con 0x7f87a80b87a0 2023-03-29T10:58:02.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.533+0000 7f87acd9a700 1 --2- 172.21.15.111:0/1029870925 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f878c069de0 0x7f878c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:02.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.533+0000 7f87acd9a700 1 --2- 172.21.15.111:0/1029870925 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f878c069de0 0x7f878c06c290 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f87a80ca820 tx=0x7f8798009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:02.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.536+0000 7f879dffb700 1 -- 172.21.15.111:0/1029870925 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f87a4052c70 con 0x7f87a80b87a0 2023-03-29T10:58:02.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.728+0000 7f87af7ff700 1 -- 172.21.15.111:0/1029870925 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f87a8160e30 con 0x7f87a80b87a0 2023-03-29T10:58:02.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.729+0000 7f879dffb700 1 -- 172.21.15.111:0/1029870925 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v28) v1 ==== 104+0+11 (secure 0 0 0) 0x7f87a4056290 con 0x7f87a80b87a0 2023-03-29T10:58:02.729 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:02.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.731+0000 7f87af7ff700 1 -- 172.21.15.111:0/1029870925 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f878c069de0 msgr2=0x7f878c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:02.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.731+0000 7f87af7ff700 1 --2- 172.21.15.111:0/1029870925 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f878c069de0 0x7f878c06c290 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f87a80ca820 tx=0x7f8798009210 comp rx=0 tx=0).stop 2023-03-29T10:58:02.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.731+0000 7f87af7ff700 1 -- 172.21.15.111:0/1029870925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87a80b87a0 msgr2=0x7f87a80caf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:02.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.731+0000 7f87af7ff700 1 --2- 172.21.15.111:0/1029870925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87a80b87a0 0x7f87a80caf00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f87a4001eb0 tx=0x7f87a400c7b0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.731+0000 7f87af7ff700 1 -- 172.21.15.111:0/1029870925 shutdown_connections 2023-03-29T10:58:02.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.731+0000 7f87af7ff700 1 --2- 172.21.15.111:0/1029870925 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f878c069de0 0x7f878c06c290 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.731+0000 7f87af7ff700 1 --2- 172.21.15.111:0/1029870925 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f87a80b87a0 0x7f87a80caf00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.731+0000 7f87af7ff700 1 --2- 172.21.15.111:0/1029870925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87a80d1100 0x7f87a814a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.731+0000 7f87af7ff700 1 --2- 172.21.15.111:0/1029870925 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f87a80cfbf0 0x7f87a815f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.731+0000 7f87af7ff700 1 -- 172.21.15.111:0/1029870925 >> 172.21.15.111:0/1029870925 conn(0x7f87a8071b10 msgr2=0x7f87a8071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:02.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.731+0000 7f87af7ff700 1 -- 172.21.15.111:0/1029870925 shutdown_connections 2023-03-29T10:58:02.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.731+0000 7f87af7ff700 1 -- 172.21.15.111:0/1029870925 wait complete. 2023-03-29T10:58:02.748 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-29T10:58:02.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.934+0000 7fd59f7ff700 1 Processor -- start 2023-03-29T10:58:02.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.934+0000 7fd59f7ff700 1 -- start start 2023-03-29T10:58:02.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.935+0000 7fd59f7ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd59815c590 0x7fd598160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:02.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.935+0000 7fd59f7ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd5980cf010 0x7fd5980cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:02.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.935+0000 7fd59f7ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5980cfd80 0x7fd5980d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:02.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.935+0000 7fd59f7ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5980cd520 con 0x7fd59815c590 2023-03-29T10:58:02.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.935+0000 7fd59f7ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5980d2640 con 0x7fd5980cf010 2023-03-29T10:58:02.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.935+0000 7fd59f7ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5981610d0 con 0x7fd5980cfd80 2023-03-29T10:58:02.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.936+0000 7fd59d59b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd59815c590 0x7fd598160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:02.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.936+0000 7fd59d59b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd59815c590 0x7fd598160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50440/0 (socket says 172.21.15.111:50440) 2023-03-29T10:58:02.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.936+0000 7fd59d59b700 1 -- 172.21.15.111:0/4221825554 learned_addr learned my addr 172.21.15.111:0/4221825554 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:02.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.936+0000 7fd59cd9a700 1 --2- 172.21.15.111:0/4221825554 >> v2:172.21.15.119:3300/0 conn(0x7fd5980cf010 0x7fd5980cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:02.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.936+0000 7fd59dd9c700 1 --2- 172.21.15.111:0/4221825554 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5980cfd80 0x7fd5980d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:02.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.936+0000 7fd59d59b700 1 -- 172.21.15.111:0/4221825554 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5980cfd80 msgr2=0x7fd5980d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:02.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.936+0000 7fd59d59b700 1 --2- 172.21.15.111:0/4221825554 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5980cfd80 0x7fd5980d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.936+0000 7fd59d59b700 1 -- 172.21.15.111:0/4221825554 >> v2:172.21.15.119:3300/0 conn(0x7fd5980cf010 msgr2=0x7fd5980cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:02.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.936+0000 7fd59d59b700 1 --2- 172.21.15.111:0/4221825554 >> v2:172.21.15.119:3300/0 conn(0x7fd5980cf010 0x7fd5980cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.936+0000 7fd59d59b700 1 -- 172.21.15.111:0/4221825554 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5980d01e0 con 0x7fd59815c590 2023-03-29T10:58:02.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.936+0000 7fd59cd9a700 1 --2- 172.21.15.111:0/4221825554 >> v2:172.21.15.119:3300/0 conn(0x7fd5980cf010 0x7fd5980cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:02.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.936+0000 7fd59d59b700 1 --2- 172.21.15.111:0/4221825554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd59815c590 0x7fd598160b90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd5980b4ee0 tx=0x7fd594009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd074b9f5fdac6b server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:02.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.937+0000 7fd58ffff700 1 -- 172.21.15.111:0/4221825554 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59400b870 con 0x7fd59815c590 2023-03-29T10:58:02.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.937+0000 7fd58ffff700 1 -- 172.21.15.111:0/4221825554 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59400b9d0 con 0x7fd59815c590 2023-03-29T10:58:02.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.937+0000 7fd58ffff700 1 -- 172.21.15.111:0/4221825554 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59400bd60 con 0x7fd59815c590 2023-03-29T10:58:02.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.937+0000 7fd59f7ff700 1 -- 172.21.15.111:0/4221825554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd59815c590 msgr2=0x7fd598160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:02.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.937+0000 7fd59f7ff700 1 --2- 172.21.15.111:0/4221825554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd59815c590 0x7fd598160b90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd5980b4ee0 tx=0x7fd594009bd0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.937+0000 7fd59f7ff700 1 -- 172.21.15.111:0/4221825554 shutdown_connections 2023-03-29T10:58:02.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.937+0000 7fd59f7ff700 1 --2- 172.21.15.111:0/4221825554 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5980cfd80 0x7fd5980d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.937+0000 7fd59f7ff700 1 --2- 172.21.15.111:0/4221825554 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd59815c590 0x7fd598160b90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.937+0000 7fd59f7ff700 1 --2- 172.21.15.111:0/4221825554 >> v2:172.21.15.119:3300/0 conn(0x7fd5980cf010 0x7fd5980cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.937+0000 7fd59f7ff700 1 -- 172.21.15.111:0/4221825554 >> 172.21.15.111:0/4221825554 conn(0x7fd598071b10 msgr2=0x7fd598071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:02.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.937+0000 7fd59f7ff700 1 -- 172.21.15.111:0/4221825554 shutdown_connections 2023-03-29T10:58:02.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.937+0000 7fd59f7ff700 1 -- 172.21.15.111:0/4221825554 wait complete. 2023-03-29T10:58:02.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.939+0000 7fd59f7ff700 1 Processor -- start 2023-03-29T10:58:02.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.939+0000 7fd59f7ff700 1 -- start start 2023-03-29T10:58:02.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.940+0000 7fd59f7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5980cf010 0x7fd5981f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:02.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.940+0000 7fd59f7ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5980cfd80 0x7fd5980c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:02.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.940+0000 7fd59f7ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd59815c590 0x7fd59815e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:02.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.940+0000 7fd59f7ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59815d830 con 0x7fd59815c590 2023-03-29T10:58:02.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.940+0000 7fd59f7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59815db10 con 0x7fd5980cf010 2023-03-29T10:58:02.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.940+0000 7fd59f7ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59815ddf0 con 0x7fd5980cfd80 2023-03-29T10:58:02.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.940+0000 7fd59dd9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd59815c590 0x7fd59815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:02.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.940+0000 7fd59dd9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd59815c590 0x7fd59815e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50442/0 (socket says 172.21.15.111:50442) 2023-03-29T10:58:02.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.940+0000 7fd59dd9c700 1 -- 172.21.15.111:0/2011230276 learned_addr learned my addr 172.21.15.111:0/2011230276 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:02.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.941+0000 7fd59d59b700 1 --2- 172.21.15.111:0/2011230276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5980cf010 0x7fd5981f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:02.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.941+0000 7fd59dd9c700 1 -- 172.21.15.111:0/2011230276 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5980cfd80 msgr2=0x7fd5980c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:02.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.941+0000 7fd59dd9c700 1 --2- 172.21.15.111:0/2011230276 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5980cfd80 0x7fd5980c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.941+0000 7fd59dd9c700 1 -- 172.21.15.111:0/2011230276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5980cf010 msgr2=0x7fd5981f17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:02.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.941+0000 7fd59dd9c700 1 --2- 172.21.15.111:0/2011230276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5980cf010 0x7fd5981f17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:02.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.941+0000 7fd59dd9c700 1 -- 172.21.15.111:0/2011230276 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd594009550 con 0x7fd59815c590 2023-03-29T10:58:02.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.941+0000 7fd59d59b700 1 --2- 172.21.15.111:0/2011230276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5980cf010 0x7fd5981f17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:02.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.941+0000 7fd59dd9c700 1 --2- 172.21.15.111:0/2011230276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd59815c590 0x7fd59815e530 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd590007e40 tx=0x7fd59000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:02.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.941+0000 7fd58dffb700 1 -- 172.21.15.111:0/2011230276 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59000eea0 con 0x7fd59815c590 2023-03-29T10:58:02.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.941+0000 7fd58dffb700 1 -- 172.21.15.111:0/2011230276 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd590014420 con 0x7fd59815c590 2023-03-29T10:58:02.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.941+0000 7fd59f7ff700 1 -- 172.21.15.111:0/2011230276 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5980cf560 con 0x7fd59815c590 2023-03-29T10:58:02.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.941+0000 7fd58dffb700 1 -- 172.21.15.111:0/2011230276 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5900147b0 con 0x7fd59815c590 2023-03-29T10:58:02.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.941+0000 7fd59f7ff700 1 -- 172.21.15.111:0/2011230276 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5980b23c0 con 0x7fd59815c590 2023-03-29T10:58:02.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.942+0000 7fd59f7ff700 1 -- 172.21.15.111:0/2011230276 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5980cd770 con 0x7fd59815c590 2023-03-29T10:58:02.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.943+0000 7fd58dffb700 1 -- 172.21.15.111:0/2011230276 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fd590014910 con 0x7fd59815c590 2023-03-29T10:58:02.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.944+0000 7fd58dffb700 1 --2- 172.21.15.111:0/2011230276 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd57c069de0 0x7fd57c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:02.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.945+0000 7fd58dffb700 1 -- 172.21.15.111:0/2011230276 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd590088a20 con 0x7fd59815c590 2023-03-29T10:58:02.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.945+0000 7fd59d59b700 1 --2- 172.21.15.111:0/2011230276 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd57c069de0 0x7fd57c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:02.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.945+0000 7fd59d59b700 1 --2- 172.21.15.111:0/2011230276 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd57c069de0 0x7fd57c06c290 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd598158280 tx=0x7fd594008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:02.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:02.949+0000 7fd58dffb700 1 -- 172.21.15.111:0/2011230276 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fd590053fd0 con 0x7fd59815c590 2023-03-29T10:58:03.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.138+0000 7fd59f7ff700 1 -- 172.21.15.111:0/2011230276 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd598074430 con 0x7fd59815c590 2023-03-29T10:58:03.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.139+0000 7fd58dffb700 1 -- 172.21.15.111:0/2011230276 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v28) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd5900575f0 con 0x7fd59815c590 2023-03-29T10:58:03.139 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-03-29T10:58:03.141 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.141+0000 7fd59f7ff700 1 -- 172.21.15.111:0/2011230276 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd57c069de0 msgr2=0x7fd57c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:03.141 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.141+0000 7fd59f7ff700 1 --2- 172.21.15.111:0/2011230276 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd57c069de0 0x7fd57c06c290 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd598158280 tx=0x7fd594008030 comp rx=0 tx=0).stop 2023-03-29T10:58:03.141 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.141+0000 7fd59f7ff700 1 -- 172.21.15.111:0/2011230276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd59815c590 msgr2=0x7fd59815e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:03.141 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.141+0000 7fd59f7ff700 1 --2- 172.21.15.111:0/2011230276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd59815c590 0x7fd59815e530 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd590007e40 tx=0x7fd59000e3c0 comp rx=0 tx=0).stop 2023-03-29T10:58:03.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.142+0000 7fd59f7ff700 1 -- 172.21.15.111:0/2011230276 shutdown_connections 2023-03-29T10:58:03.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.142+0000 7fd59f7ff700 1 --2- 172.21.15.111:0/2011230276 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd57c069de0 0x7fd57c06c290 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:03.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.142+0000 7fd59f7ff700 1 --2- 172.21.15.111:0/2011230276 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd59815c590 0x7fd59815e530 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:03.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.142+0000 7fd59f7ff700 1 --2- 172.21.15.111:0/2011230276 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5980cf010 0x7fd5981f17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:03.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.142+0000 7fd59f7ff700 1 --2- 172.21.15.111:0/2011230276 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5980cfd80 0x7fd5980c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:03.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.142+0000 7fd59f7ff700 1 -- 172.21.15.111:0/2011230276 >> 172.21.15.111:0/2011230276 conn(0x7fd598071b10 msgr2=0x7fd5980d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:03.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.142+0000 7fd59f7ff700 1 -- 172.21.15.111:0/2011230276 shutdown_connections 2023-03-29T10:58:03.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.142+0000 7fd59f7ff700 1 -- 172.21.15.111:0/2011230276 wait complete. 2023-03-29T10:58:03.158 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-29T10:58:03.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.344+0000 7f0c617ff700 1 Processor -- start 2023-03-29T10:58:03.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.344+0000 7f0c617ff700 1 -- start start 2023-03-29T10:58:03.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.346+0000 7f0c617ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0c5c15c590 0x7f0c5c160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:03.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.346+0000 7f0c617ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5c0cd520 con 0x7f0c5c0cf010 2023-03-29T10:58:03.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.346+0000 7f0c617ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c5c0d2640 con 0x7f0c5c0cff50 2023-03-29T10:58:03.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.346+0000 7f0c617ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c5c1610d0 con 0x7f0c5c15c590 2023-03-29T10:58:03.346 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.346+0000 7f0c53fff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f0c5c0cf010 0x7f0c5c0cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37008/0 (socket says 172.21.15.111:37008) 2023-03-29T10:58:03.346 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.346+0000 7f0c53fff700 1 -- 172.21.15.111:0/4153422538 learned_addr learned my addr 172.21.15.111:0/4153422538 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:03.346 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.346+0000 7f0c5affd700 1 --2- 172.21.15.111:0/4153422538 >> v2:172.21.15.119:3300/0 conn(0x7f0c5c15c590 0x7f0c5c160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:03.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.346+0000 7f0c5a7fc700 1 -- 172.21.15.111:0/4153422538 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3751387437 0 0) 0x7f0c5c0cd520 con 0x7f0c5c0cf010 2023-03-29T10:58:03.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.346+0000 7f0c5a7fc700 1 -- 172.21.15.111:0/4153422538 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c480035b0 con 0x7f0c5c0cf010 2023-03-29T10:58:03.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.346+0000 7f0c5a7fc700 1 -- 172.21.15.111:0/4153422538 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3804298669 0 0) 0x7f0c5c0d2640 con 0x7f0c5c0cff50 2023-03-29T10:58:03.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.346+0000 7f0c5a7fc700 1 -- 172.21.15.111:0/4153422538 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c5c0cd520 con 0x7f0c5c0cff50 2023-03-29T10:58:03.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.347+0000 7f0c5affd700 1 -- 172.21.15.111:0/4153422538 >> v1:172.21.15.119:6789/0 conn(0x7f0c5c0cff50 legacy=0x7f0c5c0d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:03.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.347+0000 7f0c5affd700 1 -- 172.21.15.111:0/4153422538 >> v1:172.21.15.111:6789/0 conn(0x7f0c5c0cf010 legacy=0x7f0c5c0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:03.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.347+0000 7f0c5affd700 1 -- 172.21.15.111:0/4153422538 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c0b23c0 con 0x7f0c5c15c590 2023-03-29T10:58:03.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.347+0000 7f0c5affd700 1 --2- 172.21.15.111:0/4153422538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c5c15c590 0x7f0c5c160b90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009840 tx=0x7f0c4c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b46fa6e04aade4ab server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:03.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.347+0000 7f0c5a7fc700 1 -- 172.21.15.111:0/4153422538 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c011830 con 0x7f0c5c15c590 2023-03-29T10:58:03.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.347+0000 7f0c5a7fc700 1 -- 172.21.15.111:0/4153422538 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4c011990 con 0x7f0c5c15c590 2023-03-29T10:58:03.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.347+0000 7f0c5a7fc700 1 -- 172.21.15.111:0/4153422538 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c011d20 con 0x7f0c5c15c590 2023-03-29T10:58:03.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.347+0000 7f0c617ff700 1 -- 172.21.15.111:0/4153422538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c5c15c590 msgr2=0x7f0c5c160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:03.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.347+0000 7f0c617ff700 1 --2- 172.21.15.111:0/4153422538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c5c15c590 0x7f0c5c160b90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009840 tx=0x7f0c4c009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:03.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.348+0000 7f0c617ff700 1 -- 172.21.15.111:0/4153422538 shutdown_connections 2023-03-29T10:58:03.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.348+0000 7f0c617ff700 1 --2- 172.21.15.111:0/4153422538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c5c15c590 0x7f0c5c160b90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:03.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.348+0000 7f0c617ff700 1 -- 172.21.15.111:0/4153422538 >> 172.21.15.111:0/4153422538 conn(0x7f0c5c071b10 msgr2=0x7f0c5c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:03.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.348+0000 7f0c617ff700 1 -- 172.21.15.111:0/4153422538 shutdown_connections 2023-03-29T10:58:03.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.348+0000 7f0c617ff700 1 -- 172.21.15.111:0/4153422538 wait complete. 2023-03-29T10:58:03.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.349+0000 7f0c617ff700 1 Processor -- start 2023-03-29T10:58:03.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.349+0000 7f0c617ff700 1 -- start start 2023-03-29T10:58:03.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.350+0000 7f0c617ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c5c0cf010 0x7f0c5c154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:03.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.350+0000 7f0c617ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c5c0cff50 0x7f0c5c1f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:03.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.350+0000 7f0c617ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c5c15c590 0x7f0c5c15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:03.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.350+0000 7f0c617ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0b5d00 con 0x7f0c5c0cff50 2023-03-29T10:58:03.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.350+0000 7f0c617ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c0b6280 con 0x7f0c5c0cf010 2023-03-29T10:58:03.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.350+0000 7f0c617ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c1f6140 con 0x7f0c5c15c590 2023-03-29T10:58:03.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.351+0000 7f0c53fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c5c0cff50 0x7f0c5c1f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:03.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.351+0000 7f0c53fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c5c0cff50 0x7f0c5c1f56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50448/0 (socket says 172.21.15.111:50448) 2023-03-29T10:58:03.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.351+0000 7f0c53fff700 1 -- 172.21.15.111:0/3272964931 learned_addr learned my addr 172.21.15.111:0/3272964931 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:03.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.351+0000 7f0c5affd700 1 --2- 172.21.15.111:0/3272964931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c5c0cf010 0x7f0c5c154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:03.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.351+0000 7f0c5b7fe700 1 --2- 172.21.15.111:0/3272964931 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c5c15c590 0x7f0c5c15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:03.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.351+0000 7f0c53fff700 1 -- 172.21.15.111:0/3272964931 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c5c15c590 msgr2=0x7f0c5c15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:03.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.351+0000 7f0c53fff700 1 --2- 172.21.15.111:0/3272964931 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c5c15c590 0x7f0c5c15ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:03.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.351+0000 7f0c53fff700 1 -- 172.21.15.111:0/3272964931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c5c0cf010 msgr2=0x7f0c5c154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:03.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.351+0000 7f0c53fff700 1 --2- 172.21.15.111:0/3272964931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c5c0cf010 0x7f0c5c154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:03.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.351+0000 7f0c53fff700 1 -- 172.21.15.111:0/3272964931 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c0094a0 con 0x7f0c5c0cff50 2023-03-29T10:58:03.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.351+0000 7f0c5b7fe700 1 --2- 172.21.15.111:0/3272964931 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c5c15c590 0x7f0c5c15ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:03.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.351+0000 7f0c5affd700 1 --2- 172.21.15.111:0/3272964931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c5c0cf010 0x7f0c5c154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:03.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.351+0000 7f0c53fff700 1 --2- 172.21.15.111:0/3272964931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c5c0cff50 0x7f0c5c1f56a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400c700 tx=0x7f0c4400cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:03.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.351+0000 7f0c537fe700 1 -- 172.21.15.111:0/3272964931 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c44005d70 con 0x7f0c5c0cff50 2023-03-29T10:58:03.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.352+0000 7f0c537fe700 1 -- 172.21.15.111:0/3272964931 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c440070d0 con 0x7f0c5c0cff50 2023-03-29T10:58:03.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.352+0000 7f0c617ff700 1 -- 172.21.15.111:0/3272964931 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c5c0cd770 con 0x7f0c5c0cff50 2023-03-29T10:58:03.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.352+0000 7f0c537fe700 1 -- 172.21.15.111:0/3272964931 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c44007480 con 0x7f0c5c0cff50 2023-03-29T10:58:03.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.352+0000 7f0c617ff700 1 -- 172.21.15.111:0/3272964931 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5c0cf560 con 0x7f0c5c0cff50 2023-03-29T10:58:03.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.353+0000 7f0c617ff700 1 -- 172.21.15.111:0/3272964931 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5c15c960 con 0x7f0c5c0cff50 2023-03-29T10:58:03.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.354+0000 7f0c537fe700 1 -- 172.21.15.111:0/3272964931 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f0c440075e0 con 0x7f0c5c0cff50 2023-03-29T10:58:03.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.354+0000 7f0c537fe700 1 --2- 172.21.15.111:0/3272964931 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0c3c069de0 0x7f0c3c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:03.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.354+0000 7f0c537fe700 1 -- 172.21.15.111:0/3272964931 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0c440878e0 con 0x7f0c5c0cff50 2023-03-29T10:58:03.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.355+0000 7f0c5affd700 1 --2- 172.21.15.111:0/3272964931 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0c3c069de0 0x7f0c3c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:03.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.355+0000 7f0c5affd700 1 --2- 172.21.15.111:0/3272964931 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0c3c069de0 0x7f0c3c06c290 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009fd0 tx=0x7f0c4c01c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:03.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.359+0000 7f0c537fe700 1 -- 172.21.15.111:0/3272964931 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f0c44052eb0 con 0x7f0c5c0cff50 2023-03-29T10:58:03.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.553+0000 7f0c617ff700 1 -- 172.21.15.111:0/3272964931 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0c5c0b63f0 con 0x7f0c5c0cff50 2023-03-29T10:58:03.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.554+0000 7f0c537fe700 1 -- 172.21.15.111:0/3272964931 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v28) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0c440564d0 con 0x7f0c5c0cff50 2023-03-29T10:58:03.554 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:03.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.556+0000 7f0c617ff700 1 -- 172.21.15.111:0/3272964931 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0c3c069de0 msgr2=0x7f0c3c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:03.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.556+0000 7f0c617ff700 1 --2- 172.21.15.111:0/3272964931 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0c3c069de0 0x7f0c3c06c290 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c009fd0 tx=0x7f0c4c01c040 comp rx=0 tx=0).stop 2023-03-29T10:58:03.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.556+0000 7f0c617ff700 1 -- 172.21.15.111:0/3272964931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c5c0cff50 msgr2=0x7f0c5c1f56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:03.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.556+0000 7f0c617ff700 1 --2- 172.21.15.111:0/3272964931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c5c0cff50 0x7f0c5c1f56a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400c700 tx=0x7f0c4400cac0 comp rx=0 tx=0).stop 2023-03-29T10:58:03.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.556+0000 7f0c617ff700 1 -- 172.21.15.111:0/3272964931 shutdown_connections 2023-03-29T10:58:03.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.556+0000 7f0c617ff700 1 --2- 172.21.15.111:0/3272964931 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0c3c069de0 0x7f0c3c06c290 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:03.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.556+0000 7f0c617ff700 1 --2- 172.21.15.111:0/3272964931 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0c5c15c590 0x7f0c5c15ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:03.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.556+0000 7f0c617ff700 1 --2- 172.21.15.111:0/3272964931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c5c0cf010 0x7f0c5c154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:03.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.556+0000 7f0c617ff700 1 --2- 172.21.15.111:0/3272964931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0c5c0cff50 0x7f0c5c1f56a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:03.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.556+0000 7f0c617ff700 1 -- 172.21.15.111:0/3272964931 >> 172.21.15.111:0/3272964931 conn(0x7f0c5c071b10 msgr2=0x7f0c5c072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:03.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.556+0000 7f0c617ff700 1 -- 172.21.15.111:0/3272964931 shutdown_connections 2023-03-29T10:58:03.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.556+0000 7f0c617ff700 1 -- 172.21.15.111:0/3272964931 wait complete. 2023-03-29T10:58:03.573 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-29T10:58:03.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.757+0000 7f8c85dff700 1 Processor -- start 2023-03-29T10:58:03.757 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.757+0000 7f8c85dff700 1 -- start start 2023-03-29T10:58:03.757 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.758+0000 7f8c85dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c800cf970 0x7f8c800cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:03.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.758+0000 7f8c85dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8c800cfd40 0x7f8c800d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:03.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.758+0000 7f8c85dff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c800cdc10 con 0x7f8c80160290 2023-03-29T10:58:03.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.758+0000 7f8c85dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c800b6c60 con 0x7f8c800cfd40 2023-03-29T10:58:03.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.758+0000 7f8c85dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c800cd650 con 0x7f8c800cf970 2023-03-29T10:58:03.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.758+0000 7f8c7ffff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f8c80160290 0x7f8c801606d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37022/0 (socket says 172.21.15.111:37022) 2023-03-29T10:58:03.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.758+0000 7f8c7ffff700 1 -- 172.21.15.111:0/625258881 learned_addr learned my addr 172.21.15.111:0/625258881 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:03.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.758+0000 7f8c7effd700 1 --2- 172.21.15.111:0/625258881 >> v2:172.21.15.119:3300/0 conn(0x7f8c800cfd40 0x7f8c800d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:03.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.758+0000 7f8c7f7fe700 1 --2- 172.21.15.111:0/625258881 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c800cf970 0x7f8c800cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:03.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.759+0000 7f8c7e7fc700 1 -- 172.21.15.111:0/625258881 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 959809779 0 0) 0x7f8c800cdc10 con 0x7f8c80160290 2023-03-29T10:58:03.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.759+0000 7f8c7e7fc700 1 -- 172.21.15.111:0/625258881 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c680035b0 con 0x7f8c80160290 2023-03-29T10:58:03.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.759+0000 7f8c7effd700 1 -- 172.21.15.111:0/625258881 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c800cf970 msgr2=0x7f8c800cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:03.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.759+0000 7f8c7effd700 1 --2- 172.21.15.111:0/625258881 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c800cf970 0x7f8c800cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:03.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.759+0000 7f8c7effd700 1 -- 172.21.15.111:0/625258881 >> v1:172.21.15.111:6789/0 conn(0x7f8c80160290 legacy=0x7f8c801606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:03.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.759+0000 7f8c7effd700 1 -- 172.21.15.111:0/625258881 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c800cd930 con 0x7f8c800cfd40 2023-03-29T10:58:03.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.759+0000 7f8c7f7fe700 1 --2- 172.21.15.111:0/625258881 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c800cf970 0x7f8c800cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:58:03.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.759+0000 7f8c7effd700 1 --2- 172.21.15.111:0/625258881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c800cfd40 0x7f8c800d01b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009840 tx=0x7f8c74009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9a751a3092771dd server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:03.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.759+0000 7f8c7e7fc700 1 -- 172.21.15.111:0/625258881 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74011830 con 0x7f8c800cfd40 2023-03-29T10:58:03.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.759+0000 7f8c7e7fc700 1 -- 172.21.15.111:0/625258881 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c74011990 con 0x7f8c800cfd40 2023-03-29T10:58:03.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.759+0000 7f8c7e7fc700 1 -- 172.21.15.111:0/625258881 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74011d00 con 0x7f8c800cfd40 2023-03-29T10:58:03.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.760+0000 7f8c85dff700 1 -- 172.21.15.111:0/625258881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c800cfd40 msgr2=0x7f8c800d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:03.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.760+0000 7f8c85dff700 1 --2- 172.21.15.111:0/625258881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c800cfd40 0x7f8c800d01b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009840 tx=0x7f8c74009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:03.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.760+0000 7f8c85dff700 1 -- 172.21.15.111:0/625258881 shutdown_connections 2023-03-29T10:58:03.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.760+0000 7f8c85dff700 1 --2- 172.21.15.111:0/625258881 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c800cf970 0x7f8c800cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:03.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.760+0000 7f8c85dff700 1 --2- 172.21.15.111:0/625258881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c800cfd40 0x7f8c800d01b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:03.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.760+0000 7f8c85dff700 1 -- 172.21.15.111:0/625258881 >> 172.21.15.111:0/625258881 conn(0x7f8c80071b10 msgr2=0x7f8c80071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:03.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.760+0000 7f8c85dff700 1 -- 172.21.15.111:0/625258881 shutdown_connections 2023-03-29T10:58:03.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.760+0000 7f8c85dff700 1 -- 172.21.15.111:0/625258881 wait complete. 2023-03-29T10:58:03.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.762+0000 7f8c85dff700 1 Processor -- start 2023-03-29T10:58:03.762 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.762+0000 7f8c85dff700 1 -- start start 2023-03-29T10:58:03.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.764+0000 7f8c85dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c800cf970 0x7f8c800ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:03.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.764+0000 7f8c85dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c800cfd40 0x7f8c800c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:03.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.764+0000 7f8c85dff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c80160290 0x7f8c801f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:03.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.764+0000 7f8c85dff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c801f5430 con 0x7f8c80160290 2023-03-29T10:58:03.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.764+0000 7f8c85dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c800cac20 con 0x7f8c800cfd40 2023-03-29T10:58:03.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.764+0000 7f8c85dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c800caf00 con 0x7f8c800cf970 2023-03-29T10:58:03.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.764+0000 7f8c7ffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c80160290 0x7f8c801f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:03.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.764+0000 7f8c7f7fe700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c800cf970 0x7f8c800ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:03.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.764+0000 7f8c7f7fe700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c800cf970 0x7f8c800ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:38890/0 (socket says 172.21.15.111:38890) 2023-03-29T10:58:03.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.764+0000 7f8c7f7fe700 1 -- 172.21.15.111:0/1502549229 learned_addr learned my addr 172.21.15.111:0/1502549229 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:03.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.764+0000 7f8c7effd700 1 --2- 172.21.15.111:0/1502549229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c800cfd40 0x7f8c800c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:03.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.764+0000 7f8c7ffff700 1 -- 172.21.15.111:0/1502549229 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c800cf970 msgr2=0x7f8c800ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:03.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.764+0000 7f8c7ffff700 1 --2- 172.21.15.111:0/1502549229 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c800cf970 0x7f8c800ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:03.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.764+0000 7f8c7ffff700 1 -- 172.21.15.111:0/1502549229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c800cfd40 msgr2=0x7f8c800c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:03.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.764+0000 7f8c7ffff700 1 --2- 172.21.15.111:0/1502549229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c800cfd40 0x7f8c800c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:03.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.764+0000 7f8c7ffff700 1 -- 172.21.15.111:0/1502549229 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c700094a0 con 0x7f8c80160290 2023-03-29T10:58:03.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.764+0000 7f8c7f7fe700 1 --2- 172.21.15.111:0/1502549229 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c800cf970 0x7f8c800ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:58:03.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.765+0000 7f8c7ffff700 1 --2- 172.21.15.111:0/1502549229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c80160290 0x7f8c801f1d80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800caa0 tx=0x7f8c7800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:03.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.765+0000 7f8c67fff700 1 -- 172.21.15.111:0/1502549229 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800a9e0 con 0x7f8c80160290 2023-03-29T10:58:03.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.765+0000 7f8c67fff700 1 -- 172.21.15.111:0/1502549229 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7800ab40 con 0x7f8c80160290 2023-03-29T10:58:03.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.765+0000 7f8c67fff700 1 -- 172.21.15.111:0/1502549229 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78012400 con 0x7f8c80160290 2023-03-29T10:58:03.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.765+0000 7f8c85dff700 1 -- 172.21.15.111:0/1502549229 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c740094a0 con 0x7f8c80160290 2023-03-29T10:58:03.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.765+0000 7f8c85dff700 1 -- 172.21.15.111:0/1502549229 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c800cf650 con 0x7f8c80160290 2023-03-29T10:58:03.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.766+0000 7f8c85dff700 1 -- 172.21.15.111:0/1502549229 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c800b68f0 con 0x7f8c80160290 2023-03-29T10:58:03.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.767+0000 7f8c67fff700 1 -- 172.21.15.111:0/1502549229 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f8c7801b3b0 con 0x7f8c80160290 2023-03-29T10:58:03.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.768+0000 7f8c67fff700 1 --2- 172.21.15.111:0/1502549229 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8c6c069de0 0x7f8c6c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:03.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.768+0000 7f8c67fff700 1 -- 172.21.15.111:0/1502549229 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8c78087cd0 con 0x7f8c80160290 2023-03-29T10:58:03.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.768+0000 7f8c7f7fe700 1 --2- 172.21.15.111:0/1502549229 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8c6c069de0 0x7f8c6c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:03.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.769+0000 7f8c7f7fe700 1 --2- 172.21.15.111:0/1502549229 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8c6c069de0 0x7f8c6c06c290 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009fd0 tx=0x7f8c70009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:03.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:03.772+0000 7f8c67fff700 1 -- 172.21.15.111:0/1502549229 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f8c7804c160 con 0x7f8c80160290 2023-03-29T10:58:04.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.013+0000 7f8c85dff700 1 -- 172.21.15.111:0/1502549229 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c800d06f0 con 0x7f8c80160290 2023-03-29T10:58:04.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.014+0000 7f8c67fff700 1 -- 172.21.15.111:0/1502549229 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7f8c780532e0 con 0x7f8c80160290 2023-03-29T10:58:04.014 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:04.014 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/556826332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":556826332},{"type":"v1","addr":"172.21.15.111:6837","nonce":556826332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/814623168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":814623168},{"type":"v1","addr":"172.21.15.111:6835","nonce":814623168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/514422035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":514422035},{"type":"v1","addr":"172.21.15.119:6834","nonce":514422035}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-29T10:57:27.824135+0000","modified":"2023-03-29T10:57:53.367898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/3160800581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":3160800581},{"type":"v1","addr":"172.21.15.119:6835","nonce":3160800581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-03-29T10:58:04.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.016+0000 7f8c85dff700 1 -- 172.21.15.111:0/1502549229 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8c6c069de0 msgr2=0x7f8c6c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:04.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.016+0000 7f8c85dff700 1 --2- 172.21.15.111:0/1502549229 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8c6c069de0 0x7f8c6c06c290 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009fd0 tx=0x7f8c70009210 comp rx=0 tx=0).stop 2023-03-29T10:58:04.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.016+0000 7f8c85dff700 1 -- 172.21.15.111:0/1502549229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c80160290 msgr2=0x7f8c801f1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:04.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.016+0000 7f8c85dff700 1 --2- 172.21.15.111:0/1502549229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c80160290 0x7f8c801f1d80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800caa0 tx=0x7f8c7800cdb0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.016+0000 7f8c85dff700 1 -- 172.21.15.111:0/1502549229 shutdown_connections 2023-03-29T10:58:04.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.016+0000 7f8c85dff700 1 --2- 172.21.15.111:0/1502549229 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8c6c069de0 0x7f8c6c06c290 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.016+0000 7f8c85dff700 1 --2- 172.21.15.111:0/1502549229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8c80160290 0x7f8c801f1d80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.016+0000 7f8c85dff700 1 --2- 172.21.15.111:0/1502549229 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8c800cf970 0x7f8c800ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.016+0000 7f8c85dff700 1 --2- 172.21.15.111:0/1502549229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c800cfd40 0x7f8c800c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.016+0000 7f8c85dff700 1 -- 172.21.15.111:0/1502549229 >> 172.21.15.111:0/1502549229 conn(0x7f8c80071b10 msgr2=0x7f8c800d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:04.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.016+0000 7f8c85dff700 1 -- 172.21.15.111:0/1502549229 shutdown_connections 2023-03-29T10:58:04.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.016+0000 7f8c85dff700 1 -- 172.21.15.111:0/1502549229 wait complete. 2023-03-29T10:58:04.017 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2023-03-29T10:58:04.435 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-03-29T10:57:27.824135+0000', 'modified': '2023-03-29T10:57:53.367898+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4296}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4296': {'gid': 4296, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6835/3160800581', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6833', 'nonce': 3160800581}, {'type': 'v1', 'addr': '172.21.15.119:6835', 'nonce': 3160800581}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-03-29T10:58:04.435 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-03-29T10:58:04.435 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-03-29T10:58:04.621 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.621+0000 7f64d3dff700 1 Processor -- start 2023-03-29T10:58:04.621 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.621+0000 7f64d3dff700 1 -- start start 2023-03-29T10:58:04.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.622+0000 7f64d3dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64cc0b87a0 0x7f64cc0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.622+0000 7f64d3dff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f64cc160d40 0x7f64cc0d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.622+0000 7f64d3dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64cc0cdc60 con 0x7f64cc1601e0 2023-03-29T10:58:04.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.622+0000 7f64d3dff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64cc0d02d0 con 0x7f64cc160d40 2023-03-29T10:58:04.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.622+0000 7f64d3dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc0d0cb0 con 0x7f64cc0b87a0 2023-03-29T10:58:04.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.623+0000 7f64d239c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f64cc160d40 0x7f64cc0d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:04.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.623+0000 7f64d239c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f64cc160d40 0x7f64cc0d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50468/0 (socket says 172.21.15.111:50468) 2023-03-29T10:58:04.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.623+0000 7f64d239c700 1 -- 172.21.15.111:0/572228479 learned_addr learned my addr 172.21.15.111:0/572228479 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:04.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.623+0000 7f64d1b9b700 1 --2- 172.21.15.111:0/572228479 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64cc0b87a0 0x7f64cc0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:04.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.623+0000 7f64d239c700 1 -- 172.21.15.111:0/572228479 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64cc0b87a0 msgr2=0x7f64cc0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:04.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.623+0000 7f64d239c700 1 --2- 172.21.15.111:0/572228479 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64cc0b87a0 0x7f64cc0cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.623+0000 7f64d239c700 1 -- 172.21.15.111:0/572228479 >> v1:172.21.15.119:6789/0 conn(0x7f64cc1601e0 legacy=0x7f64cc160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:04.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.623+0000 7f64d0b99700 1 -- 172.21.15.111:0/572228479 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735065260 0 0) 0x7f64cc0cdc60 con 0x7f64cc1601e0 2023-03-29T10:58:04.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.623+0000 7f64d239c700 1 -- 172.21.15.111:0/572228479 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc0cd6a0 con 0x7f64cc160d40 2023-03-29T10:58:04.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.624+0000 7f64d239c700 1 --2- 172.21.15.111:0/572228479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64cc160d40 0x7f64cc0d1100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009e50 tx=0x7f64c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44278d5e70c78553 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:04.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.624+0000 7f64d0b99700 1 -- 172.21.15.111:0/572228479 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c400e070 con 0x7f64cc160d40 2023-03-29T10:58:04.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.624+0000 7f64d0b99700 1 -- 172.21.15.111:0/572228479 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c400b910 con 0x7f64cc160d40 2023-03-29T10:58:04.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.624+0000 7f64d0b99700 1 -- 172.21.15.111:0/572228479 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c400bcc0 con 0x7f64cc160d40 2023-03-29T10:58:04.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.624+0000 7f64d3dff700 1 -- 172.21.15.111:0/572228479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64cc160d40 msgr2=0x7f64cc0d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:04.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.624+0000 7f64d3dff700 1 --2- 172.21.15.111:0/572228479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64cc160d40 0x7f64cc0d1100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009e50 tx=0x7f64c400b330 comp rx=0 tx=0).stop 2023-03-29T10:58:04.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.624+0000 7f64d3dff700 1 -- 172.21.15.111:0/572228479 shutdown_connections 2023-03-29T10:58:04.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.624+0000 7f64d3dff700 1 --2- 172.21.15.111:0/572228479 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64cc160d40 0x7f64cc0d1100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.624+0000 7f64d3dff700 1 --2- 172.21.15.111:0/572228479 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64cc0b87a0 0x7f64cc0cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.624+0000 7f64d3dff700 1 -- 172.21.15.111:0/572228479 >> 172.21.15.111:0/572228479 conn(0x7f64cc071b10 msgr2=0x7f64cc071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:04.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.624+0000 7f64d3dff700 1 -- 172.21.15.111:0/572228479 shutdown_connections 2023-03-29T10:58:04.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.625+0000 7f64d3dff700 1 -- 172.21.15.111:0/572228479 wait complete. 2023-03-29T10:58:04.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.626+0000 7f64d3dff700 1 Processor -- start 2023-03-29T10:58:04.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.626+0000 7f64d3dff700 1 -- start start 2023-03-29T10:58:04.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.627+0000 7f64d3dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64cc0b87a0 0x7f64cc154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.627+0000 7f64d3dff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64cc1601e0 0x7f64cc0b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.627+0000 7f64d3dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64cc160d40 0x7f64cc0b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.627+0000 7f64d3dff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc15dba0 con 0x7f64cc1601e0 2023-03-29T10:58:04.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.627+0000 7f64d3dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc1549a0 con 0x7f64cc0b87a0 2023-03-29T10:58:04.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.627+0000 7f64d3dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc15eb30 con 0x7f64cc160d40 2023-03-29T10:58:04.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.627+0000 7f64d139a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64cc1601e0 0x7f64cc0b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:04.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.627+0000 7f64d139a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64cc1601e0 0x7f64cc0b5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50484/0 (socket says 172.21.15.111:50484) 2023-03-29T10:58:04.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.628+0000 7f64d139a700 1 -- 172.21.15.111:0/1048752595 learned_addr learned my addr 172.21.15.111:0/1048752595 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:04.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.628+0000 7f64d1b9b700 1 --2- 172.21.15.111:0/1048752595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64cc0b87a0 0x7f64cc154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:04.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.628+0000 7f64d239c700 1 --2- 172.21.15.111:0/1048752595 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64cc160d40 0x7f64cc0b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:04.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.628+0000 7f64d139a700 1 -- 172.21.15.111:0/1048752595 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64cc160d40 msgr2=0x7f64cc0b6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:04.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.628+0000 7f64d139a700 1 --2- 172.21.15.111:0/1048752595 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64cc160d40 0x7f64cc0b6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.628+0000 7f64d139a700 1 -- 172.21.15.111:0/1048752595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64cc0b87a0 msgr2=0x7f64cc154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:04.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.628+0000 7f64d139a700 1 --2- 172.21.15.111:0/1048752595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64cc0b87a0 0x7f64cc154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.628+0000 7f64d139a700 1 -- 172.21.15.111:0/1048752595 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c4009b30 con 0x7f64cc1601e0 2023-03-29T10:58:04.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.628+0000 7f64d1b9b700 1 --2- 172.21.15.111:0/1048752595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64cc0b87a0 0x7f64cc154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:04.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.628+0000 7f64d239c700 1 --2- 172.21.15.111:0/1048752595 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64cc160d40 0x7f64cc0b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:04.628 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.628+0000 7f64d139a700 1 --2- 172.21.15.111:0/1048752595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64cc1601e0 0x7f64cc0b5ce0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f64bc0011e0 tx=0x7f64bc00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:04.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.628+0000 7f64c27fc700 1 -- 172.21.15.111:0/1048752595 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc0054e0 con 0x7f64cc1601e0 2023-03-29T10:58:04.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.628+0000 7f64c27fc700 1 -- 172.21.15.111:0/1048752595 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64bc003680 con 0x7f64cc1601e0 2023-03-29T10:58:04.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.628+0000 7f64d3dff700 1 -- 172.21.15.111:0/1048752595 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64cc0d1640 con 0x7f64cc1601e0 2023-03-29T10:58:04.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.628+0000 7f64c27fc700 1 -- 172.21.15.111:0/1048752595 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc003a30 con 0x7f64cc1601e0 2023-03-29T10:58:04.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.628+0000 7f64d3dff700 1 -- 172.21.15.111:0/1048752595 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64cc0cd980 con 0x7f64cc1601e0 2023-03-29T10:58:04.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.631+0000 7f64c27fc700 1 -- 172.21.15.111:0/1048752595 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f64bc003b90 con 0x7f64cc1601e0 2023-03-29T10:58:04.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.631+0000 7f64c27fc700 1 --2- 172.21.15.111:0/1048752595 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f64b0069de0 0x7f64b006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.632+0000 7f64c27fc700 1 -- 172.21.15.111:0/1048752595 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f64bc087880 con 0x7f64cc1601e0 2023-03-29T10:58:04.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.632+0000 7f64d1b9b700 1 --2- 172.21.15.111:0/1048752595 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f64b0069de0 0x7f64b006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:04.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.632+0000 7f64d1b9b700 1 --2- 172.21.15.111:0/1048752595 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f64b0069de0 0x7f64b006c290 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0ca900 tx=0x7f64c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:04.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.633+0000 7f64e764ab80 1 Processor -- start 2023-03-29T10:58:04.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.633+0000 7f64e764ab80 1 -- start start 2023-03-29T10:58:04.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.634+0000 7f64e764ab80 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9af060 0x55f35f9af430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.634+0000 7f64e764ab80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9eec50 0x55f35f9d51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.634+0000 7f64e764ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f35f9b06e0 0x55f35f9d56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.634+0000 7f64e764ab80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35f930960 con 0x55f35f9eec50 2023-03-29T10:58:04.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.634+0000 7f64e764ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35f8ba120 con 0x55f35f9b06e0 2023-03-29T10:58:04.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.634+0000 7f64e764ab80 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f35f90c690 con 0x55f35f9af060 2023-03-29T10:58:04.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.635+0000 7f64d139a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9eec50 0x55f35f9d51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:04.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.635+0000 7f64d139a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9eec50 0x55f35f9d51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50488/0 (socket says 172.21.15.111:50488) 2023-03-29T10:58:04.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.635+0000 7f64d139a700 1 -- 172.21.15.111:0/1322951607 learned_addr learned my addr 172.21.15.111:0/1322951607 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:04.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.635+0000 7f64d139a700 1 -- 172.21.15.111:0/1322951607 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9af060 msgr2=0x55f35f9af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:04.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.635+0000 7f64d139a700 1 --2- 172.21.15.111:0/1322951607 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9af060 0x55f35f9af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.635+0000 7f64d139a700 1 -- 172.21.15.111:0/1322951607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f35f9b06e0 msgr2=0x55f35f9d56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:04.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.635+0000 7f64d139a700 1 --2- 172.21.15.111:0/1322951607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f35f9b06e0 0x55f35f9d56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.635+0000 7f64d139a700 1 -- 172.21.15.111:0/1322951607 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f35f6f9540 con 0x55f35f9eec50 2023-03-29T10:58:04.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.635+0000 7f64d139a700 1 --2- 172.21.15.111:0/1322951607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9eec50 0x55f35f9d51a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f64bc085800 tx=0x7f64bc0566d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c650d7c6b1cfa44 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:04.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.635+0000 7f64af7fe700 1 -- 172.21.15.111:0/1322951607 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc085a90 con 0x55f35f9eec50 2023-03-29T10:58:04.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.635+0000 7f64af7fe700 1 -- 172.21.15.111:0/1322951607 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64bc054e00 con 0x55f35f9eec50 2023-03-29T10:58:04.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.635+0000 7f64af7fe700 1 -- 172.21.15.111:0/1322951607 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc056170 con 0x55f35f9eec50 2023-03-29T10:58:04.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.636+0000 7f64e764ab80 1 -- 172.21.15.111:0/1322951607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9eec50 msgr2=0x55f35f9d51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:04.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.636+0000 7f64e764ab80 1 --2- 172.21.15.111:0/1322951607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9eec50 0x55f35f9d51a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f64bc085800 tx=0x7f64bc0566d0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.636+0000 7f64e764ab80 1 -- 172.21.15.111:0/1322951607 shutdown_connections 2023-03-29T10:58:04.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.636+0000 7f64e764ab80 1 --2- 172.21.15.111:0/1322951607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f35f9b06e0 0x55f35f9d56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.636+0000 7f64e764ab80 1 --2- 172.21.15.111:0/1322951607 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9af060 0x55f35f9af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.636+0000 7f64e764ab80 1 --2- 172.21.15.111:0/1322951607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9eec50 0x55f35f9d51a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.636+0000 7f64e764ab80 1 -- 172.21.15.111:0/1322951607 >> 172.21.15.111:0/1322951607 conn(0x55f35f9a7ac0 msgr2=0x55f35f9ad030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:04.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.636+0000 7f64e764ab80 1 -- 172.21.15.111:0/1322951607 shutdown_connections 2023-03-29T10:58:04.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.636+0000 7f64e764ab80 1 -- 172.21.15.111:0/1322951607 wait complete. 2023-03-29T10:58:04.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.637+0000 7f64e764ab80 1 -- start start 2023-03-29T10:58:04.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.637+0000 7f64e764ab80 1 Processor -- start 2023-03-29T10:58:04.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.638+0000 7f64e764ab80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9af060 0x55f35f9d8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.638+0000 7f64e764ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f35f9b06e0 0x55f35f9f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.638+0000 7f64e764ab80 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9eec50 0x55f35f9fa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.638+0000 7f64e764ab80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35f905960 con 0x55f35f9af060 2023-03-29T10:58:04.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.638+0000 7f64e764ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35f6fa140 con 0x55f35f9b06e0 2023-03-29T10:58:04.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.638+0000 7f64e764ab80 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f35f9f9750 con 0x55f35f9eec50 2023-03-29T10:58:04.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.638+0000 7f64e764ab80 10 client.0 _ll_register_callbacks cb 0x55f35f8a31c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-03-29T10:58:04.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.638+0000 7f64d1b9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9af060 0x55f35f9d8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:04.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.638+0000 7f64d1b9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9af060 0x55f35f9d8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50492/0 (socket says 172.21.15.111:50492) 2023-03-29T10:58:04.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.638+0000 7f64d1b9b700 1 -- 172.21.15.111:0/1598718545 learned_addr learned my addr 172.21.15.111:0/1598718545 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:04.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.638+0000 7f64d139a700 1 --2- 172.21.15.111:0/1598718545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f35f9b06e0 0x55f35f9f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:04.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.638+0000 7f64d239c700 1 --2- 172.21.15.111:0/1598718545 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9eec50 0x55f35f9fa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:04.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.639+0000 7f64d1b9b700 1 -- 172.21.15.111:0/1598718545 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9eec50 msgr2=0x55f35f9fa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:04.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.639+0000 7f64d1b9b700 1 --2- 172.21.15.111:0/1598718545 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9eec50 0x55f35f9fa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.639+0000 7f64d1b9b700 1 -- 172.21.15.111:0/1598718545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f35f9b06e0 msgr2=0x55f35f9f8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:04.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.639+0000 7f64d1b9b700 1 --2- 172.21.15.111:0/1598718545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f35f9b06e0 0x55f35f9f8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.639+0000 7f64d1b9b700 1 -- 172.21.15.111:0/1598718545 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64cc0cd980 con 0x55f35f9af060 2023-03-29T10:58:04.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.639+0000 7f64d239c700 1 --2- 172.21.15.111:0/1598718545 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9eec50 0x55f35f9fa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:04.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.639+0000 7f64d1b9b700 1 --2- 172.21.15.111:0/1598718545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9af060 0x55f35f9d8f60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f64c80156c0 tx=0x7f64c8009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b80fb2d93d19393 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:04.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.639+0000 7f64acff9700 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-03-29T10:58:04.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.639+0000 7f64acff9700 1 -- 172.21.15.111:0/1598718545 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8025070 con 0x55f35f9af060 2023-03-29T10:58:04.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.639+0000 7f64e764ab80 1 -- 172.21.15.111:0/1598718545 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f35f9a50d0 con 0x55f35f9af060 2023-03-29T10:58:04.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.639+0000 7f64acff9700 1 -- 172.21.15.111:0/1598718545 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f64c8011e40 con 0x55f35f9af060 2023-03-29T10:58:04.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.639+0000 7f64acff9700 1 -- 172.21.15.111:0/1598718545 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64c8004020 con 0x55f35f9af060 2023-03-29T10:58:04.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.639+0000 7f64e764ab80 10 client.4818 fetch_fsmap learned FSMap version 9 2023-03-29T10:58:04.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.639+0000 7f64e764ab80 1 -- 172.21.15.111:0/1598718545 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55f35f733b10 con 0x55f35f9af060 2023-03-29T10:58:04.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.639+0000 7f64acff9700 1 -- 172.21.15.111:0/1598718545 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f35f9a50d0 con 0x55f35f9af060 2023-03-29T10:58:04.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.640+0000 7f64acff9700 1 -- 172.21.15.111:0/1598718545 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f64c8028020 con 0x55f35f9af060 2023-03-29T10:58:04.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.640+0000 7f64e764ab80 10 client.4818 fetch_fsmap finished waiting for FSMap version 9 2023-03-29T10:58:04.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.640+0000 7f64e764ab80 10 client.4818 resolve_mds: resolved name 'd' to daemon mds.d 2023-03-29T10:58:04.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.640+0000 7f64e764ab80 20 client.4818 populate_metadata read hostname 'smithi111' 2023-03-29T10:58:04.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.640+0000 7f64e764ab80 1 --2- 172.21.15.111:0/1598718545 >> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] conn(0x55f35f9fa7f0 0x55f35f9f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.640+0000 7f64e764ab80 4 client.4818 mds_command: new command op to 4296 tid=0[{"prefix": "get_command_descriptions"}] 2023-03-29T10:58:04.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.640+0000 7f64e764ab80 1 -- 172.21.15.111:0/1598718545 --> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f35f9824c0 con 0x55f35f9fa7f0 2023-03-29T10:58:04.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.640+0000 7f64d139a700 1 --2- 172.21.15.111:0/1598718545 >> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] conn(0x55f35f9fa7f0 0x55f35f9f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:04.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.641+0000 7f64acff9700 10 client.4818 ms_handle_connect on v2:172.21.15.119:6833/3160800581 2023-03-29T10:58:04.641 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.641+0000 7f64d139a700 1 --2- 172.21.15.111:0/1598718545 >> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] conn(0x55f35f9fa7f0 0x55f35f9f6eb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4002eb06ff1d8ba2 server_cookie=fbd2a73a167b1adb in_seq=0 out_seq=0 2023-03-29T10:58:04.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.641+0000 7f64acff9700 1 -- 172.21.15.111:0/1598718545 <== mds.0 v2:172.21.15.119:6833/3160800581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18278 (crc 0 0 0) 0x55f35f9824c0 con 0x55f35f9fa7f0 2023-03-29T10:58:04.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.641+0000 7f64acff9700 10 client.4818 handle_command_reply: tid=0 2023-03-29T10:58:04.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.642+0000 7f64e764ab80 1 client.4818 shutdown 2023-03-29T10:58:04.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.642+0000 7f64e764ab80 1 -- 172.21.15.111:0/1598718545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9af060 msgr2=0x55f35f9d8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:04.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.642+0000 7f64e764ab80 1 --2- 172.21.15.111:0/1598718545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9af060 0x55f35f9d8f60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f64c80156c0 tx=0x7f64c8009210 comp rx=0 tx=0).stop 2023-03-29T10:58:04.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.642+0000 7f64e764ab80 1 -- 172.21.15.111:0/1598718545 shutdown_connections 2023-03-29T10:58:04.642 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.642+0000 7f64e764ab80 1 --2- 172.21.15.111:0/1598718545 >> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] conn(0x55f35f9fa7f0 0x55f35f9f6eb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.642+0000 7f64e764ab80 1 --2- 172.21.15.111:0/1598718545 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9af060 0x55f35f9d8f60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.642+0000 7f64e764ab80 1 --2- 172.21.15.111:0/1598718545 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9eec50 0x55f35f9fa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.642+0000 7f64e764ab80 1 --2- 172.21.15.111:0/1598718545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f35f9b06e0 0x55f35f9f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.642+0000 7f64acff9700 0 client.4818 ms_handle_reset on v2:172.21.15.119:6833/3160800581 2023-03-29T10:58:04.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.642+0000 7f64e764ab80 1 -- 172.21.15.111:0/1598718545 >> 172.21.15.111:0/1598718545 conn(0x55f35f9a7ac0 msgr2=0x55f35f9b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:04.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.642+0000 7f64e764ab80 1 -- 172.21.15.111:0/1598718545 shutdown_connections 2023-03-29T10:58:04.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.642+0000 7f64e764ab80 1 -- 172.21.15.111:0/1598718545 wait complete. 2023-03-29T10:58:04.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.642+0000 7f64e764ab80 20 client.4818 trim_cache size 0 max 16384 2023-03-29T10:58:04.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.660+0000 7f64e764ab80 1 Processor -- start 2023-03-29T10:58:04.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.660+0000 7f64e764ab80 1 -- start start 2023-03-29T10:58:04.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.661+0000 7f64e764ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f35f9a7ac0 0x55f35f9d8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.661+0000 7f64e764ab80 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9af060 0x55f35f9ad000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.661+0000 7f64e764ab80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9b06e0 0x55f35f9adef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.661+0000 7f64e764ab80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35f9d98d0 con 0x55f35f9b06e0 2023-03-29T10:58:04.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.661+0000 7f64e764ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35fa08510 con 0x55f35f9a7ac0 2023-03-29T10:58:04.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.661+0000 7f64e764ab80 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f35f9ad700 con 0x55f35f9af060 2023-03-29T10:58:04.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.661+0000 7f64d239c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9b06e0 0x55f35f9adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:04.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.661+0000 7f64d239c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9b06e0 0x55f35f9adef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50508/0 (socket says 172.21.15.111:50508) 2023-03-29T10:58:04.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.661+0000 7f64d239c700 1 -- 172.21.15.111:0/3777022439 learned_addr learned my addr 172.21.15.111:0/3777022439 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:04.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.661+0000 7f64d139a700 1 --2- 172.21.15.111:0/3777022439 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9af060 0x55f35f9ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:04.661 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.661+0000 7f64d1b9b700 1 --2- 172.21.15.111:0/3777022439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f35f9a7ac0 0x55f35f9d8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:04.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.661+0000 7f64d239c700 1 -- 172.21.15.111:0/3777022439 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9af060 msgr2=0x55f35f9ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:04.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.661+0000 7f64d239c700 1 --2- 172.21.15.111:0/3777022439 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9af060 0x55f35f9ad000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.661+0000 7f64d239c700 1 -- 172.21.15.111:0/3777022439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f35f9a7ac0 msgr2=0x55f35f9d8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:04.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.661+0000 7f64d239c700 1 --2- 172.21.15.111:0/3777022439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f35f9a7ac0 0x55f35f9d8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.662+0000 7f64d239c700 1 -- 172.21.15.111:0/3777022439 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f35f8cc230 con 0x55f35f9b06e0 2023-03-29T10:58:04.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.662+0000 7f64d239c700 1 --2- 172.21.15.111:0/3777022439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9b06e0 0x55f35f9adef0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f64cc072760 tx=0x7f64c400e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b212269ac324940 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:04.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.662+0000 7f64affff700 1 -- 172.21.15.111:0/3777022439 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c400bc80 con 0x55f35f9b06e0 2023-03-29T10:58:04.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.662+0000 7f64affff700 1 -- 172.21.15.111:0/3777022439 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c40099b0 con 0x55f35f9b06e0 2023-03-29T10:58:04.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.662+0000 7f64affff700 1 -- 172.21.15.111:0/3777022439 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4003c60 con 0x55f35f9b06e0 2023-03-29T10:58:04.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.662+0000 7f64e764ab80 1 -- 172.21.15.111:0/3777022439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9b06e0 msgr2=0x55f35f9adef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:04.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.662+0000 7f64e764ab80 1 --2- 172.21.15.111:0/3777022439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9b06e0 0x55f35f9adef0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f64cc072760 tx=0x7f64c400e000 comp rx=0 tx=0).stop 2023-03-29T10:58:04.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.663+0000 7f64e764ab80 1 -- 172.21.15.111:0/3777022439 shutdown_connections 2023-03-29T10:58:04.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.663+0000 7f64e764ab80 1 --2- 172.21.15.111:0/3777022439 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9b06e0 0x55f35f9adef0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.663+0000 7f64e764ab80 1 --2- 172.21.15.111:0/3777022439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f35f9a7ac0 0x55f35f9d8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.663+0000 7f64e764ab80 1 --2- 172.21.15.111:0/3777022439 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9af060 0x55f35f9ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.663+0000 7f64e764ab80 1 -- 172.21.15.111:0/3777022439 >> 172.21.15.111:0/3777022439 conn(0x55f35f9b2810 msgr2=0x55f35f9fa000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:04.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.663+0000 7f64e764ab80 1 -- 172.21.15.111:0/3777022439 shutdown_connections 2023-03-29T10:58:04.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.663+0000 7f64e764ab80 1 -- 172.21.15.111:0/3777022439 wait complete. 2023-03-29T10:58:04.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.663+0000 7f64e764ab80 1 -- start start 2023-03-29T10:58:04.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.663+0000 7f64e764ab80 1 Processor -- start 2023-03-29T10:58:04.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.664+0000 7f64e764ab80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f35f9a7ac0 0x55f35f9ac690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.664+0000 7f64e764ab80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9af060 0x55f35fa11340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.664+0000 7f64e764ab80 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9b06e0 0x55f35fa58790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.664+0000 7f64e764ab80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35f9edbf0 con 0x55f35f9af060 2023-03-29T10:58:04.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.664+0000 7f64e764ab80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f35f9d7b60 con 0x55f35f9a7ac0 2023-03-29T10:58:04.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.664+0000 7f64e764ab80 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f35f9fb660 con 0x55f35f9b06e0 2023-03-29T10:58:04.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.665+0000 7f64e764ab80 10 client.0 _ll_register_callbacks cb 0x55f35f8a31c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-03-29T10:58:04.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.665+0000 7f64d139a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9af060 0x55f35fa11340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:04.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.665+0000 7f64d1b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f35f9a7ac0 0x55f35f9ac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:04.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.665+0000 7f64d139a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9af060 0x55f35fa11340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50518/0 (socket says 172.21.15.111:50518) 2023-03-29T10:58:04.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.665+0000 7f64d139a700 1 -- 172.21.15.111:0/1907134857 learned_addr learned my addr 172.21.15.111:0/1907134857 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:04.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.665+0000 7f64d239c700 1 --2- 172.21.15.111:0/1907134857 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9b06e0 0x55f35fa58790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:04.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.665+0000 7f64d139a700 1 -- 172.21.15.111:0/1907134857 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9b06e0 msgr2=0x55f35fa58790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:04.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.665+0000 7f64d139a700 1 --2- 172.21.15.111:0/1907134857 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9b06e0 0x55f35fa58790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.665+0000 7f64d139a700 1 -- 172.21.15.111:0/1907134857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f35f9a7ac0 msgr2=0x55f35f9ac690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:04.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.665+0000 7f64d139a700 1 --2- 172.21.15.111:0/1907134857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f35f9a7ac0 0x55f35f9ac690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.665+0000 7f64d139a700 1 -- 172.21.15.111:0/1907134857 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c400be70 con 0x55f35f9af060 2023-03-29T10:58:04.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.665+0000 7f64d239c700 1 --2- 172.21.15.111:0/1907134857 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9b06e0 0x55f35fa58790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:04.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.665+0000 7f64d139a700 1 --2- 172.21.15.111:0/1907134857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9af060 0x55f35fa11340 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f64bc05ecb0 tx=0x7f64bc05c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691003d2109744b7 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:04.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.665+0000 7f64acff9700 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-03-29T10:58:04.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.665+0000 7f64acff9700 1 -- 172.21.15.111:0/1907134857 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc05e860 con 0x55f35f9af060 2023-03-29T10:58:04.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.666+0000 7f64acff9700 1 -- 172.21.15.111:0/1907134857 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f64bc085070 con 0x55f35f9af060 2023-03-29T10:58:04.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.666+0000 7f64e764ab80 1 -- 172.21.15.111:0/1907134857 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f35f8feaa0 con 0x55f35f9af060 2023-03-29T10:58:04.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.666+0000 7f64acff9700 1 -- 172.21.15.111:0/1907134857 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64bc056210 con 0x55f35f9af060 2023-03-29T10:58:04.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.666+0000 7f64e764ab80 10 client.4824 fetch_fsmap learned FSMap version 9 2023-03-29T10:58:04.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.666+0000 7f64e764ab80 1 -- 172.21.15.111:0/1907134857 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55f35f71c990 con 0x55f35f9af060 2023-03-29T10:58:04.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.666+0000 7f64acff9700 1 -- 172.21.15.111:0/1907134857 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f35f8feaa0 con 0x55f35f9af060 2023-03-29T10:58:04.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.666+0000 7f64acff9700 1 -- 172.21.15.111:0/1907134857 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 9) v1 ==== 3122+0+0 (secure 0 0 0) 0x7f64bc085230 con 0x55f35f9af060 2023-03-29T10:58:04.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.666+0000 7f64e764ab80 10 client.4824 fetch_fsmap finished waiting for FSMap version 9 2023-03-29T10:58:04.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.666+0000 7f64e764ab80 10 client.4824 resolve_mds: resolved name 'd' to daemon mds.d 2023-03-29T10:58:04.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.666+0000 7f64e764ab80 20 client.4824 populate_metadata read hostname 'smithi111' 2023-03-29T10:58:04.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.666+0000 7f64e764ab80 1 --2- 172.21.15.111:0/1907134857 >> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] conn(0x55f35f9eec50 0x55f35fa63d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:04.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.666+0000 7f64e764ab80 4 client.4824 mds_command: new command op to 4296 tid=0[{"prefix": "status"}] 2023-03-29T10:58:04.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.666+0000 7f64e764ab80 1 -- 172.21.15.111:0/1907134857 --> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f35f754ca0 con 0x55f35f9eec50 2023-03-29T10:58:04.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.666+0000 7f64d1b9b700 1 --2- 172.21.15.111:0/1907134857 >> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] conn(0x55f35f9eec50 0x55f35fa63d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:04.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.667+0000 7f64acff9700 10 client.4824 ms_handle_connect on v2:172.21.15.119:6833/3160800581 2023-03-29T10:58:04.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.667+0000 7f64d1b9b700 1 --2- 172.21.15.111:0/1907134857 >> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] conn(0x55f35f9eec50 0x55f35fa63d20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba90165629178510 server_cookie=bff16700037fd9b9 in_seq=0 out_seq=0 2023-03-29T10:58:04.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.667+0000 7f64acff9700 1 -- 172.21.15.111:0/1907134857 <== mds.0 v2:172.21.15.119:6833/3160800581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55f35f754ca0 con 0x55f35f9eec50 2023-03-29T10:58:04.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.667+0000 7f64acff9700 10 client.4824 handle_command_reply: tid=0 2023-03-29T10:58:04.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.667+0000 7f64e764ab80 1 client.4824 shutdown 2023-03-29T10:58:04.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.668+0000 7f64e764ab80 1 -- 172.21.15.111:0/1907134857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9af060 msgr2=0x55f35fa11340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:04.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.668+0000 7f64e764ab80 1 --2- 172.21.15.111:0/1907134857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9af060 0x55f35fa11340 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f64bc05ecb0 tx=0x7f64bc05c050 comp rx=0 tx=0).stop 2023-03-29T10:58:04.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.668+0000 7f64e764ab80 1 -- 172.21.15.111:0/1907134857 shutdown_connections 2023-03-29T10:58:04.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.668+0000 7f64e764ab80 1 --2- 172.21.15.111:0/1907134857 >> [v2:172.21.15.119:6833/3160800581,v1:172.21.15.119:6835/3160800581] conn(0x55f35f9eec50 0x55f35fa63d20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.668+0000 7f64e764ab80 1 --2- 172.21.15.111:0/1907134857 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x55f35f9b06e0 0x55f35fa58790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.668+0000 7f64e764ab80 1 --2- 172.21.15.111:0/1907134857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55f35f9a7ac0 0x55f35f9ac690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.668+0000 7f64e764ab80 1 --2- 172.21.15.111:0/1907134857 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x55f35f9af060 0x55f35fa11340 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.668+0000 7f64e764ab80 1 -- 172.21.15.111:0/1907134857 >> 172.21.15.111:0/1907134857 conn(0x55f35f9b2810 msgr2=0x55f35f9f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:04.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.668+0000 7f64acff9700 0 client.4824 ms_handle_reset on v2:172.21.15.119:6833/3160800581 2023-03-29T10:58:04.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.668+0000 7f64e764ab80 1 -- 172.21.15.111:0/1907134857 shutdown_connections 2023-03-29T10:58:04.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.668+0000 7f64e764ab80 1 -- 172.21.15.111:0/1907134857 wait complete. 2023-03-29T10:58:04.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.668+0000 7f64e764ab80 20 client.4824 trim_cache size 0 max 16384 2023-03-29T10:58:04.670 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-03-29T10:58:04.670 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "c8526f71-9b58-4b2e-978f-708199ec8aef", 2023-03-29T10:58:04.670 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 0, 2023-03-29T10:58:04.670 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 4296, 2023-03-29T10:58:04.670 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2023-03-29T10:58:04.670 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2023-03-29T10:58:04.670 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2023-03-29T10:58:04.670 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 36.841074685000002, 2023-03-29T10:58:04.670 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 9, 2023-03-29T10:58:04.670 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 28, 2023-03-29T10:58:04.670 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 28, 2023-03-29T10:58:04.670 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 42.132173289000001 2023-03-29T10:58:04.670 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-03-29T10:58:04.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.669+0000 7f64d3dff700 1 -- 172.21.15.111:0/1048752595 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f64b0069de0 msgr2=0x7f64b006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:04.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.669+0000 7f64d3dff700 1 --2- 172.21.15.111:0/1048752595 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f64b0069de0 0x7f64b006c290 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0ca900 tx=0x7f64c8008040 comp rx=0 tx=0).stop 2023-03-29T10:58:04.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.669+0000 7f64d3dff700 1 -- 172.21.15.111:0/1048752595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64cc1601e0 msgr2=0x7f64cc0b5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:04.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.669+0000 7f64d3dff700 1 --2- 172.21.15.111:0/1048752595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64cc1601e0 0x7f64cc0b5ce0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f64bc0011e0 tx=0x7f64bc00cae0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.669+0000 7f64d3dff700 1 -- 172.21.15.111:0/1048752595 shutdown_connections 2023-03-29T10:58:04.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.669+0000 7f64d3dff700 1 --2- 172.21.15.111:0/1048752595 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f64b0069de0 0x7f64b006c290 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.669+0000 7f64d3dff700 1 --2- 172.21.15.111:0/1048752595 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64cc160d40 0x7f64cc0b6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.669+0000 7f64d3dff700 1 --2- 172.21.15.111:0/1048752595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64cc0b87a0 0x7f64cc154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.669+0000 7f64d3dff700 1 --2- 172.21.15.111:0/1048752595 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64cc1601e0 0x7f64cc0b5ce0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:04.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.669+0000 7f64d3dff700 1 -- 172.21.15.111:0/1048752595 >> 172.21.15.111:0/1048752595 conn(0x7f64cc071b10 msgr2=0x7f64cc15c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:04.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.669+0000 7f64d3dff700 1 -- 172.21.15.111:0/1048752595 shutdown_connections 2023-03-29T10:58:04.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:04.669+0000 7f64d3dff700 1 -- 172.21.15.111:0/1048752595 wait complete. 2023-03-29T10:58:04.685 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-03-29T10:58:04.696 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-03-29T10:58:04.696 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-03-29T10:58:04.697 INFO:tasks.ceph_fuse:client.0 config is {} 2023-03-29T10:58:04.697 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-03-29T10:58:04.697 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-03-29T10:58:04.697 INFO:tasks.ceph_fuse:client.2 config is {} 2023-03-29T10:58:04.697 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-03-29T10:58:04.697 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-03-29T10:58:04.697 INFO:tasks.ceph_fuse:client.1 config is {} 2023-03-29T10:58:04.697 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-03-29T10:58:04.697 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-03-29T10:58:04.697 INFO:tasks.ceph_fuse:client.3 config is {} 2023-03-29T10:58:04.697 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-03-29T10:58:04.697 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-03-29T10:58:04.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:04.697 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-03-29T10:58:04.738 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-03-29T10:58:04.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:04.739 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-03-29T10:58:04.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:04.841 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link delete ceph-brx 2023-03-29T10:58:04.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:04.942 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-03-29T10:58:04.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:04.989 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2023-03-29T10:58:05.059 INFO:teuthology.orchestra.run.smithi119.stderr:Cannot find device "ceph-brx" 2023-03-29T10:58:05.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T10:58:05.062 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-03-29T10:58:05.063 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-03-29T10:58:05.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.247+0000 7f9863dff700 1 Processor -- start 2023-03-29T10:58:05.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.247+0000 7f9863dff700 1 -- start start 2023-03-29T10:58:05.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.248+0000 7f9863dff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f985c15c950 0x7f985c0cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:05.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.249+0000 7f9863dff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985c0cdc80 con 0x7f985c0cf950 2023-03-29T10:58:05.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.249+0000 7f9863dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f985c0cd6c0 con 0x7f985c0d3a30 2023-03-29T10:58:05.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.249+0000 7f9863dff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985c0d3e00 con 0x7f985c15c950 2023-03-29T10:58:05.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.249+0000 7f9861b9b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f985c0cf950 0x7f985c0cfd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37028/0 (socket says 172.21.15.111:37028) 2023-03-29T10:58:05.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.249+0000 7f986139a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f985c15c950 0x7f985c0cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:05.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.249+0000 7f9861b9b700 1 -- 172.21.15.111:0/236036538 learned_addr learned my addr 172.21.15.111:0/236036538 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:05.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.249+0000 7f986139a700 1 -- 172.21.15.111:0/236036538 >> v1:172.21.15.119:6789/0 conn(0x7f985c0d3a30 legacy=0x7f985c0b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:05.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.249+0000 7f986139a700 1 -- 172.21.15.111:0/236036538 >> v1:172.21.15.111:6789/0 conn(0x7f985c0cf950 legacy=0x7f985c0cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:05.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.249+0000 7f986139a700 1 -- 172.21.15.111:0/236036538 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985c0cd9a0 con 0x7f985c15c950 2023-03-29T10:58:05.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.249+0000 7f9860b99700 1 -- 172.21.15.111:0/236036538 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2802762013 0 0) 0x7f985c0cdc80 con 0x7f985c0cf950 2023-03-29T10:58:05.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.249+0000 7f986139a700 1 --2- 172.21.15.111:0/236036538 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f985c15c950 0x7f985c0cf0f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f984c009840 tx=0x7f984c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ce0f7b937e6f55 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:05.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.250+0000 7f9860b99700 1 -- 172.21.15.111:0/236036538 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c011830 con 0x7f985c15c950 2023-03-29T10:58:05.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.250+0000 7f9860b99700 1 -- 172.21.15.111:0/236036538 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c011990 con 0x7f985c15c950 2023-03-29T10:58:05.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.250+0000 7f9860b99700 1 -- 172.21.15.111:0/236036538 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c011d20 con 0x7f985c15c950 2023-03-29T10:58:05.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.250+0000 7f9863dff700 1 -- 172.21.15.111:0/236036538 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f985c15c950 msgr2=0x7f985c0cf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:05.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.250+0000 7f9863dff700 1 --2- 172.21.15.111:0/236036538 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f985c15c950 0x7f985c0cf0f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f984c009840 tx=0x7f984c009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:05.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.250+0000 7f9863dff700 1 -- 172.21.15.111:0/236036538 shutdown_connections 2023-03-29T10:58:05.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.250+0000 7f9863dff700 1 --2- 172.21.15.111:0/236036538 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f985c15c950 0x7f985c0cf0f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:05.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.250+0000 7f9863dff700 1 -- 172.21.15.111:0/236036538 >> 172.21.15.111:0/236036538 conn(0x7f985c071b10 msgr2=0x7f985c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:05.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.250+0000 7f9863dff700 1 -- 172.21.15.111:0/236036538 shutdown_connections 2023-03-29T10:58:05.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.250+0000 7f9863dff700 1 -- 172.21.15.111:0/236036538 wait complete. 2023-03-29T10:58:05.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.252+0000 7f9863dff700 1 Processor -- start 2023-03-29T10:58:05.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.252+0000 7f9863dff700 1 -- start start 2023-03-29T10:58:05.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.253+0000 7f9863dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f985c0cf950 0x7f985c0c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:05.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.253+0000 7f9863dff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f985c0d3a30 0x7f985c15f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:05.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.253+0000 7f9863dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f985c15c950 0x7f985c15f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:05.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.253+0000 7f9863dff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c15fc50 con 0x7f985c0d3a30 2023-03-29T10:58:05.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.253+0000 7f9863dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985c0ca010 con 0x7f985c15c950 2023-03-29T10:58:05.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.253+0000 7f9863dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f985c0c83a0 con 0x7f985c0cf950 2023-03-29T10:58:05.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.253+0000 7f986139a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f985c0d3a30 0x7f985c15f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:05.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.253+0000 7f986139a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f985c0d3a30 0x7f985c15f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50538/0 (socket says 172.21.15.111:50538) 2023-03-29T10:58:05.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.253+0000 7f986139a700 1 -- 172.21.15.111:0/4189569078 learned_addr learned my addr 172.21.15.111:0/4189569078 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:05.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.253+0000 7f9861b9b700 1 --2- 172.21.15.111:0/4189569078 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f985c0cf950 0x7f985c0c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:05.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.253+0000 7f986239c700 1 --2- 172.21.15.111:0/4189569078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f985c15c950 0x7f985c15f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:05.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.254+0000 7f986139a700 1 -- 172.21.15.111:0/4189569078 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f985c0cf950 msgr2=0x7f985c0c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:05.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.254+0000 7f986139a700 1 --2- 172.21.15.111:0/4189569078 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f985c0cf950 0x7f985c0c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:05.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.254+0000 7f986139a700 1 -- 172.21.15.111:0/4189569078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f985c15c950 msgr2=0x7f985c15f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:05.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.254+0000 7f986139a700 1 --2- 172.21.15.111:0/4189569078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f985c15c950 0x7f985c15f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:05.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.254+0000 7f986139a700 1 -- 172.21.15.111:0/4189569078 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984c0094a0 con 0x7f985c0d3a30 2023-03-29T10:58:05.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.254+0000 7f986139a700 1 --2- 172.21.15.111:0/4189569078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f985c0d3a30 0x7f985c15f1d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f984c014040 tx=0x7f984c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:05.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.254+0000 7f98527fc700 1 -- 172.21.15.111:0/4189569078 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c01a7e0 con 0x7f985c0d3a30 2023-03-29T10:58:05.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.254+0000 7f98527fc700 1 -- 172.21.15.111:0/4189569078 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c01a940 con 0x7f985c0d3a30 2023-03-29T10:58:05.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.254+0000 7f98527fc700 1 -- 172.21.15.111:0/4189569078 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c01acb0 con 0x7f985c0d3a30 2023-03-29T10:58:05.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.254+0000 7f9863dff700 1 -- 172.21.15.111:0/4189569078 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f985c0d19a0 con 0x7f985c0d3a30 2023-03-29T10:58:05.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.254+0000 7f9863dff700 1 -- 172.21.15.111:0/4189569078 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985c0d16c0 con 0x7f985c0d3a30 2023-03-29T10:58:05.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.255+0000 7f9863dff700 1 -- 172.21.15.111:0/4189569078 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985c0b64a0 con 0x7f985c0d3a30 2023-03-29T10:58:05.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.256+0000 7f98527fc700 1 -- 172.21.15.111:0/4189569078 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f984c022070 con 0x7f985c0d3a30 2023-03-29T10:58:05.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.257+0000 7f98527fc700 1 --2- 172.21.15.111:0/4189569078 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9840069de0 0x7f984006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:05.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.257+0000 7f98527fc700 1 -- 172.21.15.111:0/4189569078 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f984c088ec0 con 0x7f985c0d3a30 2023-03-29T10:58:05.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.257+0000 7f9861b9b700 1 --2- 172.21.15.111:0/4189569078 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9840069de0 0x7f984006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:05.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.258+0000 7f9861b9b700 1 --2- 172.21.15.111:0/4189569078 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9840069de0 0x7f984006c290 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9858007b10 tx=0x7f9858007a60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:05.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.261+0000 7f98527fc700 1 -- 172.21.15.111:0/4189569078 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f984c0545c0 con 0x7f985c0d3a30 2023-03-29T10:58:05.501 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.500+0000 7f9863dff700 1 -- 172.21.15.111:0/4189569078 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f985c074430 con 0x7f985c0d3a30 2023-03-29T10:58:05.501 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.500+0000 7f98527fc700 1 -- 172.21.15.111:0/4189569078 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f984c029070 con 0x7f985c0d3a30 2023-03-29T10:58:05.501 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-03-29T10:58:05.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.503+0000 7f9863dff700 1 -- 172.21.15.111:0/4189569078 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9840069de0 msgr2=0x7f984006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:05.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.503+0000 7f9863dff700 1 --2- 172.21.15.111:0/4189569078 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9840069de0 0x7f984006c290 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9858007b10 tx=0x7f9858007a60 comp rx=0 tx=0).stop 2023-03-29T10:58:05.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.503+0000 7f9863dff700 1 -- 172.21.15.111:0/4189569078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f985c0d3a30 msgr2=0x7f985c15f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:05.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.503+0000 7f9863dff700 1 --2- 172.21.15.111:0/4189569078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f985c0d3a30 0x7f985c15f1d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f984c014040 tx=0x7f984c004750 comp rx=0 tx=0).stop 2023-03-29T10:58:05.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.503+0000 7f9863dff700 1 -- 172.21.15.111:0/4189569078 shutdown_connections 2023-03-29T10:58:05.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.503+0000 7f9863dff700 1 --2- 172.21.15.111:0/4189569078 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9840069de0 0x7f984006c290 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:05.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.503+0000 7f9863dff700 1 --2- 172.21.15.111:0/4189569078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f985c15c950 0x7f985c15f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:05.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.503+0000 7f9863dff700 1 --2- 172.21.15.111:0/4189569078 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f985c0cf950 0x7f985c0c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:05.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.503+0000 7f9863dff700 1 --2- 172.21.15.111:0/4189569078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f985c0d3a30 0x7f985c15f1d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:05.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.503+0000 7f9863dff700 1 -- 172.21.15.111:0/4189569078 >> 172.21.15.111:0/4189569078 conn(0x7f985c071b10 msgr2=0x7f985c071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:05.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.503+0000 7f9863dff700 1 -- 172.21.15.111:0/4189569078 shutdown_connections 2023-03-29T10:58:05.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:05.503+0000 7f9863dff700 1 -- 172.21.15.111:0/4189569078 wait complete. 2023-03-29T10:58:05.521 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-03-29T10:58:05.521 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-03-29T10:58:05.521 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-03-29T10:58:05.521 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-03-29T10:58:05.521 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-03-29T10:58:05.521 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-03-29T10:58:05.521 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-03-29T10:58:05.521 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-03-29T10:58:05.521 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-03-29T10:58:05.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:05.521 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-03-29T10:58:05.561 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-29T10:58:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-29T10:58:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-03-29T10:58:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-03-29T10:58:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:05.562 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-29T10:58:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-03-29T10:58:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-03-29T10:58:05.562 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-29T10:58:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-03-29T10:58:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-03-29T10:58:05.562 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-29T10:58:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-03-29T10:58:05.562 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-03-29T10:58:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-03-29T10:58:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-03-29T10:58:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42332sec preferred_lft 42332sec 2023-03-29T10:58:05.563 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-03-29T10:58:05.563 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:05.563 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-03-29T10:58:05.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:05.564 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-03-29T10:58:05.564 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-03-29T10:58:05.564 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add name ceph-brx type bridge 2023-03-29T10:58:05.564 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr flush dev ceph-brx 2023-03-29T10:58:05.564 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set ceph-brx up 2023-03-29T10:58:05.564 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-03-29T10:58:05.564 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-03-29T10:58:05.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:05.777 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-03-29T10:58:05.838 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-03-29T10:58:05.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:05.841 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-03-29T10:58:05.884 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-03-29T10:58:05.884 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-03-29T10:58:05.884 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-03-29T10:58:05.884 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-03-29T10:58:05.884 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-03-29T10:58:05.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:05.885 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-03-29T10:58:05.885 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-03-29T10:58:05.885 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-03-29T10:58:05.886 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-03-29T10:58:05.886 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-03-29T10:58:05.886 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-03-29T10:58:06.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:06.075 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-03-29T10:58:06.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:06.116 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list-id 2023-03-29T10:58:06.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:06.198 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-03-29T10:58:06.198 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-03-29T10:58:06.198 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-03-29T10:58:06.198 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-03-29T10:58:06.198 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-03-29T10:58:06.363 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-03-29T10:58:06.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:06.363 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-03-29T10:58:06.363 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-03-29T10:58:06.364 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-03-29T10:58:06.364 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-03-29T10:58:06.364 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-03-29T10:58:06.364 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-03-29T10:58:06.364 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-03-29T10:58:06.364 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-03-29T10:58:06.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:06.574 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-03-29T10:58:06.574 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-03-29T10:58:06.574 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-03-29T10:58:06.574 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set dev brx.0 master ceph-brx 2023-03-29T10:58:06.574 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-03-29T10:58:06.738 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-03-29T10:58:06.738 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-29T10:58:06.738 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-03-29T10:58:06.779 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-03-29T10:58:06.779 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-29T10:58:06.779 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-03-29T10:58:06.858 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2023-03-29T10:58:06.961 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-03-29T10:58:06.961 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-03-29T10:58:07.021 INFO:teuthology.orchestra.run.smithi111.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-03-29T10:58:07.023 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-03-29T10:58:07.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:07.023 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2023-03-29T10:58:07.063 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-03-29T10:58:07.063 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-03-29T10:58:07.107 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2023-03-29T10:58:07.169 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-03-29T10:58:07.169 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-03-29T10:58:07.229 INFO:teuthology.orchestra.run.smithi111.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-03-29T10:58:07.232 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-03-29T10:58:07.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:07.232 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2023-03-29T10:58:07.259 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:2023-03-29T10:58:07.258+0000 7f6867b9d5c0 -1 init, newargv = 0x555dce71faf0 newargc=15 2023-03-29T10:58:07.259 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[122136]: starting ceph client 2023-03-29T10:58:07.267 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[122136]: starting fuse 2023-03-29T10:58:07.274 INFO:teuthology.orchestra.run.smithi111.stdout:42 2023-03-29T10:58:07.275 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-03-29T10:58:07.275 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-03-29T10:58:07.275 DEBUG:teuthology.orchestra.run.smithi111:> import glob 2023-03-29T10:58:07.275 DEBUG:teuthology.orchestra.run.smithi111:> import re 2023-03-29T10:58:07.275 DEBUG:teuthology.orchestra.run.smithi111:> import os 2023-03-29T10:58:07.275 DEBUG:teuthology.orchestra.run.smithi111:> import subprocess 2023-03-29T10:58:07.275 DEBUG:teuthology.orchestra.run.smithi111:> 2023-03-29T10:58:07.275 DEBUG:teuthology.orchestra.run.smithi111:> def _find_admin_socket(client_name): 2023-03-29T10:58:07.275 DEBUG:teuthology.orchestra.run.smithi111:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-03-29T10:58:07.275 DEBUG:teuthology.orchestra.run.smithi111:> files = glob.glob(asok_path) 2023-03-29T10:58:07.275 DEBUG:teuthology.orchestra.run.smithi111:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-03-29T10:58:07.275 DEBUG:teuthology.orchestra.run.smithi111:> 2023-03-29T10:58:07.275 DEBUG:teuthology.orchestra.run.smithi111:> # Given a non-glob path, it better be there 2023-03-29T10:58:07.276 DEBUG:teuthology.orchestra.run.smithi111:> if "*" not in asok_path: 2023-03-29T10:58:07.276 DEBUG:teuthology.orchestra.run.smithi111:> assert(len(files) == 1) 2023-03-29T10:58:07.276 DEBUG:teuthology.orchestra.run.smithi111:> return files[0] 2023-03-29T10:58:07.276 DEBUG:teuthology.orchestra.run.smithi111:> 2023-03-29T10:58:07.276 DEBUG:teuthology.orchestra.run.smithi111:> for f in files: 2023-03-29T10:58:07.276 DEBUG:teuthology.orchestra.run.smithi111:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-03-29T10:58:07.276 DEBUG:teuthology.orchestra.run.smithi111:> if os.path.exists("/proc/{0}".format(pid)): 2023-03-29T10:58:07.276 DEBUG:teuthology.orchestra.run.smithi111:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-03-29T10:58:07.276 DEBUG:teuthology.orchestra.run.smithi111:> contents = proc_f.read() 2023-03-29T10:58:07.276 DEBUG:teuthology.orchestra.run.smithi111:> if mountpoint in contents: 2023-03-29T10:58:07.276 DEBUG:teuthology.orchestra.run.smithi111:> return f 2023-03-29T10:58:07.276 DEBUG:teuthology.orchestra.run.smithi111:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-03-29T10:58:07.276 DEBUG:teuthology.orchestra.run.smithi111:> 2023-03-29T10:58:07.276 DEBUG:teuthology.orchestra.run.smithi111:> print(_find_admin_socket("client.0")) 2023-03-29T10:58:07.276 DEBUG:teuthology.orchestra.run.smithi111:> ' 2023-03-29T10:58:07.450 INFO:teuthology.orchestra.run.smithi111.stdout:/var/run/ceph/ceph-client.0.122136.asok 2023-03-29T10:58:07.628 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.122136.asok 2023-03-29T10:58:07.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:07.628 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.122136.asok status 2023-03-29T10:58:07.842 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-03-29T10:58:07.843 INFO:teuthology.orchestra.run.smithi111.stdout: "metadata": { 2023-03-29T10:58:07.843 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_sha1": "f03d69a8741c605ae6711afe2c9991ac46095f7d", 2023-03-29T10:58:07.843 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 18.0.0-3066-gf03d69a8 (f03d69a8741c605ae6711afe2c9991ac46095f7d) reef (dev)", 2023-03-29T10:58:07.843 INFO:teuthology.orchestra.run.smithi111.stdout: "entity_id": "0", 2023-03-29T10:58:07.843 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": "smithi111", 2023-03-29T10:58:07.843 INFO:teuthology.orchestra.run.smithi111.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-03-29T10:58:07.843 INFO:teuthology.orchestra.run.smithi111.stdout: "pid": "122136", 2023-03-29T10:58:07.843 INFO:teuthology.orchestra.run.smithi111.stdout: "root": "/" 2023-03-29T10:58:07.843 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:58:07.843 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_count": 0, 2023-03-29T10:58:07.843 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_pinned_count": 0, 2023-03-29T10:58:07.843 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 4839, 2023-03-29T10:58:07.843 INFO:teuthology.orchestra.run.smithi111.stdout: "inst": { 2023-03-29T10:58:07.843 INFO:teuthology.orchestra.run.smithi111.stdout: "name": { 2023-03-29T10:58:07.843 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "client", 2023-03-29T10:58:07.843 INFO:teuthology.orchestra.run.smithi111.stdout: "num": 4839 2023-03-29T10:58:07.844 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:58:07.844 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2023-03-29T10:58:07.844 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2023-03-29T10:58:07.844 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2023-03-29T10:58:07.844 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 3759017467 2023-03-29T10:58:07.844 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:58:07.844 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:58:07.844 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2023-03-29T10:58:07.844 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2023-03-29T10:58:07.844 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2023-03-29T10:58:07.844 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 3759017467 2023-03-29T10:58:07.844 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:58:07.844 INFO:teuthology.orchestra.run.smithi111.stdout: "inst_str": "client.4839 192.168.0.1:0/3759017467", 2023-03-29T10:58:07.844 INFO:teuthology.orchestra.run.smithi111.stdout: "addr_str": "192.168.0.1:0/3759017467", 2023-03-29T10:58:07.844 INFO:teuthology.orchestra.run.smithi111.stdout: "inode_count": 1, 2023-03-29T10:58:07.844 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_epoch": 9, 2023-03-29T10:58:07.844 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch": 28, 2023-03-29T10:58:07.844 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch_barrier": 0, 2023-03-29T10:58:07.845 INFO:teuthology.orchestra.run.smithi111.stdout: "blocklisted": false, 2023-03-29T10:58:07.845 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs" 2023-03-29T10:58:07.845 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-03-29T10:58:07.857 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-03-29T10:58:08.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.040+0000 7f7a58fff700 1 Processor -- start 2023-03-29T10:58:08.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.040+0000 7f7a58fff700 1 -- start start 2023-03-29T10:58:08.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.041+0000 7f7a58fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a540caa40 0x7f7a54070d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:08.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.041+0000 7f7a58fff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a540cc9b0 con 0x7f7a540cda50 2023-03-29T10:58:08.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.041+0000 7f7a58fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a540ccc90 con 0x7f7a54158d30 2023-03-29T10:58:08.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.041+0000 7f7a58fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a54071dd0 con 0x7f7a540caa40 2023-03-29T10:58:08.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.042+0000 7f7a52d9d700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f7a540cda50 0x7f7a5406cdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:37042/0 (socket says 172.21.15.111:37042) 2023-03-29T10:58:08.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.042+0000 7f7a52d9d700 1 -- 172.21.15.111:0/3290102725 learned_addr learned my addr 172.21.15.111:0/3290102725 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:08.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.042+0000 7f7a51d9b700 1 --2- 172.21.15.111:0/3290102725 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a540caa40 0x7f7a54070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:08.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.042+0000 7f7a5159a700 1 -- 172.21.15.111:0/3290102725 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2294181089 0 0) 0x7f7a540cc9b0 con 0x7f7a540cda50 2023-03-29T10:58:08.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.042+0000 7f7a5159a700 1 -- 172.21.15.111:0/3290102725 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a3c0034d0 con 0x7f7a540cda50 2023-03-29T10:58:08.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.042+0000 7f7a5159a700 1 -- 172.21.15.111:0/3290102725 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3746592198 0 0) 0x7f7a540ccc90 con 0x7f7a54158d30 2023-03-29T10:58:08.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.042+0000 7f7a5159a700 1 -- 172.21.15.111:0/3290102725 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a540cc9b0 con 0x7f7a54158d30 2023-03-29T10:58:08.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.042+0000 7f7a51d9b700 1 -- 172.21.15.111:0/3290102725 >> v1:172.21.15.119:6789/0 conn(0x7f7a54158d30 legacy=0x7f7a540ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:08.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.042+0000 7f7a51d9b700 1 -- 172.21.15.111:0/3290102725 >> v1:172.21.15.111:6789/0 conn(0x7f7a540cda50 legacy=0x7f7a5406cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:08.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.042+0000 7f7a51d9b700 1 -- 172.21.15.111:0/3290102725 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a540ccf70 con 0x7f7a540caa40 2023-03-29T10:58:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.042+0000 7f7a51d9b700 1 --2- 172.21.15.111:0/3290102725 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a540caa40 0x7f7a54070d60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009870 tx=0x7f7a48009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d63598f558c743ae server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.043+0000 7f7a5159a700 1 -- 172.21.15.111:0/3290102725 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a48011830 con 0x7f7a540caa40 2023-03-29T10:58:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.043+0000 7f7a5159a700 1 -- 172.21.15.111:0/3290102725 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a48011990 con 0x7f7a540caa40 2023-03-29T10:58:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.043+0000 7f7a5159a700 1 -- 172.21.15.111:0/3290102725 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a48011d20 con 0x7f7a540caa40 2023-03-29T10:58:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.043+0000 7f7a58fff700 1 -- 172.21.15.111:0/3290102725 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a540caa40 msgr2=0x7f7a54070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.043+0000 7f7a58fff700 1 --2- 172.21.15.111:0/3290102725 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a540caa40 0x7f7a54070d60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7a48009870 tx=0x7f7a48009b80 comp rx=0 tx=0).stop 2023-03-29T10:58:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.043+0000 7f7a58fff700 1 -- 172.21.15.111:0/3290102725 shutdown_connections 2023-03-29T10:58:08.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.043+0000 7f7a58fff700 1 --2- 172.21.15.111:0/3290102725 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a540caa40 0x7f7a54070d60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:08.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.043+0000 7f7a58fff700 1 -- 172.21.15.111:0/3290102725 >> 172.21.15.111:0/3290102725 conn(0x7f7a540cb6c0 msgr2=0x7f7a540cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:08.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.044+0000 7f7a58fff700 1 -- 172.21.15.111:0/3290102725 shutdown_connections 2023-03-29T10:58:08.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.044+0000 7f7a58fff700 1 -- 172.21.15.111:0/3290102725 wait complete. 2023-03-29T10:58:08.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.045+0000 7f7a58fff700 1 Processor -- start 2023-03-29T10:58:08.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.045+0000 7f7a58fff700 1 -- start start 2023-03-29T10:58:08.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.046+0000 7f7a58fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a540cda50 0x7f7a541f6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:08.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.046+0000 7f7a58fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a54158d30 0x7f7a5414fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:08.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.046+0000 7f7a58fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a540c54e0 0x7f7a540c5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:08.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.046+0000 7f7a58fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a541f6550 con 0x7f7a540c54e0 2023-03-29T10:58:08.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.046+0000 7f7a58fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a540c7bd0 con 0x7f7a540cda50 2023-03-29T10:58:08.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.046+0000 7f7a58fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a540c3da0 con 0x7f7a54158d30 2023-03-29T10:58:08.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.046+0000 7f7a52d9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a540c54e0 0x7f7a540c5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:08.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.047+0000 7f7a52d9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a540c54e0 0x7f7a540c5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:50542/0 (socket says 172.21.15.111:50542) 2023-03-29T10:58:08.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.047+0000 7f7a52d9d700 1 -- 172.21.15.111:0/1193916114 learned_addr learned my addr 172.21.15.111:0/1193916114 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:08.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.047+0000 7f7a5259c700 1 --2- 172.21.15.111:0/1193916114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a540cda50 0x7f7a541f6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:08.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.047+0000 7f7a51d9b700 1 --2- 172.21.15.111:0/1193916114 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a54158d30 0x7f7a5414fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:08.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.047+0000 7f7a52d9d700 1 -- 172.21.15.111:0/1193916114 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a54158d30 msgr2=0x7f7a5414fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:08.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.047+0000 7f7a52d9d700 1 --2- 172.21.15.111:0/1193916114 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a54158d30 0x7f7a5414fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:08.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.047+0000 7f7a52d9d700 1 -- 172.21.15.111:0/1193916114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a540cda50 msgr2=0x7f7a541f6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:08.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.047+0000 7f7a52d9d700 1 --2- 172.21.15.111:0/1193916114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a540cda50 0x7f7a541f6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:08.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.047+0000 7f7a52d9d700 1 -- 172.21.15.111:0/1193916114 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a480094a0 con 0x7f7a540c54e0 2023-03-29T10:58:08.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.047+0000 7f7a52d9d700 1 --2- 172.21.15.111:0/1193916114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a540c54e0 0x7f7a540c5d10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00cbd0 tx=0x7f7a4c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:08.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.047+0000 7f7a42ffd700 1 -- 172.21.15.111:0/1193916114 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c00ab10 con 0x7f7a540c54e0 2023-03-29T10:58:08.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.047+0000 7f7a42ffd700 1 -- 172.21.15.111:0/1193916114 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c00ac70 con 0x7f7a540c54e0 2023-03-29T10:58:08.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.047+0000 7f7a58fff700 1 -- 172.21.15.111:0/1193916114 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a540cd530 con 0x7f7a540c54e0 2023-03-29T10:58:08.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.047+0000 7f7a42ffd700 1 -- 172.21.15.111:0/1193916114 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c012400 con 0x7f7a540c54e0 2023-03-29T10:58:08.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.048+0000 7f7a58fff700 1 -- 172.21.15.111:0/1193916114 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a540cd250 con 0x7f7a540c54e0 2023-03-29T10:58:08.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.049+0000 7f7a58fff700 1 -- 172.21.15.111:0/1193916114 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a540712a0 con 0x7f7a540c54e0 2023-03-29T10:58:08.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.049+0000 7f7a42ffd700 1 -- 172.21.15.111:0/1193916114 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f7a4c01b3b0 con 0x7f7a540c54e0 2023-03-29T10:58:08.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.050+0000 7f7a42ffd700 1 --2- 172.21.15.111:0/1193916114 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7a34069de0 0x7f7a3406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:08.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.051+0000 7f7a42ffd700 1 -- 172.21.15.111:0/1193916114 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7a4c087fd0 con 0x7f7a540c54e0 2023-03-29T10:58:08.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.051+0000 7f7a5259c700 1 --2- 172.21.15.111:0/1193916114 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7a34069de0 0x7f7a3406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:08.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.051+0000 7f7a5259c700 1 --2- 172.21.15.111:0/1193916114 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7a34069de0 0x7f7a3406c290 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7a44007fd0 tx=0x7f7a44009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:08.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.055+0000 7f7a42ffd700 1 -- 172.21.15.111:0/1193916114 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f7a4c04c160 con 0x7f7a540c54e0 2023-03-29T10:58:08.297 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.297+0000 7f7a58fff700 1 -- 172.21.15.111:0/1193916114 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7a54074430 con 0x7f7a540c54e0 2023-03-29T10:58:08.298 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.297+0000 7f7a42ffd700 1 -- 172.21.15.111:0/1193916114 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f7a4c017070 con 0x7f7a540c54e0 2023-03-29T10:58:08.298 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-03-29T10:58:08.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.299+0000 7f7a58fff700 1 -- 172.21.15.111:0/1193916114 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7a34069de0 msgr2=0x7f7a3406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:08.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.299+0000 7f7a58fff700 1 --2- 172.21.15.111:0/1193916114 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7a34069de0 0x7f7a3406c290 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7a44007fd0 tx=0x7f7a44009040 comp rx=0 tx=0).stop 2023-03-29T10:58:08.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.299+0000 7f7a58fff700 1 -- 172.21.15.111:0/1193916114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a540c54e0 msgr2=0x7f7a540c5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:08.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.299+0000 7f7a58fff700 1 --2- 172.21.15.111:0/1193916114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a540c54e0 0x7f7a540c5d10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00cbd0 tx=0x7f7a4c00cf90 comp rx=0 tx=0).stop 2023-03-29T10:58:08.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.299+0000 7f7a58fff700 1 -- 172.21.15.111:0/1193916114 shutdown_connections 2023-03-29T10:58:08.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.299+0000 7f7a58fff700 1 --2- 172.21.15.111:0/1193916114 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7a34069de0 0x7f7a3406c290 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:08.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.299+0000 7f7a58fff700 1 --2- 172.21.15.111:0/1193916114 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f7a540c54e0 0x7f7a540c5d10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:08.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.299+0000 7f7a58fff700 1 --2- 172.21.15.111:0/1193916114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a540cda50 0x7f7a541f6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:08.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.299+0000 7f7a58fff700 1 --2- 172.21.15.111:0/1193916114 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f7a54158d30 0x7f7a5414fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:08.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.299+0000 7f7a58fff700 1 -- 172.21.15.111:0/1193916114 >> 172.21.15.111:0/1193916114 conn(0x7f7a540cb6c0 msgr2=0x7f7a54152f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:08.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.300+0000 7f7a58fff700 1 -- 172.21.15.111:0/1193916114 shutdown_connections 2023-03-29T10:58:08.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:08.300+0000 7f7a58fff700 1 -- 172.21.15.111:0/1193916114 wait complete. 2023-03-29T10:58:08.317 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-03-29T10:58:08.317 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-03-29T10:58:08.317 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-03-29T10:58:08.317 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-03-29T10:58:08.317 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-03-29T10:58:08.317 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-03-29T10:58:08.318 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-03-29T10:58:08.318 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-03-29T10:58:08.318 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-03-29T10:58:08.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:08.318 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-03-29T10:58:08.359 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-29T10:58:08.359 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-29T10:58:08.359 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-03-29T10:58:08.359 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:08.359 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-03-29T10:58:08.359 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:08.359 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-29T10:58:08.359 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-03-29T10:58:08.359 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-03-29T10:58:08.359 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-29T10:58:08.360 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-03-29T10:58:08.360 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-03-29T10:58:08.360 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-29T10:58:08.360 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-03-29T10:58:08.360 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-03-29T10:58:08.360 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-03-29T10:58:08.360 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-03-29T10:58:08.360 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42329sec preferred_lft 42329sec 2023-03-29T10:58:08.360 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-03-29T10:58:08.360 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:08.360 INFO:teuthology.orchestra.run.smithi111.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-03-29T10:58:08.360 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether f2:0a:47:b9:bd:43 brd ff:ff:ff:ff:ff:ff 2023-03-29T10:58:08.360 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-03-29T10:58:08.360 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:08.360 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::54c9:aeff:fe97:250b/64 scope link 2023-03-29T10:58:08.361 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:08.361 INFO:teuthology.orchestra.run.smithi111.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-03-29T10:58:08.361 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether f2:0a:47:b9:bd:43 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-03-29T10:58:08.361 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::f00a:47ff:feb9:bd43/64 scope link 2023-03-29T10:58:08.361 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:08.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:08.362 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-03-29T10:58:08.461 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-03-29T10:58:08.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:08.464 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-03-29T10:58:08.510 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-03-29T10:58:08.510 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-03-29T10:58:08.510 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-03-29T10:58:08.510 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-03-29T10:58:08.510 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-03-29T10:58:08.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:08.510 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-03-29T10:58:08.510 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-03-29T10:58:08.510 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-03-29T10:58:08.510 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-03-29T10:58:08.511 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-03-29T10:58:08.511 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-03-29T10:58:08.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:08.662 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-03-29T10:58:08.702 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-03-29T10:58:08.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:08.703 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list-id 2023-03-29T10:58:08.783 INFO:teuthology.orchestra.run.smithi111.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-03-29T10:58:08.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:08.784 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-03-29T10:58:08.784 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-03-29T10:58:08.784 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-03-29T10:58:08.784 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-03-29T10:58:08.784 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-03-29T10:58:08.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:08.948 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-03-29T10:58:09.010 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-29T10:58:09.010 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-29T10:58:09.010 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-03-29T10:58:09.010 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:09.010 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-03-29T10:58:09.010 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:09.010 INFO:teuthology.orchestra.run.smithi111.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-03-29T10:58:09.010 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 92:6d:47:37:92:52 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-03-29T10:58:09.011 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-03-29T10:58:09.011 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:09.011 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::906d:47ff:fe37:9252/64 scope link 2023-03-29T10:58:09.011 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:09.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:09.013 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-03-29T10:58:09.075 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-29T10:58:09.075 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-29T10:58:09.075 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-03-29T10:58:09.075 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:09.075 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-03-29T10:58:09.075 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:09.075 INFO:teuthology.orchestra.run.smithi111.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-03-29T10:58:09.075 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 92:6d:47:37:92:52 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-03-29T10:58:09.075 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-03-29T10:58:09.075 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:09.075 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::906d:47ff:fe37:9252/64 scope link 2023-03-29T10:58:09.076 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:09.078 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-03-29T10:58:09.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:09.078 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-03-29T10:58:09.078 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-03-29T10:58:09.078 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-03-29T10:58:09.078 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-03-29T10:58:09.079 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-03-29T10:58:09.079 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-03-29T10:58:09.079 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-03-29T10:58:09.079 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-03-29T10:58:09.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:09.298 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-03-29T10:58:09.298 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-03-29T10:58:09.298 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.1 up 2023-03-29T10:58:09.298 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set dev brx.1 master ceph-brx 2023-03-29T10:58:09.298 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-03-29T10:58:09.462 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-03-29T10:58:09.462 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-29T10:58:09.462 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-03-29T10:58:09.505 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-03-29T10:58:09.505 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-29T10:58:09.505 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-03-29T10:58:09.585 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2023-03-29T10:58:09.687 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-03-29T10:58:09.688 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-03-29T10:58:09.748 INFO:teuthology.orchestra.run.smithi111.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-03-29T10:58:09.750 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-03-29T10:58:09.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:09.751 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2023-03-29T10:58:09.791 INFO:teuthology.orchestra.run.smithi111.stdout:42 2023-03-29T10:58:09.791 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2023-03-29T10:58:09.791 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-03-29T10:58:09.835 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2023-03-29T10:58:09.897 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-03-29T10:58:09.897 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-03-29T10:58:09.958 INFO:teuthology.orchestra.run.smithi111.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-03-29T10:58:09.960 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-03-29T10:58:09.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:09.961 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2023-03-29T10:58:09.988 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi111.stderr:2023-03-29T10:58:09.988+0000 7f7c8b7075c0 -1 init, newargv = 0x556cf2132ca0 newargc=15 2023-03-29T10:58:09.989 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi111.stderr:ceph-fuse[122860]: starting ceph client 2023-03-29T10:58:09.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi111.stderr:ceph-fuse[122860]: starting fuse 2023-03-29T10:58:10.003 INFO:teuthology.orchestra.run.smithi111.stdout:42 2023-03-29T10:58:10.003 INFO:teuthology.orchestra.run.smithi111.stdout:44 2023-03-29T10:58:10.004 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 44] 2023-03-29T10:58:10.004 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-03-29T10:58:10.004 DEBUG:teuthology.orchestra.run.smithi111:> import glob 2023-03-29T10:58:10.004 DEBUG:teuthology.orchestra.run.smithi111:> import re 2023-03-29T10:58:10.004 DEBUG:teuthology.orchestra.run.smithi111:> import os 2023-03-29T10:58:10.004 DEBUG:teuthology.orchestra.run.smithi111:> import subprocess 2023-03-29T10:58:10.004 DEBUG:teuthology.orchestra.run.smithi111:> 2023-03-29T10:58:10.005 DEBUG:teuthology.orchestra.run.smithi111:> def _find_admin_socket(client_name): 2023-03-29T10:58:10.005 DEBUG:teuthology.orchestra.run.smithi111:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-03-29T10:58:10.005 DEBUG:teuthology.orchestra.run.smithi111:> files = glob.glob(asok_path) 2023-03-29T10:58:10.005 DEBUG:teuthology.orchestra.run.smithi111:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-03-29T10:58:10.005 DEBUG:teuthology.orchestra.run.smithi111:> 2023-03-29T10:58:10.005 DEBUG:teuthology.orchestra.run.smithi111:> # Given a non-glob path, it better be there 2023-03-29T10:58:10.005 DEBUG:teuthology.orchestra.run.smithi111:> if "*" not in asok_path: 2023-03-29T10:58:10.005 DEBUG:teuthology.orchestra.run.smithi111:> assert(len(files) == 1) 2023-03-29T10:58:10.005 DEBUG:teuthology.orchestra.run.smithi111:> return files[0] 2023-03-29T10:58:10.005 DEBUG:teuthology.orchestra.run.smithi111:> 2023-03-29T10:58:10.005 DEBUG:teuthology.orchestra.run.smithi111:> for f in files: 2023-03-29T10:58:10.005 DEBUG:teuthology.orchestra.run.smithi111:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-03-29T10:58:10.005 DEBUG:teuthology.orchestra.run.smithi111:> if os.path.exists("/proc/{0}".format(pid)): 2023-03-29T10:58:10.005 DEBUG:teuthology.orchestra.run.smithi111:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-03-29T10:58:10.005 DEBUG:teuthology.orchestra.run.smithi111:> contents = proc_f.read() 2023-03-29T10:58:10.005 DEBUG:teuthology.orchestra.run.smithi111:> if mountpoint in contents: 2023-03-29T10:58:10.005 DEBUG:teuthology.orchestra.run.smithi111:> return f 2023-03-29T10:58:10.005 DEBUG:teuthology.orchestra.run.smithi111:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-03-29T10:58:10.005 DEBUG:teuthology.orchestra.run.smithi111:> 2023-03-29T10:58:10.006 DEBUG:teuthology.orchestra.run.smithi111:> print(_find_admin_socket("client.2")) 2023-03-29T10:58:10.006 DEBUG:teuthology.orchestra.run.smithi111:> ' 2023-03-29T10:58:10.180 INFO:teuthology.orchestra.run.smithi111.stdout:/var/run/ceph/ceph-client.2.122860.asok 2023-03-29T10:58:10.358 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.122860.asok 2023-03-29T10:58:10.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:10.358 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.122860.asok status 2023-03-29T10:58:10.578 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-03-29T10:58:10.578 INFO:teuthology.orchestra.run.smithi111.stdout: "metadata": { 2023-03-29T10:58:10.578 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_sha1": "f03d69a8741c605ae6711afe2c9991ac46095f7d", 2023-03-29T10:58:10.578 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 18.0.0-3066-gf03d69a8 (f03d69a8741c605ae6711afe2c9991ac46095f7d) reef (dev)", 2023-03-29T10:58:10.578 INFO:teuthology.orchestra.run.smithi111.stdout: "entity_id": "2", 2023-03-29T10:58:10.578 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": "smithi111", 2023-03-29T10:58:10.578 INFO:teuthology.orchestra.run.smithi111.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-03-29T10:58:10.578 INFO:teuthology.orchestra.run.smithi111.stdout: "pid": "122860", 2023-03-29T10:58:10.578 INFO:teuthology.orchestra.run.smithi111.stdout: "root": "/" 2023-03-29T10:58:10.578 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:58:10.578 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_count": 0, 2023-03-29T10:58:10.578 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_pinned_count": 0, 2023-03-29T10:58:10.578 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 4851, 2023-03-29T10:58:10.579 INFO:teuthology.orchestra.run.smithi111.stdout: "inst": { 2023-03-29T10:58:10.579 INFO:teuthology.orchestra.run.smithi111.stdout: "name": { 2023-03-29T10:58:10.579 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "client", 2023-03-29T10:58:10.579 INFO:teuthology.orchestra.run.smithi111.stdout: "num": 4851 2023-03-29T10:58:10.579 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:58:10.579 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2023-03-29T10:58:10.579 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2023-03-29T10:58:10.579 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.2:0", 2023-03-29T10:58:10.579 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 2895267406 2023-03-29T10:58:10.579 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:58:10.580 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:58:10.580 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2023-03-29T10:58:10.580 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2023-03-29T10:58:10.580 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.2:0", 2023-03-29T10:58:10.580 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 2895267406 2023-03-29T10:58:10.580 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:58:10.580 INFO:teuthology.orchestra.run.smithi111.stdout: "inst_str": "client.4851 192.168.0.2:0/2895267406", 2023-03-29T10:58:10.580 INFO:teuthology.orchestra.run.smithi111.stdout: "addr_str": "192.168.0.2:0/2895267406", 2023-03-29T10:58:10.580 INFO:teuthology.orchestra.run.smithi111.stdout: "inode_count": 1, 2023-03-29T10:58:10.580 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_epoch": 9, 2023-03-29T10:58:10.580 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch": 30, 2023-03-29T10:58:10.580 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch_barrier": 0, 2023-03-29T10:58:10.580 INFO:teuthology.orchestra.run.smithi111.stdout: "blocklisted": false, 2023-03-29T10:58:10.580 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs" 2023-03-29T10:58:10.580 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-03-29T10:58:10.591 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-03-29T10:58:10.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.774+0000 7f8103bff700 1 Processor -- start 2023-03-29T10:58:10.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.774+0000 7f8103bff700 1 -- start start 2023-03-29T10:58:10.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.775+0000 7f8103bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f80fc0cf950 0x7f80fc0cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:10.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.775+0000 7f8103bff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f80fc15c950 0x7f80fc0cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:10.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.775+0000 7f8103bff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80fc0cdc80 con 0x7f80fc0b5d90 2023-03-29T10:58:10.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.775+0000 7f8103bff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80fc0d0260 con 0x7f80fc15c950 2023-03-29T10:58:10.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.775+0000 7f8103bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80fc0cd6c0 con 0x7f80fc0cf950 2023-03-29T10:58:10.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.775+0000 7f810119a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f80fc15c950 0x7f80fc0cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:10.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.775+0000 7f810119a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f80fc15c950 0x7f80fc0cf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55664/0 (socket says 172.21.15.111:55664) 2023-03-29T10:58:10.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.775+0000 7f810119a700 1 -- 172.21.15.111:0/3099440990 learned_addr learned my addr 172.21.15.111:0/3099440990 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:10.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.775+0000 7f810219c700 1 --1- 172.21.15.111:0/3099440990 >> v1:172.21.15.111:6789/0 conn(0x7f80fc0b5d90 0x7f80fc0d1ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58926/0 (socket says 172.21.15.111:58926) 2023-03-29T10:58:10.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.776+0000 7f810199b700 1 --2- 172.21.15.111:0/3099440990 >> v2:172.21.15.119:3300/0 conn(0x7f80fc0cf950 0x7f80fc0cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:10.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.776+0000 7f810119a700 1 -- 172.21.15.111:0/3099440990 >> v2:172.21.15.119:3300/0 conn(0x7f80fc0cf950 msgr2=0x7f80fc0cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:10.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.776+0000 7f810119a700 1 --2- 172.21.15.111:0/3099440990 >> v2:172.21.15.119:3300/0 conn(0x7f80fc0cf950 0x7f80fc0cfd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:10.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.776+0000 7f810119a700 1 -- 172.21.15.111:0/3099440990 >> v1:172.21.15.111:6789/0 conn(0x7f80fc0b5d90 legacy=0x7f80fc0d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:10.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.776+0000 7f8100999700 1 -- 172.21.15.111:0/3099440990 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999155022 0 0) 0x7f80fc0cdc80 con 0x7f80fc0b5d90 2023-03-29T10:58:10.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.776+0000 7f810119a700 1 -- 172.21.15.111:0/3099440990 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80fc0cd9a0 con 0x7f80fc15c950 2023-03-29T10:58:10.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.776+0000 7f810119a700 1 --2- 172.21.15.111:0/3099440990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc15c950 0x7f80fc0cf0f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f80fc08bff0 tx=0x7f80ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82737838cbe7fb2f server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:10.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.776+0000 7f8100999700 1 -- 172.21.15.111:0/3099440990 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec011830 con 0x7f80fc15c950 2023-03-29T10:58:10.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.776+0000 7f8100999700 1 -- 172.21.15.111:0/3099440990 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ec011990 con 0x7f80fc15c950 2023-03-29T10:58:10.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.776+0000 7f8100999700 1 -- 172.21.15.111:0/3099440990 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec011d20 con 0x7f80fc15c950 2023-03-29T10:58:10.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.777+0000 7f8103bff700 1 -- 172.21.15.111:0/3099440990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc15c950 msgr2=0x7f80fc0cf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:10.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.777+0000 7f8103bff700 1 --2- 172.21.15.111:0/3099440990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc15c950 0x7f80fc0cf0f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f80fc08bff0 tx=0x7f80ec009bd0 comp rx=0 tx=0).stop 2023-03-29T10:58:10.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.777+0000 7f8103bff700 1 -- 172.21.15.111:0/3099440990 shutdown_connections 2023-03-29T10:58:10.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.777+0000 7f8103bff700 1 --2- 172.21.15.111:0/3099440990 >> v2:172.21.15.119:3300/0 conn(0x7f80fc0cf950 0x7f80fc0cfd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:10.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.777+0000 7f8103bff700 1 --2- 172.21.15.111:0/3099440990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc15c950 0x7f80fc0cf0f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:10.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.777+0000 7f8103bff700 1 -- 172.21.15.111:0/3099440990 >> 172.21.15.111:0/3099440990 conn(0x7f80fc071b10 msgr2=0x7f80fc071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:10.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.777+0000 7f8103bff700 1 -- 172.21.15.111:0/3099440990 shutdown_connections 2023-03-29T10:58:10.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.777+0000 7f8103bff700 1 -- 172.21.15.111:0/3099440990 wait complete. 2023-03-29T10:58:10.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.779+0000 7f8103bff700 1 Processor -- start 2023-03-29T10:58:10.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.779+0000 7f8103bff700 1 -- start start 2023-03-29T10:58:10.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.780+0000 7f8103bff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc0b5d90 0x7f80fc0cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:10.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.780+0000 7f8103bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f80fc0cf950 0x7f80fc15dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:10.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.780+0000 7f8103bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80fc15c950 0x7f80fc1538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:10.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.780+0000 7f8103bff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc15e020 con 0x7f80fc0b5d90 2023-03-29T10:58:10.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.780+0000 7f8103bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc15d650 con 0x7f80fc15c950 2023-03-29T10:58:10.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.780+0000 7f8103bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc154910 con 0x7f80fc0cf950 2023-03-29T10:58:10.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.780+0000 7f810199b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc0b5d90 0x7f80fc0cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:10.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.780+0000 7f810199b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc0b5d90 0x7f80fc0cbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55666/0 (socket says 172.21.15.111:55666) 2023-03-29T10:58:10.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.780+0000 7f810199b700 1 -- 172.21.15.111:0/681588841 learned_addr learned my addr 172.21.15.111:0/681588841 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:10.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.780+0000 7f810219c700 1 --2- 172.21.15.111:0/681588841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80fc15c950 0x7f80fc1538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:10.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.780+0000 7f810119a700 1 --2- 172.21.15.111:0/681588841 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f80fc0cf950 0x7f80fc15dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:10.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.780+0000 7f810199b700 1 -- 172.21.15.111:0/681588841 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f80fc0cf950 msgr2=0x7f80fc15dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:10.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.780+0000 7f810199b700 1 --2- 172.21.15.111:0/681588841 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f80fc0cf950 0x7f80fc15dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:10.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.780+0000 7f810199b700 1 -- 172.21.15.111:0/681588841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80fc15c950 msgr2=0x7f80fc1538c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:10.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.780+0000 7f810199b700 1 --2- 172.21.15.111:0/681588841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80fc15c950 0x7f80fc1538c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:10.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.780+0000 7f810199b700 1 -- 172.21.15.111:0/681588841 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ec009550 con 0x7f80fc0b5d90 2023-03-29T10:58:10.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.781+0000 7f810199b700 1 --2- 172.21.15.111:0/681588841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc0b5d90 0x7f80fc0cbdb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f80f800e9b0 tx=0x7f80f800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:10.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.781+0000 7f80f27fc700 1 -- 172.21.15.111:0/681588841 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f8009710 con 0x7f80fc0b5d90 2023-03-29T10:58:10.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.781+0000 7f80f27fc700 1 -- 172.21.15.111:0/681588841 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f800c850 con 0x7f80fc0b5d90 2023-03-29T10:58:10.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.781+0000 7f80f27fc700 1 -- 172.21.15.111:0/681588841 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f800cbc0 con 0x7f80fc0b5d90 2023-03-29T10:58:10.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.781+0000 7f8103bff700 1 -- 172.21.15.111:0/681588841 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80fc0d19a0 con 0x7f80fc0b5d90 2023-03-29T10:58:10.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.781+0000 7f8103bff700 1 -- 172.21.15.111:0/681588841 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80fc0d16c0 con 0x7f80fc0b5d90 2023-03-29T10:58:10.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.782+0000 7f8103bff700 1 -- 172.21.15.111:0/681588841 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80fc0cf630 con 0x7f80fc0b5d90 2023-03-29T10:58:10.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.783+0000 7f80f27fc700 1 -- 172.21.15.111:0/681588841 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f80f800cd20 con 0x7f80fc0b5d90 2023-03-29T10:58:10.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.784+0000 7f80f27fc700 1 --2- 172.21.15.111:0/681588841 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f80e0069de0 0x7f80e006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:10.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.784+0000 7f80f27fc700 1 -- 172.21.15.111:0/681588841 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f80f8088460 con 0x7f80fc0b5d90 2023-03-29T10:58:10.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.784+0000 7f810119a700 1 --2- 172.21.15.111:0/681588841 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f80e0069de0 0x7f80e006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:10.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.785+0000 7f810119a700 1 --2- 172.21.15.111:0/681588841 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f80e0069de0 0x7f80e006c290 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f80ec014040 tx=0x7f80ec008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:10.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:10.788+0000 7f80f27fc700 1 -- 172.21.15.111:0/681588841 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f80f8057370 con 0x7f80fc0b5d90 2023-03-29T10:58:11.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:11.022+0000 7f8103bff700 1 -- 172.21.15.111:0/681588841 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f80fc074430 con 0x7f80fc0b5d90 2023-03-29T10:58:11.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:11.023+0000 7f80f27fc700 1 -- 172.21.15.111:0/681588841 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f80f8056f60 con 0x7f80fc0b5d90 2023-03-29T10:58:11.023 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-03-29T10:58:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:11.025+0000 7f8103bff700 1 -- 172.21.15.111:0/681588841 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f80e0069de0 msgr2=0x7f80e006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:11.025+0000 7f8103bff700 1 --2- 172.21.15.111:0/681588841 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f80e0069de0 0x7f80e006c290 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f80ec014040 tx=0x7f80ec008030 comp rx=0 tx=0).stop 2023-03-29T10:58:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:11.025+0000 7f8103bff700 1 -- 172.21.15.111:0/681588841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc0b5d90 msgr2=0x7f80fc0cbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:11.025+0000 7f8103bff700 1 --2- 172.21.15.111:0/681588841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc0b5d90 0x7f80fc0cbdb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f80f800e9b0 tx=0x7f80f800ed70 comp rx=0 tx=0).stop 2023-03-29T10:58:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:11.025+0000 7f8103bff700 1 -- 172.21.15.111:0/681588841 shutdown_connections 2023-03-29T10:58:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:11.025+0000 7f8103bff700 1 --2- 172.21.15.111:0/681588841 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f80e0069de0 0x7f80e006c290 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:11.025+0000 7f8103bff700 1 --2- 172.21.15.111:0/681588841 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f80fc0b5d90 0x7f80fc0cbdb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:11.025+0000 7f8103bff700 1 --2- 172.21.15.111:0/681588841 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80fc15c950 0x7f80fc1538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:11.025+0000 7f8103bff700 1 --2- 172.21.15.111:0/681588841 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f80fc0cf950 0x7f80fc15dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:11.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:11.025+0000 7f8103bff700 1 -- 172.21.15.111:0/681588841 >> 172.21.15.111:0/681588841 conn(0x7f80fc071b10 msgr2=0x7f80fc0b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:11.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:11.025+0000 7f8103bff700 1 -- 172.21.15.111:0/681588841 shutdown_connections 2023-03-29T10:58:11.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:11.025+0000 7f8103bff700 1 -- 172.21.15.111:0/681588841 wait complete. 2023-03-29T10:58:11.043 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-03-29T10:58:11.043 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-03-29T10:58:11.043 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-03-29T10:58:11.043 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-03-29T10:58:11.043 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-03-29T10:58:11.043 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-03-29T10:58:11.043 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-03-29T10:58:11.043 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-03-29T10:58:11.043 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-03-29T10:58:11.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:11.044 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-03-29T10:58:11.090 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-29T10:58:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-29T10:58:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-03-29T10:58:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-03-29T10:58:11.090 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:11.091 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-29T10:58:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-03-29T10:58:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-03-29T10:58:11.091 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-29T10:58:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-03-29T10:58:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-03-29T10:58:11.091 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-29T10:58:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-03-29T10:58:11.091 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-03-29T10:58:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-03-29T10:58:11.091 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-03-29T10:58:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42302sec preferred_lft 42302sec 2023-03-29T10:58:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-03-29T10:58:11.092 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:11.093 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-03-29T10:58:11.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:11.093 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-03-29T10:58:11.093 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-03-29T10:58:11.093 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add name ceph-brx type bridge 2023-03-29T10:58:11.093 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr flush dev ceph-brx 2023-03-29T10:58:11.093 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set ceph-brx up 2023-03-29T10:58:11.093 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-03-29T10:58:11.094 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-03-29T10:58:11.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:11.562 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-03-29T10:58:11.627 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-03-29T10:58:11.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:11.630 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-03-29T10:58:11.686 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-03-29T10:58:11.686 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-03-29T10:58:11.686 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-03-29T10:58:11.686 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-03-29T10:58:11.686 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-03-29T10:58:11.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:11.687 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-03-29T10:58:11.687 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-03-29T10:58:11.687 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-03-29T10:58:11.687 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-03-29T10:58:11.687 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-03-29T10:58:11.687 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-03-29T10:58:12.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:12.331 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-03-29T10:58:12.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:12.373 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list-id 2023-03-29T10:58:12.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:12.454 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-03-29T10:58:12.454 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-03-29T10:58:12.454 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-03-29T10:58:12.454 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-03-29T10:58:12.454 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-03-29T10:58:12.622 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-03-29T10:58:12.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:12.622 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-03-29T10:58:12.622 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-03-29T10:58:12.622 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-03-29T10:58:12.622 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-03-29T10:58:12.623 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-03-29T10:58:12.623 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-03-29T10:58:12.623 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-03-29T10:58:12.623 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-03-29T10:58:12.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:12.858 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-03-29T10:58:12.858 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-03-29T10:58:12.858 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-03-29T10:58:12.859 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set dev brx.0 master ceph-brx 2023-03-29T10:58:12.859 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-03-29T10:58:13.023 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-03-29T10:58:13.023 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-29T10:58:13.023 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-03-29T10:58:13.063 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-03-29T10:58:13.064 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-29T10:58:13.064 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-03-29T10:58:13.143 DEBUG:teuthology.orchestra.run.smithi119:> sudo modprobe fuse 2023-03-29T10:58:13.246 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-03-29T10:58:13.246 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-03-29T10:58:13.307 INFO:teuthology.orchestra.run.smithi119.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-03-29T10:58:13.309 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-03-29T10:58:13.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:13.310 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2023-03-29T10:58:13.350 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-03-29T10:58:13.350 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-03-29T10:58:13.394 DEBUG:teuthology.orchestra.run.smithi119:> sudo modprobe fuse 2023-03-29T10:58:13.457 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-03-29T10:58:13.457 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-03-29T10:58:13.520 INFO:teuthology.orchestra.run.smithi119.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-03-29T10:58:13.522 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-03-29T10:58:13.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:13.523 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2023-03-29T10:58:13.549 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi119.stderr:2023-03-29T10:58:13.550+0000 7f4c98f6e5c0 -1 init, newargv = 0x5581a1c11b50 newargc=15 2023-03-29T10:58:13.550 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi119.stderr:ceph-fuse[117360]: starting ceph client 2023-03-29T10:58:13.558 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi119.stderr:ceph-fuse[117360]: starting fuse 2023-03-29T10:58:13.567 INFO:teuthology.orchestra.run.smithi119.stdout:42 2023-03-29T10:58:13.567 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-03-29T10:58:13.568 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-03-29T10:58:13.568 DEBUG:teuthology.orchestra.run.smithi119:> import glob 2023-03-29T10:58:13.568 DEBUG:teuthology.orchestra.run.smithi119:> import re 2023-03-29T10:58:13.568 DEBUG:teuthology.orchestra.run.smithi119:> import os 2023-03-29T10:58:13.568 DEBUG:teuthology.orchestra.run.smithi119:> import subprocess 2023-03-29T10:58:13.568 DEBUG:teuthology.orchestra.run.smithi119:> 2023-03-29T10:58:13.568 DEBUG:teuthology.orchestra.run.smithi119:> def _find_admin_socket(client_name): 2023-03-29T10:58:13.568 DEBUG:teuthology.orchestra.run.smithi119:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-03-29T10:58:13.568 DEBUG:teuthology.orchestra.run.smithi119:> files = glob.glob(asok_path) 2023-03-29T10:58:13.568 DEBUG:teuthology.orchestra.run.smithi119:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-03-29T10:58:13.568 DEBUG:teuthology.orchestra.run.smithi119:> 2023-03-29T10:58:13.568 DEBUG:teuthology.orchestra.run.smithi119:> # Given a non-glob path, it better be there 2023-03-29T10:58:13.568 DEBUG:teuthology.orchestra.run.smithi119:> if "*" not in asok_path: 2023-03-29T10:58:13.568 DEBUG:teuthology.orchestra.run.smithi119:> assert(len(files) == 1) 2023-03-29T10:58:13.568 DEBUG:teuthology.orchestra.run.smithi119:> return files[0] 2023-03-29T10:58:13.568 DEBUG:teuthology.orchestra.run.smithi119:> 2023-03-29T10:58:13.568 DEBUG:teuthology.orchestra.run.smithi119:> for f in files: 2023-03-29T10:58:13.568 DEBUG:teuthology.orchestra.run.smithi119:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-03-29T10:58:13.569 DEBUG:teuthology.orchestra.run.smithi119:> if os.path.exists("/proc/{0}".format(pid)): 2023-03-29T10:58:13.569 DEBUG:teuthology.orchestra.run.smithi119:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-03-29T10:58:13.569 DEBUG:teuthology.orchestra.run.smithi119:> contents = proc_f.read() 2023-03-29T10:58:13.569 DEBUG:teuthology.orchestra.run.smithi119:> if mountpoint in contents: 2023-03-29T10:58:13.569 DEBUG:teuthology.orchestra.run.smithi119:> return f 2023-03-29T10:58:13.569 DEBUG:teuthology.orchestra.run.smithi119:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-03-29T10:58:13.569 DEBUG:teuthology.orchestra.run.smithi119:> 2023-03-29T10:58:13.569 DEBUG:teuthology.orchestra.run.smithi119:> print(_find_admin_socket("client.1")) 2023-03-29T10:58:13.569 DEBUG:teuthology.orchestra.run.smithi119:> ' 2023-03-29T10:58:13.741 INFO:teuthology.orchestra.run.smithi119.stdout:/var/run/ceph/ceph-client.1.117360.asok 2023-03-29T10:58:13.919 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.117360.asok 2023-03-29T10:58:13.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:13.919 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.117360.asok status 2023-03-29T10:58:14.131 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-03-29T10:58:14.131 INFO:teuthology.orchestra.run.smithi119.stdout: "metadata": { 2023-03-29T10:58:14.131 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_sha1": "f03d69a8741c605ae6711afe2c9991ac46095f7d", 2023-03-29T10:58:14.131 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_version": "ceph version 18.0.0-3066-gf03d69a8 (f03d69a8741c605ae6711afe2c9991ac46095f7d) reef (dev)", 2023-03-29T10:58:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "entity_id": "1", 2023-03-29T10:58:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "hostname": "smithi119", 2023-03-29T10:58:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-03-29T10:58:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "pid": "117360", 2023-03-29T10:58:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "root": "/" 2023-03-29T10:58:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-03-29T10:58:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "dentry_count": 0, 2023-03-29T10:58:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "dentry_pinned_count": 0, 2023-03-29T10:58:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 4631, 2023-03-29T10:58:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "inst": { 2023-03-29T10:58:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "name": { 2023-03-29T10:58:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "client", 2023-03-29T10:58:14.132 INFO:teuthology.orchestra.run.smithi119.stdout: "num": 4631 2023-03-29T10:58:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-03-29T10:58:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": { 2023-03-29T10:58:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2023-03-29T10:58:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "192.168.0.1:0", 2023-03-29T10:58:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 1481936705 2023-03-29T10:58:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-03-29T10:58:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-03-29T10:58:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": { 2023-03-29T10:58:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2023-03-29T10:58:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "192.168.0.1:0", 2023-03-29T10:58:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 1481936705 2023-03-29T10:58:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-03-29T10:58:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "inst_str": "client.4631 192.168.0.1:0/1481936705", 2023-03-29T10:58:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "addr_str": "192.168.0.1:0/1481936705", 2023-03-29T10:58:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "inode_count": 1, 2023-03-29T10:58:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_epoch": 9, 2023-03-29T10:58:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_epoch": 30, 2023-03-29T10:58:14.133 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_epoch_barrier": 0, 2023-03-29T10:58:14.134 INFO:teuthology.orchestra.run.smithi119.stdout: "blocklisted": false, 2023-03-29T10:58:14.134 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs" 2023-03-29T10:58:14.134 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-03-29T10:58:14.145 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-03-29T10:58:14.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.350+0000 7f29073ff700 1 Processor -- start 2023-03-29T10:58:14.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.350+0000 7f29073ff700 1 -- start start 2023-03-29T10:58:14.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.351+0000 7f29073ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2900071470 0x7f2900163ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:14.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.351+0000 7f29073ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29000cc9b0 con 0x7f2900158d30 2023-03-29T10:58:14.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.351+0000 7f29073ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29000ccc90 con 0x7f29000caa40 2023-03-29T10:58:14.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.351+0000 7f29073ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2900071dd0 con 0x7f2900071470 2023-03-29T10:58:14.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.351+0000 7f290519b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f2900158d30 0x7f29000ca330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58938/0 (socket says 172.21.15.111:58938) 2023-03-29T10:58:14.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.351+0000 7f290519b700 1 -- 172.21.15.111:0/1788487775 learned_addr learned my addr 172.21.15.111:0/1788487775 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:14.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.351+0000 7f290599c700 1 --2- 172.21.15.111:0/1788487775 >> v2:172.21.15.111:3300/0 conn(0x7f2900071470 0x7f2900163ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:14.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.352+0000 7f290599c700 1 -- 172.21.15.111:0/1788487775 >> v1:172.21.15.119:6789/0 conn(0x7f29000caa40 legacy=0x7f2900070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:14.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.352+0000 7f290599c700 1 -- 172.21.15.111:0/1788487775 >> v1:172.21.15.111:6789/0 conn(0x7f2900158d30 legacy=0x7f29000ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:14.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.352+0000 7f290599c700 1 -- 172.21.15.111:0/1788487775 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29000ccf70 con 0x7f2900071470 2023-03-29T10:58:14.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.352+0000 7f28f7fff700 1 -- 172.21.15.111:0/1788487775 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1085035092 0 0) 0x7f29000cc9b0 con 0x7f2900158d30 2023-03-29T10:58:14.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.352+0000 7f290599c700 1 --2- 172.21.15.111:0/1788487775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2900071470 0x7f2900163ab0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009d90 tx=0x7f28f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169520e9d6f39a75 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:14.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.352+0000 7f28f7fff700 1 -- 172.21.15.111:0/1788487775 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f800e070 con 0x7f2900071470 2023-03-29T10:58:14.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.352+0000 7f28f7fff700 1 -- 172.21.15.111:0/1788487775 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f800bbd0 con 0x7f2900071470 2023-03-29T10:58:14.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.352+0000 7f28f7fff700 1 -- 172.21.15.111:0/1788487775 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f8012430 con 0x7f2900071470 2023-03-29T10:58:14.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.353+0000 7f29073ff700 1 -- 172.21.15.111:0/1788487775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2900071470 msgr2=0x7f2900163ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:14.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.353+0000 7f29073ff700 1 --2- 172.21.15.111:0/1788487775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2900071470 0x7f2900163ab0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f28f8009d90 tx=0x7f28f800b330 comp rx=0 tx=0).stop 2023-03-29T10:58:14.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.353+0000 7f29073ff700 1 -- 172.21.15.111:0/1788487775 shutdown_connections 2023-03-29T10:58:14.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.353+0000 7f29073ff700 1 --2- 172.21.15.111:0/1788487775 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2900071470 0x7f2900163ab0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:14.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.353+0000 7f29073ff700 1 -- 172.21.15.111:0/1788487775 >> 172.21.15.111:0/1788487775 conn(0x7f29000cb6c0 msgr2=0x7f29000cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:14.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.353+0000 7f29073ff700 1 -- 172.21.15.111:0/1788487775 shutdown_connections 2023-03-29T10:58:14.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.353+0000 7f29073ff700 1 -- 172.21.15.111:0/1788487775 wait complete. 2023-03-29T10:58:14.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.355+0000 7f29073ff700 1 Processor -- start 2023-03-29T10:58:14.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.356+0000 7f29073ff700 1 -- start start 2023-03-29T10:58:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.357+0000 7f29073ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2900071470 0x7f29000c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.357+0000 7f29073ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2900158d30 0x7f290014da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.357+0000 7f29073ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2900148c30 0x7f29001490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.357+0000 7f29073ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290014c980 con 0x7f2900148c30 2023-03-29T10:58:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.357+0000 7f29073ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290014cc60 con 0x7f2900158d30 2023-03-29T10:58:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.357+0000 7f29073ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290014cf40 con 0x7f2900071470 2023-03-29T10:58:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.357+0000 7f290599c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2900148c30 0x7f29001490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.357+0000 7f290599c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2900148c30 0x7f29001490a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55684/0 (socket says 172.21.15.111:55684) 2023-03-29T10:58:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.357+0000 7f290599c700 1 -- 172.21.15.111:0/341652956 learned_addr learned my addr 172.21.15.111:0/341652956 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.357+0000 7f290519b700 1 --2- 172.21.15.111:0/341652956 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2900071470 0x7f29000c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:14.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.357+0000 7f290499a700 1 --2- 172.21.15.111:0/341652956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2900158d30 0x7f290014da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:14.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.357+0000 7f290599c700 1 -- 172.21.15.111:0/341652956 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2900071470 msgr2=0x7f29000c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:14.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.357+0000 7f290599c700 1 --2- 172.21.15.111:0/341652956 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2900071470 0x7f29000c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:14.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.357+0000 7f290599c700 1 -- 172.21.15.111:0/341652956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2900158d30 msgr2=0x7f290014da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:14.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.357+0000 7f290599c700 1 --2- 172.21.15.111:0/341652956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2900158d30 0x7f290014da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:14.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.358+0000 7f290599c700 1 -- 172.21.15.111:0/341652956 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f8009a70 con 0x7f2900148c30 2023-03-29T10:58:14.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.358+0000 7f290599c700 1 --2- 172.21.15.111:0/341652956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2900148c30 0x7f29001490a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f28f8014040 tx=0x7f28f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:14.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.358+0000 7f28f5ffb700 1 -- 172.21.15.111:0/341652956 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f801b810 con 0x7f2900148c30 2023-03-29T10:58:14.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.358+0000 7f28f5ffb700 1 -- 172.21.15.111:0/341652956 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f8020070 con 0x7f2900148c30 2023-03-29T10:58:14.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.358+0000 7f29073ff700 1 -- 172.21.15.111:0/341652956 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29000cd530 con 0x7f2900148c30 2023-03-29T10:58:14.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.358+0000 7f28f5ffb700 1 -- 172.21.15.111:0/341652956 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f801bbb0 con 0x7f2900148c30 2023-03-29T10:58:14.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.358+0000 7f29073ff700 1 -- 172.21.15.111:0/341652956 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29000cd250 con 0x7f2900148c30 2023-03-29T10:58:14.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.359+0000 7f29073ff700 1 -- 172.21.15.111:0/341652956 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29000cda50 con 0x7f2900148c30 2023-03-29T10:58:14.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.360+0000 7f28f5ffb700 1 -- 172.21.15.111:0/341652956 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f28f801bd10 con 0x7f2900148c30 2023-03-29T10:58:14.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.361+0000 7f28f5ffb700 1 --2- 172.21.15.111:0/341652956 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f28e4069de0 0x7f28e406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:14.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.361+0000 7f28f5ffb700 1 -- 172.21.15.111:0/341652956 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f28f8022070 con 0x7f2900148c30 2023-03-29T10:58:14.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.361+0000 7f290519b700 1 --2- 172.21.15.111:0/341652956 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f28e4069de0 0x7f28e406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:14.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.362+0000 7f290519b700 1 --2- 172.21.15.111:0/341652956 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f28e4069de0 0x7f28e406c290 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f28fc000f60 tx=0x7f28fc004a50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:14.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.365+0000 7f28f5ffb700 1 -- 172.21.15.111:0/341652956 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f28f8055db0 con 0x7f2900148c30 2023-03-29T10:58:14.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.602+0000 7f29073ff700 1 -- 172.21.15.111:0/341652956 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f290006b6d0 con 0x7f2900148c30 2023-03-29T10:58:14.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.603+0000 7f28f5ffb700 1 -- 172.21.15.111:0/341652956 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f28f80593d0 con 0x7f2900148c30 2023-03-29T10:58:14.604 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-03-29T10:58:14.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.605+0000 7f29073ff700 1 -- 172.21.15.111:0/341652956 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f28e4069de0 msgr2=0x7f28e406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:14.606 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.605+0000 7f29073ff700 1 --2- 172.21.15.111:0/341652956 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f28e4069de0 0x7f28e406c290 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f28fc000f60 tx=0x7f28fc004a50 comp rx=0 tx=0).stop 2023-03-29T10:58:14.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.605+0000 7f29073ff700 1 -- 172.21.15.111:0/341652956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2900148c30 msgr2=0x7f29001490a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:14.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.605+0000 7f29073ff700 1 --2- 172.21.15.111:0/341652956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2900148c30 0x7f29001490a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f28f8014040 tx=0x7f28f8004ad0 comp rx=0 tx=0).stop 2023-03-29T10:58:14.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.605+0000 7f29073ff700 1 -- 172.21.15.111:0/341652956 shutdown_connections 2023-03-29T10:58:14.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.605+0000 7f29073ff700 1 --2- 172.21.15.111:0/341652956 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f28e4069de0 0x7f28e406c290 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:14.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.605+0000 7f29073ff700 1 --2- 172.21.15.111:0/341652956 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2900148c30 0x7f29001490a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:14.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.605+0000 7f29073ff700 1 --2- 172.21.15.111:0/341652956 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2900071470 0x7f29000c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:14.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.605+0000 7f29073ff700 1 --2- 172.21.15.111:0/341652956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2900158d30 0x7f290014da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:14.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.605+0000 7f29073ff700 1 -- 172.21.15.111:0/341652956 >> 172.21.15.111:0/341652956 conn(0x7f29000cb6c0 msgr2=0x7f29000cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:14.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.606+0000 7f29073ff700 1 -- 172.21.15.111:0/341652956 shutdown_connections 2023-03-29T10:58:14.608 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:14.606+0000 7f29073ff700 1 -- 172.21.15.111:0/341652956 wait complete. 2023-03-29T10:58:14.623 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-03-29T10:58:14.624 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-03-29T10:58:14.624 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-03-29T10:58:14.624 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-03-29T10:58:14.624 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-03-29T10:58:14.624 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-03-29T10:58:14.624 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-03-29T10:58:14.624 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-03-29T10:58:14.624 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-03-29T10:58:14.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:14.624 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-03-29T10:58:14.668 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-29T10:58:14.668 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-29T10:58:14.668 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-03-29T10:58:14.668 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:14.668 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-03-29T10:58:14.668 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:14.668 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-29T10:58:14.669 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-03-29T10:58:14.669 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-03-29T10:58:14.669 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-29T10:58:14.669 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-03-29T10:58:14.669 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-03-29T10:58:14.669 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-29T10:58:14.669 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-03-29T10:58:14.669 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-03-29T10:58:14.669 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-03-29T10:58:14.669 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-03-29T10:58:14.669 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42298sec preferred_lft 42298sec 2023-03-29T10:58:14.669 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-03-29T10:58:14.669 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:14.669 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-03-29T10:58:14.669 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 1a:ab:b4:6b:87:43 brd ff:ff:ff:ff:ff:ff 2023-03-29T10:58:14.669 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-03-29T10:58:14.669 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:14.670 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::c431:91ff:fe0b:3d95/64 scope link 2023-03-29T10:58:14.670 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:14.670 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-03-29T10:58:14.670 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 1a:ab:b4:6b:87:43 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-03-29T10:58:14.670 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::18ab:b4ff:fe6b:8743/64 scope link 2023-03-29T10:58:14.670 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:14.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:14.670 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-03-29T10:58:14.769 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-03-29T10:58:14.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:14.772 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-03-29T10:58:14.814 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-03-29T10:58:14.814 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-03-29T10:58:14.814 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-03-29T10:58:14.814 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-03-29T10:58:14.815 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-03-29T10:58:14.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:14.816 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-03-29T10:58:14.816 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-03-29T10:58:14.816 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-03-29T10:58:14.816 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-03-29T10:58:14.816 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-03-29T10:58:14.816 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-03-29T10:58:15.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:15.005 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-03-29T10:58:15.045 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-03-29T10:58:15.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:15.046 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list-id 2023-03-29T10:58:15.125 INFO:teuthology.orchestra.run.smithi119.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-03-29T10:58:15.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:15.127 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-03-29T10:58:15.127 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-03-29T10:58:15.127 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-03-29T10:58:15.127 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-03-29T10:58:15.127 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-03-29T10:58:15.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:15.291 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-03-29T10:58:15.353 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-29T10:58:15.353 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-29T10:58:15.353 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-03-29T10:58:15.353 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:15.353 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-03-29T10:58:15.354 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:15.354 INFO:teuthology.orchestra.run.smithi119.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-03-29T10:58:15.354 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether ee:4e:aa:86:df:c6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-03-29T10:58:15.354 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-03-29T10:58:15.354 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:15.354 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4e:aaff:fe86:dfc6/64 scope link 2023-03-29T10:58:15.354 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:15.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:15.355 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-03-29T10:58:15.419 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-29T10:58:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-29T10:58:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-03-29T10:58:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-03-29T10:58:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:15.419 INFO:teuthology.orchestra.run.smithi119.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-03-29T10:58:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether ee:4e:aa:86:df:c6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-03-29T10:58:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-03-29T10:58:15.419 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4e:aaff:fe86:dfc6/64 scope link 2023-03-29T10:58:15.420 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:58:15.421 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-03-29T10:58:15.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:15.421 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-03-29T10:58:15.421 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-03-29T10:58:15.422 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-03-29T10:58:15.422 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-03-29T10:58:15.422 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-03-29T10:58:15.422 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-03-29T10:58:15.422 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-03-29T10:58:15.422 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-03-29T10:58:15.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:15.639 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-03-29T10:58:15.639 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-03-29T10:58:15.639 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.1 up 2023-03-29T10:58:15.639 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set dev brx.1 master ceph-brx 2023-03-29T10:58:15.639 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-03-29T10:58:15.802 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-03-29T10:58:15.803 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-29T10:58:15.803 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-03-29T10:58:15.843 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-03-29T10:58:15.843 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-29T10:58:15.844 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-03-29T10:58:15.924 DEBUG:teuthology.orchestra.run.smithi119:> sudo modprobe fuse 2023-03-29T10:58:16.028 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-03-29T10:58:16.028 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-03-29T10:58:16.088 INFO:teuthology.orchestra.run.smithi119.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-03-29T10:58:16.090 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-03-29T10:58:16.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:16.091 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2023-03-29T10:58:16.131 INFO:teuthology.orchestra.run.smithi119.stdout:42 2023-03-29T10:58:16.131 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2023-03-29T10:58:16.131 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-03-29T10:58:16.175 DEBUG:teuthology.orchestra.run.smithi119:> sudo modprobe fuse 2023-03-29T10:58:16.238 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-03-29T10:58:16.238 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-03-29T10:58:16.300 INFO:teuthology.orchestra.run.smithi119.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-03-29T10:58:16.302 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-03-29T10:58:16.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:16.303 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2023-03-29T10:58:16.335 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi119.stderr:2023-03-29T10:58:16.335+0000 7f066f5645c0 -1 init, newargv = 0x561190df8d50 newargc=15 2023-03-29T10:58:16.335 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi119.stderr:ceph-fuse[118035]: starting ceph client 2023-03-29T10:58:16.341 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi119.stderr:ceph-fuse[118035]: starting fuse 2023-03-29T10:58:16.350 INFO:teuthology.orchestra.run.smithi119.stdout:42 2023-03-29T10:58:16.350 INFO:teuthology.orchestra.run.smithi119.stdout:44 2023-03-29T10:58:16.350 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 44] 2023-03-29T10:58:16.351 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-03-29T10:58:16.351 DEBUG:teuthology.orchestra.run.smithi119:> import glob 2023-03-29T10:58:16.351 DEBUG:teuthology.orchestra.run.smithi119:> import re 2023-03-29T10:58:16.351 DEBUG:teuthology.orchestra.run.smithi119:> import os 2023-03-29T10:58:16.351 DEBUG:teuthology.orchestra.run.smithi119:> import subprocess 2023-03-29T10:58:16.351 DEBUG:teuthology.orchestra.run.smithi119:> 2023-03-29T10:58:16.351 DEBUG:teuthology.orchestra.run.smithi119:> def _find_admin_socket(client_name): 2023-03-29T10:58:16.351 DEBUG:teuthology.orchestra.run.smithi119:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-03-29T10:58:16.351 DEBUG:teuthology.orchestra.run.smithi119:> files = glob.glob(asok_path) 2023-03-29T10:58:16.351 DEBUG:teuthology.orchestra.run.smithi119:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-03-29T10:58:16.351 DEBUG:teuthology.orchestra.run.smithi119:> 2023-03-29T10:58:16.351 DEBUG:teuthology.orchestra.run.smithi119:> # Given a non-glob path, it better be there 2023-03-29T10:58:16.351 DEBUG:teuthology.orchestra.run.smithi119:> if "*" not in asok_path: 2023-03-29T10:58:16.351 DEBUG:teuthology.orchestra.run.smithi119:> assert(len(files) == 1) 2023-03-29T10:58:16.352 DEBUG:teuthology.orchestra.run.smithi119:> return files[0] 2023-03-29T10:58:16.352 DEBUG:teuthology.orchestra.run.smithi119:> 2023-03-29T10:58:16.352 DEBUG:teuthology.orchestra.run.smithi119:> for f in files: 2023-03-29T10:58:16.352 DEBUG:teuthology.orchestra.run.smithi119:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-03-29T10:58:16.352 DEBUG:teuthology.orchestra.run.smithi119:> if os.path.exists("/proc/{0}".format(pid)): 2023-03-29T10:58:16.352 DEBUG:teuthology.orchestra.run.smithi119:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-03-29T10:58:16.352 DEBUG:teuthology.orchestra.run.smithi119:> contents = proc_f.read() 2023-03-29T10:58:16.352 DEBUG:teuthology.orchestra.run.smithi119:> if mountpoint in contents: 2023-03-29T10:58:16.352 DEBUG:teuthology.orchestra.run.smithi119:> return f 2023-03-29T10:58:16.352 DEBUG:teuthology.orchestra.run.smithi119:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-03-29T10:58:16.352 DEBUG:teuthology.orchestra.run.smithi119:> 2023-03-29T10:58:16.352 DEBUG:teuthology.orchestra.run.smithi119:> print(_find_admin_socket("client.3")) 2023-03-29T10:58:16.352 DEBUG:teuthology.orchestra.run.smithi119:> ' 2023-03-29T10:58:16.526 INFO:teuthology.orchestra.run.smithi119.stdout:/var/run/ceph/ceph-client.3.118035.asok 2023-03-29T10:58:16.704 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.118035.asok 2023-03-29T10:58:16.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:16.705 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.118035.asok status 2023-03-29T10:58:16.918 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-03-29T10:58:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "metadata": { 2023-03-29T10:58:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_sha1": "f03d69a8741c605ae6711afe2c9991ac46095f7d", 2023-03-29T10:58:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_version": "ceph version 18.0.0-3066-gf03d69a8 (f03d69a8741c605ae6711afe2c9991ac46095f7d) reef (dev)", 2023-03-29T10:58:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "entity_id": "3", 2023-03-29T10:58:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "hostname": "smithi119", 2023-03-29T10:58:16.919 INFO:teuthology.orchestra.run.smithi119.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-03-29T10:58:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "pid": "118035", 2023-03-29T10:58:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "root": "/" 2023-03-29T10:58:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-03-29T10:58:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "dentry_count": 0, 2023-03-29T10:58:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "dentry_pinned_count": 0, 2023-03-29T10:58:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 4637, 2023-03-29T10:58:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "inst": { 2023-03-29T10:58:16.920 INFO:teuthology.orchestra.run.smithi119.stdout: "name": { 2023-03-29T10:58:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "client", 2023-03-29T10:58:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "num": 4637 2023-03-29T10:58:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-03-29T10:58:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": { 2023-03-29T10:58:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2023-03-29T10:58:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "192.168.0.2:0", 2023-03-29T10:58:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 3391381988 2023-03-29T10:58:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-03-29T10:58:16.921 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-03-29T10:58:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": { 2023-03-29T10:58:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2023-03-29T10:58:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "192.168.0.2:0", 2023-03-29T10:58:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 3391381988 2023-03-29T10:58:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-03-29T10:58:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "inst_str": "client.4637 192.168.0.2:0/3391381988", 2023-03-29T10:58:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "addr_str": "192.168.0.2:0/3391381988", 2023-03-29T10:58:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "inode_count": 1, 2023-03-29T10:58:16.922 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_epoch": 9, 2023-03-29T10:58:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_epoch": 30, 2023-03-29T10:58:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_epoch_barrier": 0, 2023-03-29T10:58:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: "blocklisted": false, 2023-03-29T10:58:16.923 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs" 2023-03-29T10:58:16.923 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-03-29T10:58:16.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:16.932 DEBUG:teuthology.orchestra.run.smithi111:> stat --file-system '--printf=%T 2023-03-29T10:58:16.932 DEBUG:teuthology.orchestra.run.smithi111:> ' -- /home/ubuntu/cephtest/mnt.0 2023-03-29T10:58:16.983 INFO:teuthology.orchestra.run.smithi111.stdout:fuseblk 2023-03-29T10:58:16.983 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-03-29T10:58:16.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:16.983 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-03-29T10:58:17.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:17.057 DEBUG:teuthology.orchestra.run.smithi111:> stat --file-system '--printf=%T 2023-03-29T10:58:17.057 DEBUG:teuthology.orchestra.run.smithi111:> ' -- /home/ubuntu/cephtest/mnt.2 2023-03-29T10:58:17.103 INFO:teuthology.orchestra.run.smithi111.stdout:fuseblk 2023-03-29T10:58:17.103 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-03-29T10:58:17.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:17.104 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-03-29T10:58:17.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:17.206 DEBUG:teuthology.orchestra.run.smithi119:> stat --file-system '--printf=%T 2023-03-29T10:58:17.206 DEBUG:teuthology.orchestra.run.smithi119:> ' -- /home/ubuntu/cephtest/mnt.1 2023-03-29T10:58:17.248 INFO:teuthology.orchestra.run.smithi119.stdout:fuseblk 2023-03-29T10:58:17.248 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-03-29T10:58:17.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:17.249 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-03-29T10:58:17.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:17.351 DEBUG:teuthology.orchestra.run.smithi119:> stat --file-system '--printf=%T 2023-03-29T10:58:17.351 DEBUG:teuthology.orchestra.run.smithi119:> ' -- /home/ubuntu/cephtest/mnt.3 2023-03-29T10:58:17.392 INFO:teuthology.orchestra.run.smithi119.stdout:fuseblk 2023-03-29T10:58:17.392 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-03-29T10:58:17.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:17.392 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-03-29T10:58:17.493 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-03-29T10:58:17.511 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:58:17.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.696+0000 7f2f2dfff700 1 Processor -- start 2023-03-29T10:58:17.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.696+0000 7f2f2dfff700 1 -- start start 2023-03-29T10:58:17.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.697+0000 7f2f2dfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2f280c99a0 0x7f2f281652f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:17.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.697+0000 7f2f2dfff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f280b16b0 con 0x7f2f28143f10 2023-03-29T10:58:17.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.697+0000 7f2f2dfff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2806d970 con 0x7f2f280b0dd0 2023-03-29T10:58:17.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.697+0000 7f2f2dfff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f2815ac50 con 0x7f2f280c99a0 2023-03-29T10:58:17.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.697+0000 7f2f27fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2f280c99a0 0x7f2f281652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:17.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.698+0000 7f2f277fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f2f28143f10 0x7f2f280b06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:58950/0 (socket says 172.21.15.111:58950) 2023-03-29T10:58:17.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.698+0000 7f2f277fe700 1 -- 172.21.15.111:0/571196079 learned_addr learned my addr 172.21.15.111:0/571196079 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:17.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.698+0000 7f2f27fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f2f280c99a0 0x7f2f281652f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55692/0 (socket says 172.21.15.111:55692) 2023-03-29T10:58:17.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.698+0000 7f2f27fff700 1 -- 172.21.15.111:0/571196079 >> v1:172.21.15.119:6789/0 conn(0x7f2f280b0dd0 legacy=0x7f2f280c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:17.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.698+0000 7f2f27fff700 1 -- 172.21.15.111:0/571196079 >> v1:172.21.15.111:6789/0 conn(0x7f2f28143f10 legacy=0x7f2f280b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:17.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.698+0000 7f2f27fff700 1 -- 172.21.15.111:0/571196079 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2806dc50 con 0x7f2f280c99a0 2023-03-29T10:58:17.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.698+0000 7f2f267fc700 1 -- 172.21.15.111:0/571196079 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 241584286 0 0) 0x7f2f280b16b0 con 0x7f2f28143f10 2023-03-29T10:58:17.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.698+0000 7f2f27fff700 1 --2- 172.21.15.111:0/571196079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f280c99a0 0x7f2f281652f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009d90 tx=0x7f2f2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b438fffb437a7741 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:17.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.699+0000 7f2f267fc700 1 -- 172.21.15.111:0/571196079 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2000e070 con 0x7f2f280c99a0 2023-03-29T10:58:17.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.699+0000 7f2f267fc700 1 -- 172.21.15.111:0/571196079 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2000bbd0 con 0x7f2f280c99a0 2023-03-29T10:58:17.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.699+0000 7f2f267fc700 1 -- 172.21.15.111:0/571196079 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f20012430 con 0x7f2f280c99a0 2023-03-29T10:58:17.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.699+0000 7f2f2dfff700 1 -- 172.21.15.111:0/571196079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f280c99a0 msgr2=0x7f2f281652f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:17.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.699+0000 7f2f2dfff700 1 --2- 172.21.15.111:0/571196079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f280c99a0 0x7f2f281652f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009d90 tx=0x7f2f2000b330 comp rx=0 tx=0).stop 2023-03-29T10:58:17.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.699+0000 7f2f2dfff700 1 -- 172.21.15.111:0/571196079 shutdown_connections 2023-03-29T10:58:17.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.699+0000 7f2f2dfff700 1 --2- 172.21.15.111:0/571196079 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f280c99a0 0x7f2f281652f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:17.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.699+0000 7f2f2dfff700 1 -- 172.21.15.111:0/571196079 >> 172.21.15.111:0/571196079 conn(0x7f2f280cb6c0 msgr2=0x7f2f280cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:17.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.699+0000 7f2f2dfff700 1 -- 172.21.15.111:0/571196079 shutdown_connections 2023-03-29T10:58:17.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.699+0000 7f2f2dfff700 1 -- 172.21.15.111:0/571196079 wait complete. 2023-03-29T10:58:17.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.701+0000 7f2f2dfff700 1 Processor -- start 2023-03-29T10:58:17.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.701+0000 7f2f2dfff700 1 -- start start 2023-03-29T10:58:17.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.702+0000 7f2f2dfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f280b0dd0 0x7f2f28071200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:17.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.702+0000 7f2f2dfff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f280c99a0 0x7f2f28071740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:17.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.702+0000 7f2f2dfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f28143f10 0x7f2f280c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:17.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.702+0000 7f2f2dfff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f280a9b60 con 0x7f2f280b0dd0 2023-03-29T10:58:17.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.702+0000 7f2f2dfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f280a74a0 con 0x7f2f28143f10 2023-03-29T10:58:17.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.702+0000 7f2f2dfff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f280a7780 con 0x7f2f280c99a0 2023-03-29T10:58:17.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.702+0000 7f2f277fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f280b0dd0 0x7f2f28071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:17.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.702+0000 7f2f277fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f280b0dd0 0x7f2f28071200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:55704/0 (socket says 172.21.15.111:55704) 2023-03-29T10:58:17.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.702+0000 7f2f277fe700 1 -- 172.21.15.111:0/1225680249 learned_addr learned my addr 172.21.15.111:0/1225680249 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:17.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.702+0000 7f2f27fff700 1 --2- 172.21.15.111:0/1225680249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f28143f10 0x7f2f280c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:17.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.702+0000 7f2f26ffd700 1 --2- 172.21.15.111:0/1225680249 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f280c99a0 0x7f2f28071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:17.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.703+0000 7f2f277fe700 1 -- 172.21.15.111:0/1225680249 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f280c99a0 msgr2=0x7f2f28071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:17.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.703+0000 7f2f277fe700 1 --2- 172.21.15.111:0/1225680249 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f280c99a0 0x7f2f28071740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:17.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.703+0000 7f2f277fe700 1 -- 172.21.15.111:0/1225680249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f28143f10 msgr2=0x7f2f280c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:17.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.703+0000 7f2f277fe700 1 --2- 172.21.15.111:0/1225680249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f28143f10 0x7f2f280c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:17.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.703+0000 7f2f277fe700 1 -- 172.21.15.111:0/1225680249 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f20009a70 con 0x7f2f280b0dd0 2023-03-29T10:58:17.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.703+0000 7f2f277fe700 1 --2- 172.21.15.111:0/1225680249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f280b0dd0 0x7f2f28071200 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800c520 tx=0x7f2f1800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:17.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.703+0000 7f2f0ffff700 1 -- 172.21.15.111:0/1225680249 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18004c80 con 0x7f2f280b0dd0 2023-03-29T10:58:17.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.703+0000 7f2f0ffff700 1 -- 172.21.15.111:0/1225680249 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f18004e00 con 0x7f2f280b0dd0 2023-03-29T10:58:17.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.703+0000 7f2f2dfff700 1 -- 172.21.15.111:0/1225680249 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2806e210 con 0x7f2f280b0dd0 2023-03-29T10:58:17.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.703+0000 7f2f0ffff700 1 -- 172.21.15.111:0/1225680249 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f18007310 con 0x7f2f280b0dd0 2023-03-29T10:58:17.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.703+0000 7f2f2dfff700 1 -- 172.21.15.111:0/1225680249 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2806df30 con 0x7f2f280b0dd0 2023-03-29T10:58:17.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.704+0000 7f2f2dfff700 1 -- 172.21.15.111:0/1225680249 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f281442e0 con 0x7f2f280b0dd0 2023-03-29T10:58:17.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.705+0000 7f2f0ffff700 1 -- 172.21.15.111:0/1225680249 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f2f18007470 con 0x7f2f280b0dd0 2023-03-29T10:58:17.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.706+0000 7f2f0ffff700 1 --2- 172.21.15.111:0/1225680249 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2f14069de0 0x7f2f1406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:17.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.706+0000 7f2f0ffff700 1 -- 172.21.15.111:0/1225680249 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2f18087fa0 con 0x7f2f280b0dd0 2023-03-29T10:58:17.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.706+0000 7f2f26ffd700 1 --2- 172.21.15.111:0/1225680249 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2f14069de0 0x7f2f1406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:17.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.707+0000 7f2f26ffd700 1 --2- 172.21.15.111:0/1225680249 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2f14069de0 0x7f2f1406c290 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2f281f3630 tx=0x7f2f1c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:17.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.710+0000 7f2f0ffff700 1 -- 172.21.15.111:0/1225680249 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f2f18053480 con 0x7f2f280b0dd0 2023-03-29T10:58:17.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.904+0000 7f2f2dfff700 1 -- 172.21.15.111:0/1225680249 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f28074430 con 0x7f2f280b0dd0 2023-03-29T10:58:17.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.905+0000 7f2f0ffff700 1 -- 172.21.15.111:0/1225680249 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19892 (secure 0 0 0) 0x7f2f18056aa0 con 0x7f2f280b0dd0 2023-03-29T10:58:17.906 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:17.906 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":30,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:58:09.476334+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.26","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:58:17.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.908+0000 7f2f2dfff700 1 -- 172.21.15.111:0/1225680249 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2f14069de0 msgr2=0x7f2f1406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:17.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.908+0000 7f2f2dfff700 1 --2- 172.21.15.111:0/1225680249 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2f14069de0 0x7f2f1406c290 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f2f281f3630 tx=0x7f2f1c009040 comp rx=0 tx=0).stop 2023-03-29T10:58:17.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.908+0000 7f2f2dfff700 1 -- 172.21.15.111:0/1225680249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f280b0dd0 msgr2=0x7f2f28071200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:17.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.908+0000 7f2f2dfff700 1 --2- 172.21.15.111:0/1225680249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f280b0dd0 0x7f2f28071200 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800c520 tx=0x7f2f1800c830 comp rx=0 tx=0).stop 2023-03-29T10:58:17.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.908+0000 7f2f2dfff700 1 -- 172.21.15.111:0/1225680249 shutdown_connections 2023-03-29T10:58:17.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.908+0000 7f2f2dfff700 1 --2- 172.21.15.111:0/1225680249 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2f14069de0 0x7f2f1406c290 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:17.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.908+0000 7f2f2dfff700 1 --2- 172.21.15.111:0/1225680249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f280b0dd0 0x7f2f28071200 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:17.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.908+0000 7f2f2dfff700 1 --2- 172.21.15.111:0/1225680249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f28143f10 0x7f2f280c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:17.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.908+0000 7f2f2dfff700 1 --2- 172.21.15.111:0/1225680249 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f280c99a0 0x7f2f28071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:17.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.908+0000 7f2f2dfff700 1 -- 172.21.15.111:0/1225680249 >> 172.21.15.111:0/1225680249 conn(0x7f2f280cb6c0 msgr2=0x7f2f280b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:17.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.908+0000 7f2f2dfff700 1 -- 172.21.15.111:0/1225680249 shutdown_connections 2023-03-29T10:58:17.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:17.908+0000 7f2f2dfff700 1 -- 172.21.15.111:0/1225680249 wait complete. 2023-03-29T10:58:17.926 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-29T10:57:16.183979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-29T10:57:21.239809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-29T10:57:26.192124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-29T10:57:26.796777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-29T10:57:29.874795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-29T10:58:17.926 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-29T10:58:18.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.112+0000 7f45dbfff700 1 Processor -- start 2023-03-29T10:58:18.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.112+0000 7f45dbfff700 1 -- start start 2023-03-29T10:58:18.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.113+0000 7f45dbfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f45d40cfbf0 0x7f45d41608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:18.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.113+0000 7f45dbfff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45d40d1100 0x7f45d40d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:18.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.113+0000 7f45dbfff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d40d0020 con 0x7f45d40b87a0 2023-03-29T10:58:18.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.114+0000 7f45dbfff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d40d02d0 con 0x7f45d40cfbf0 2023-03-29T10:58:18.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.114+0000 7f45dbfff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d40d0cb0 con 0x7f45d40d1100 2023-03-29T10:58:18.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.114+0000 7f45d959a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f45d40cfbf0 0x7f45d41608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:18.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.114+0000 7f45d959a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f45d40cfbf0 0x7f45d41608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52740/0 (socket says 172.21.15.111:52740) 2023-03-29T10:58:18.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.114+0000 7f45d959a700 1 -- 172.21.15.111:0/3964716432 learned_addr learned my addr 172.21.15.111:0/3964716432 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:18.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.114+0000 7f45d9d9b700 1 --1- 172.21.15.111:0/3964716432 >> v1:172.21.15.111:6789/0 conn(0x7f45d40b87a0 0x7f45d41601e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:39818/0 (socket says 172.21.15.111:39818) 2023-03-29T10:58:18.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.114+0000 7f45da59c700 1 --2- 172.21.15.111:0/3964716432 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45d40d1100 0x7f45d40d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:18.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.114+0000 7f45d959a700 1 -- 172.21.15.111:0/3964716432 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45d40d1100 msgr2=0x7f45d40d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:18.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.114+0000 7f45d959a700 1 --2- 172.21.15.111:0/3964716432 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45d40d1100 0x7f45d40d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.114+0000 7f45d959a700 1 -- 172.21.15.111:0/3964716432 >> v1:172.21.15.111:6789/0 conn(0x7f45d40b87a0 legacy=0x7f45d41601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:18.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.114+0000 7f45d8d99700 1 -- 172.21.15.111:0/3964716432 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685378308 0 0) 0x7f45d40d0020 con 0x7f45d40b87a0 2023-03-29T10:58:18.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.114+0000 7f45d959a700 1 -- 172.21.15.111:0/3964716432 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d40cdc60 con 0x7f45d40cfbf0 2023-03-29T10:58:18.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.114+0000 7f45da59c700 1 --2- 172.21.15.111:0/3964716432 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45d40d1100 0x7f45d40d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:18.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.114+0000 7f45d959a700 1 --2- 172.21.15.111:0/3964716432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45d40cfbf0 0x7f45d41608f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f45c40098f0 tx=0x7f45c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0c6bf736d9aa20 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:18.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.115+0000 7f45d8d99700 1 -- 172.21.15.111:0/3964716432 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c400b8a0 con 0x7f45d40cfbf0 2023-03-29T10:58:18.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.115+0000 7f45d8d99700 1 -- 172.21.15.111:0/3964716432 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c400ba00 con 0x7f45d40cfbf0 2023-03-29T10:58:18.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.115+0000 7f45d8d99700 1 -- 172.21.15.111:0/3964716432 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c400bd90 con 0x7f45d40cfbf0 2023-03-29T10:58:18.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.115+0000 7f45dbfff700 1 -- 172.21.15.111:0/3964716432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45d40cfbf0 msgr2=0x7f45d41608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:18.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.115+0000 7f45dbfff700 1 --2- 172.21.15.111:0/3964716432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45d40cfbf0 0x7f45d41608f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f45c40098f0 tx=0x7f45c4009c00 comp rx=0 tx=0).stop 2023-03-29T10:58:18.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.115+0000 7f45dbfff700 1 -- 172.21.15.111:0/3964716432 shutdown_connections 2023-03-29T10:58:18.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.115+0000 7f45dbfff700 1 --2- 172.21.15.111:0/3964716432 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45d40d1100 0x7f45d40d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.115+0000 7f45dbfff700 1 --2- 172.21.15.111:0/3964716432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45d40cfbf0 0x7f45d41608f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.115+0000 7f45dbfff700 1 -- 172.21.15.111:0/3964716432 >> 172.21.15.111:0/3964716432 conn(0x7f45d4071b10 msgr2=0x7f45d4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:18.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.115+0000 7f45dbfff700 1 -- 172.21.15.111:0/3964716432 shutdown_connections 2023-03-29T10:58:18.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.115+0000 7f45dbfff700 1 -- 172.21.15.111:0/3964716432 wait complete. 2023-03-29T10:58:18.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.117+0000 7f45dbfff700 1 Processor -- start 2023-03-29T10:58:18.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.117+0000 7f45dbfff700 1 -- start start 2023-03-29T10:58:18.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.118+0000 7f45dbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45d40b87a0 0x7f45d4150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:18.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.118+0000 7f45dbfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45d40cfbf0 0x7f45d4159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:18.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.118+0000 7f45dbfff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45d40d1100 0x7f45d41f5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:18.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.118+0000 7f45dbfff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d4159890 con 0x7f45d40cfbf0 2023-03-29T10:58:18.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.118+0000 7f45dbfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d415ee10 con 0x7f45d40b87a0 2023-03-29T10:58:18.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.118+0000 7f45dbfff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d4158ec0 con 0x7f45d40d1100 2023-03-29T10:58:18.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.118+0000 7f45d959a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45d40cfbf0 0x7f45d4159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:18.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.118+0000 7f45d959a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45d40cfbf0 0x7f45d4159350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52746/0 (socket says 172.21.15.111:52746) 2023-03-29T10:58:18.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.118+0000 7f45d959a700 1 -- 172.21.15.111:0/4130936899 learned_addr learned my addr 172.21.15.111:0/4130936899 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:18.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.118+0000 7f45d9d9b700 1 --2- 172.21.15.111:0/4130936899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45d40b87a0 0x7f45d4150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:18.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.118+0000 7f45da59c700 1 --2- 172.21.15.111:0/4130936899 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45d40d1100 0x7f45d41f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:18.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.119+0000 7f45d959a700 1 -- 172.21.15.111:0/4130936899 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45d40d1100 msgr2=0x7f45d41f5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:18.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.119+0000 7f45d959a700 1 --2- 172.21.15.111:0/4130936899 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45d40d1100 0x7f45d41f5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.119+0000 7f45d959a700 1 -- 172.21.15.111:0/4130936899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45d40b87a0 msgr2=0x7f45d4150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:18.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.119+0000 7f45d959a700 1 --2- 172.21.15.111:0/4130936899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45d40b87a0 0x7f45d4150cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.119+0000 7f45d959a700 1 -- 172.21.15.111:0/4130936899 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c4009550 con 0x7f45d40cfbf0 2023-03-29T10:58:18.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.119+0000 7f45d9d9b700 1 --2- 172.21.15.111:0/4130936899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45d40b87a0 0x7f45d4150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:18.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.119+0000 7f45d959a700 1 --2- 172.21.15.111:0/4130936899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45d40cfbf0 0x7f45d4159350 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f45d4072760 tx=0x7f45c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:18.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.119+0000 7f45ca7fc700 1 -- 172.21.15.111:0/4130936899 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c40036a0 con 0x7f45d40cfbf0 2023-03-29T10:58:18.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.119+0000 7f45ca7fc700 1 -- 172.21.15.111:0/4130936899 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c4003800 con 0x7f45d40cfbf0 2023-03-29T10:58:18.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.119+0000 7f45ca7fc700 1 -- 172.21.15.111:0/4130936899 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c4003b70 con 0x7f45d40cfbf0 2023-03-29T10:58:18.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.119+0000 7f45dbfff700 1 -- 172.21.15.111:0/4130936899 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d40cd980 con 0x7f45d40cfbf0 2023-03-29T10:58:18.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.119+0000 7f45dbfff700 1 -- 172.21.15.111:0/4130936899 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d40cd6a0 con 0x7f45d40cfbf0 2023-03-29T10:58:18.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.120+0000 7f45dbfff700 1 -- 172.21.15.111:0/4130936899 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d4160e30 con 0x7f45d40cfbf0 2023-03-29T10:58:18.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.121+0000 7f45ca7fc700 1 -- 172.21.15.111:0/4130936899 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f45c4003cd0 con 0x7f45d40cfbf0 2023-03-29T10:58:18.123 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.122+0000 7f45ca7fc700 1 --2- 172.21.15.111:0/4130936899 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f45b8069de0 0x7f45b806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:18.123 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.122+0000 7f45ca7fc700 1 -- 172.21.15.111:0/4130936899 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f45c4021030 con 0x7f45d40cfbf0 2023-03-29T10:58:18.123 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.123+0000 7f45d9d9b700 1 --2- 172.21.15.111:0/4130936899 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f45b8069de0 0x7f45b806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:18.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.123+0000 7f45d9d9b700 1 --2- 172.21.15.111:0/4130936899 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f45b8069de0 0x7f45b806c290 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f45d0000f60 tx=0x7f45d0007cc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:18.127 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.127+0000 7f45ca7fc700 1 -- 172.21.15.111:0/4130936899 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f45c40550d0 con 0x7f45d40cfbf0 2023-03-29T10:58:18.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.321+0000 7f45dbfff700 1 -- 172.21.15.111:0/4130936899 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f45d4074430 con 0x7f45d40cfbf0 2023-03-29T10:58:18.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.321+0000 7f45ca7fc700 1 -- 172.21.15.111:0/4130936899 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f45c402b020 con 0x7f45d40cfbf0 2023-03-29T10:58:18.322 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-03-29T10:58:18.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.323+0000 7f45dbfff700 1 -- 172.21.15.111:0/4130936899 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f45b8069de0 msgr2=0x7f45b806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:18.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.323+0000 7f45dbfff700 1 --2- 172.21.15.111:0/4130936899 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f45b8069de0 0x7f45b806c290 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f45d0000f60 tx=0x7f45d0007cc0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.323+0000 7f45dbfff700 1 -- 172.21.15.111:0/4130936899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45d40cfbf0 msgr2=0x7f45d4159350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:18.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.323+0000 7f45dbfff700 1 --2- 172.21.15.111:0/4130936899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45d40cfbf0 0x7f45d4159350 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f45d4072760 tx=0x7f45c4004750 comp rx=0 tx=0).stop 2023-03-29T10:58:18.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.324+0000 7f45dbfff700 1 -- 172.21.15.111:0/4130936899 shutdown_connections 2023-03-29T10:58:18.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.324+0000 7f45dbfff700 1 --2- 172.21.15.111:0/4130936899 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f45b8069de0 0x7f45b806c290 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.324+0000 7f45dbfff700 1 --2- 172.21.15.111:0/4130936899 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45d40d1100 0x7f45d41f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.324+0000 7f45dbfff700 1 --2- 172.21.15.111:0/4130936899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45d40b87a0 0x7f45d4150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.324+0000 7f45dbfff700 1 --2- 172.21.15.111:0/4130936899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45d40cfbf0 0x7f45d4159350 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.324+0000 7f45dbfff700 1 -- 172.21.15.111:0/4130936899 >> 172.21.15.111:0/4130936899 conn(0x7f45d4071b10 msgr2=0x7f45d40b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:18.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.324+0000 7f45dbfff700 1 -- 172.21.15.111:0/4130936899 shutdown_connections 2023-03-29T10:58:18.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.324+0000 7f45dbfff700 1 -- 172.21.15.111:0/4130936899 wait complete. 2023-03-29T10:58:18.342 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-29T10:58:18.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.528+0000 7fe9321ff700 1 Processor -- start 2023-03-29T10:58:18.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.529+0000 7fe9321ff700 1 -- start start 2023-03-29T10:58:18.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.530+0000 7fe9321ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe92c15c590 0x7fe92c160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:18.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.530+0000 7fe9321ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe92c0cf010 0x7fe92c0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:18.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.530+0000 7fe9321ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92c0cd520 con 0x7fe92c0cfd80 2023-03-29T10:58:18.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.530+0000 7fe9321ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe92c0d2640 con 0x7fe92c15c590 2023-03-29T10:58:18.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.530+0000 7fe9321ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe92c1610d0 con 0x7fe92c0cf010 2023-03-29T10:58:18.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.530+0000 7fe92b7fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe92c15c590 0x7fe92c160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:18.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.530+0000 7fe92b7fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe92c15c590 0x7fe92c160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52760/0 (socket says 172.21.15.111:52760) 2023-03-29T10:58:18.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.530+0000 7fe92b7fe700 1 -- 172.21.15.111:0/433266004 learned_addr learned my addr 172.21.15.111:0/433266004 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:18.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.530+0000 7fe92affd700 1 --2- 172.21.15.111:0/433266004 >> v2:172.21.15.119:3300/0 conn(0x7fe92c0cf010 0x7fe92c0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:18.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.530+0000 7fe92b7fe700 1 -- 172.21.15.111:0/433266004 >> v2:172.21.15.119:3300/0 conn(0x7fe92c0cf010 msgr2=0x7fe92c0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:18.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.530+0000 7fe92b7fe700 1 --2- 172.21.15.111:0/433266004 >> v2:172.21.15.119:3300/0 conn(0x7fe92c0cf010 0x7fe92c0cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.530+0000 7fe92b7fe700 1 -- 172.21.15.111:0/433266004 >> v1:172.21.15.119:6789/0 conn(0x7fe92c0cfd80 legacy=0x7fe92c0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:18.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.530+0000 7fe92b7fe700 1 -- 172.21.15.111:0/433266004 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c0b23c0 con 0x7fe92c15c590 2023-03-29T10:58:18.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.531+0000 7fe92b7fe700 1 --2- 172.21.15.111:0/433266004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c15c590 0x7fe92c160b90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0b4ee0 tx=0x7fe91c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527b759884de62c9 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:18.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.531+0000 7fe92a7fc700 1 -- 172.21.15.111:0/433266004 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c00b870 con 0x7fe92c15c590 2023-03-29T10:58:18.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.531+0000 7fe92a7fc700 1 -- 172.21.15.111:0/433266004 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91c00b9d0 con 0x7fe92c15c590 2023-03-29T10:58:18.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.531+0000 7fe92a7fc700 1 -- 172.21.15.111:0/433266004 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c00bd60 con 0x7fe92c15c590 2023-03-29T10:58:18.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.531+0000 7fe9321ff700 1 -- 172.21.15.111:0/433266004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c15c590 msgr2=0x7fe92c160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:18.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.531+0000 7fe9321ff700 1 --2- 172.21.15.111:0/433266004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c15c590 0x7fe92c160b90 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe92c0b4ee0 tx=0x7fe91c009bd0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.531+0000 7fe9321ff700 1 -- 172.21.15.111:0/433266004 shutdown_connections 2023-03-29T10:58:18.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.531+0000 7fe9321ff700 1 --2- 172.21.15.111:0/433266004 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c15c590 0x7fe92c160b90 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.531+0000 7fe9321ff700 1 --2- 172.21.15.111:0/433266004 >> v2:172.21.15.119:3300/0 conn(0x7fe92c0cf010 0x7fe92c0cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.531+0000 7fe9321ff700 1 -- 172.21.15.111:0/433266004 >> 172.21.15.111:0/433266004 conn(0x7fe92c071b10 msgr2=0x7fe92c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:18.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.531+0000 7fe9321ff700 1 -- 172.21.15.111:0/433266004 shutdown_connections 2023-03-29T10:58:18.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.532+0000 7fe9321ff700 1 -- 172.21.15.111:0/433266004 wait complete. 2023-03-29T10:58:18.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.533+0000 7fe9321ff700 1 Processor -- start 2023-03-29T10:58:18.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.533+0000 7fe9321ff700 1 -- start start 2023-03-29T10:58:18.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.534+0000 7fe9321ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe92c0cf010 0x7fe92c0b7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:18.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.534+0000 7fe9321ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe92c0cfd80 0x7fe92c0c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:18.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.534+0000 7fe9321ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c15c590 0x7fe92c0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:18.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.534+0000 7fe9321ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0b7690 con 0x7fe92c15c590 2023-03-29T10:58:18.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.534+0000 7fe9321ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0b67c0 con 0x7fe92c0cfd80 2023-03-29T10:58:18.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.534+0000 7fe9321ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0b6a10 con 0x7fe92c0cf010 2023-03-29T10:58:18.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.534+0000 7fe92bfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c15c590 0x7fe92c0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:18.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.535+0000 7fe92b7fe700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe92c0cf010 0x7fe92c0b7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:18.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.535+0000 7fe92bfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c15c590 0x7fe92c0cb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52764/0 (socket says 172.21.15.111:52764) 2023-03-29T10:58:18.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.535+0000 7fe92bfff700 1 -- 172.21.15.111:0/3802222771 learned_addr learned my addr 172.21.15.111:0/3802222771 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:18.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.535+0000 7fe92affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe92c0cfd80 0x7fe92c0c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:18.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.535+0000 7fe92bfff700 1 -- 172.21.15.111:0/3802222771 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe92c0cf010 msgr2=0x7fe92c0b7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:18.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.535+0000 7fe92bfff700 1 --2- 172.21.15.111:0/3802222771 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe92c0cf010 0x7fe92c0b7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.535+0000 7fe92bfff700 1 -- 172.21.15.111:0/3802222771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe92c0cfd80 msgr2=0x7fe92c0c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:18.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.535+0000 7fe92bfff700 1 --2- 172.21.15.111:0/3802222771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe92c0cfd80 0x7fe92c0c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.535+0000 7fe92bfff700 1 -- 172.21.15.111:0/3802222771 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91c009550 con 0x7fe92c15c590 2023-03-29T10:58:18.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.535+0000 7fe92bfff700 1 --2- 172.21.15.111:0/3802222771 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c15c590 0x7fe92c0cb5e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe92400cc50 tx=0x7fe92400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:18.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.535+0000 7fe913fff700 1 -- 172.21.15.111:0/3802222771 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924003c00 con 0x7fe92c15c590 2023-03-29T10:58:18.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.535+0000 7fe913fff700 1 -- 172.21.15.111:0/3802222771 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe924003d60 con 0x7fe92c15c590 2023-03-29T10:58:18.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.535+0000 7fe9321ff700 1 -- 172.21.15.111:0/3802222771 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92c0cd770 con 0x7fe92c15c590 2023-03-29T10:58:18.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.535+0000 7fe913fff700 1 -- 172.21.15.111:0/3802222771 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92400aab0 con 0x7fe92c15c590 2023-03-29T10:58:18.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.536+0000 7fe9321ff700 1 -- 172.21.15.111:0/3802222771 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92c0cf560 con 0x7fe92c15c590 2023-03-29T10:58:18.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.537+0000 7fe9321ff700 1 -- 172.21.15.111:0/3802222771 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92c0d01a0 con 0x7fe92c15c590 2023-03-29T10:58:18.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.538+0000 7fe913fff700 1 -- 172.21.15.111:0/3802222771 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fe92400ac10 con 0x7fe92c15c590 2023-03-29T10:58:18.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.538+0000 7fe913fff700 1 --2- 172.21.15.111:0/3802222771 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe918069de0 0x7fe91806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:18.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.539+0000 7fe913fff700 1 -- 172.21.15.111:0/3802222771 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe924088330 con 0x7fe92c15c590 2023-03-29T10:58:18.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.539+0000 7fe92b7fe700 1 --2- 172.21.15.111:0/3802222771 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe918069de0 0x7fe91806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:18.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.539+0000 7fe92b7fe700 1 --2- 172.21.15.111:0/3802222771 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe918069de0 0x7fe91806c290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe91c000c00 tx=0x7fe91c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:18.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.543+0000 7fe913fff700 1 -- 172.21.15.111:0/3802222771 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fe924053810 con 0x7fe92c15c590 2023-03-29T10:58:18.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.736+0000 7fe9321ff700 1 -- 172.21.15.111:0/3802222771 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe92c0b77d0 con 0x7fe92c15c590 2023-03-29T10:58:18.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.737+0000 7fe913fff700 1 -- 172.21.15.111:0/3802222771 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe92c0b77d0 con 0x7fe92c15c590 2023-03-29T10:58:18.738 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 8 2023-03-29T10:58:18.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.739+0000 7fe9321ff700 1 -- 172.21.15.111:0/3802222771 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe918069de0 msgr2=0x7fe91806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:18.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.739+0000 7fe9321ff700 1 --2- 172.21.15.111:0/3802222771 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe918069de0 0x7fe91806c290 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe91c000c00 tx=0x7fe91c008030 comp rx=0 tx=0).stop 2023-03-29T10:58:18.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.739+0000 7fe9321ff700 1 -- 172.21.15.111:0/3802222771 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c15c590 msgr2=0x7fe92c0cb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:18.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.739+0000 7fe9321ff700 1 --2- 172.21.15.111:0/3802222771 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c15c590 0x7fe92c0cb5e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe92400cc50 tx=0x7fe92400cf60 comp rx=0 tx=0).stop 2023-03-29T10:58:18.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.739+0000 7fe9321ff700 1 -- 172.21.15.111:0/3802222771 shutdown_connections 2023-03-29T10:58:18.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.739+0000 7fe9321ff700 1 --2- 172.21.15.111:0/3802222771 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe918069de0 0x7fe91806c290 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.739+0000 7fe9321ff700 1 --2- 172.21.15.111:0/3802222771 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe92c15c590 0x7fe92c0cb5e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.739+0000 7fe9321ff700 1 --2- 172.21.15.111:0/3802222771 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe92c0cf010 0x7fe92c0b7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.739+0000 7fe9321ff700 1 --2- 172.21.15.111:0/3802222771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe92c0cfd80 0x7fe92c0c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.739+0000 7fe9321ff700 1 -- 172.21.15.111:0/3802222771 >> 172.21.15.111:0/3802222771 conn(0x7fe92c071b10 msgr2=0x7fe92c0cab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:18.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.740+0000 7fe9321ff700 1 -- 172.21.15.111:0/3802222771 shutdown_connections 2023-03-29T10:58:18.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.740+0000 7fe9321ff700 1 -- 172.21.15.111:0/3802222771 wait complete. 2023-03-29T10:58:18.757 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-29T10:58:18.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.943+0000 7f80761ff700 1 Processor -- start 2023-03-29T10:58:18.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.944+0000 7f80761ff700 1 -- start start 2023-03-29T10:58:18.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.945+0000 7f80761ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f807015a5a0 0x7f807006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:18.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.945+0000 7f80761ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80700cb280 con 0x7f8070071200 2023-03-29T10:58:18.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.945+0000 7f80761ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8070071700 con 0x7f807006a920 2023-03-29T10:58:18.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.945+0000 7f80761ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807006adc0 con 0x7f807015a5a0 2023-03-29T10:58:18.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.945+0000 7f806f7fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f807015a5a0 0x7f807006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:18.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.945+0000 7f806f7fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f807015a5a0 0x7f807006a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52774/0 (socket says 172.21.15.111:52774) 2023-03-29T10:58:18.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.945+0000 7f806f7fe700 1 -- 172.21.15.111:0/4107366958 learned_addr learned my addr 172.21.15.111:0/4107366958 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:18.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.946+0000 7f806f7fe700 1 -- 172.21.15.111:0/4107366958 >> v1:172.21.15.119:6789/0 conn(0x7f807006a920 legacy=0x7f8070070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:18.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.946+0000 7f806f7fe700 1 -- 172.21.15.111:0/4107366958 >> v1:172.21.15.111:6789/0 conn(0x7f8070071200 legacy=0x7f807006d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:18.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.946+0000 7f806f7fe700 1 -- 172.21.15.111:0/4107366958 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80700719e0 con 0x7f807015a5a0 2023-03-29T10:58:18.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.946+0000 7f806f7fe700 1 --2- 172.21.15.111:0/4107366958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f807015a5a0 0x7f807006a3e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f80600098f0 tx=0x7f8060009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b756f6fd7f89e707 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:18.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.946+0000 7f806effd700 1 -- 172.21.15.111:0/4107366958 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806000b8a0 con 0x7f807015a5a0 2023-03-29T10:58:18.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.946+0000 7f806effd700 1 -- 172.21.15.111:0/4107366958 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806000ba00 con 0x7f807015a5a0 2023-03-29T10:58:18.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.946+0000 7f806effd700 1 -- 172.21.15.111:0/4107366958 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806000bd90 con 0x7f807015a5a0 2023-03-29T10:58:18.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.947+0000 7f80761ff700 1 -- 172.21.15.111:0/4107366958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f807015a5a0 msgr2=0x7f807006a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:18.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.947+0000 7f80761ff700 1 --2- 172.21.15.111:0/4107366958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f807015a5a0 0x7f807006a3e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f80600098f0 tx=0x7f8060009c00 comp rx=0 tx=0).stop 2023-03-29T10:58:18.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.947+0000 7f80761ff700 1 -- 172.21.15.111:0/4107366958 shutdown_connections 2023-03-29T10:58:18.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.947+0000 7f80761ff700 1 --2- 172.21.15.111:0/4107366958 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f807015a5a0 0x7f807006a3e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.947+0000 7f80761ff700 1 -- 172.21.15.111:0/4107366958 >> 172.21.15.111:0/4107366958 conn(0x7f80700cf720 msgr2=0x7f80700cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:18.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.947+0000 7f80761ff700 1 -- 172.21.15.111:0/4107366958 shutdown_connections 2023-03-29T10:58:18.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.947+0000 7f80761ff700 1 -- 172.21.15.111:0/4107366958 wait complete. 2023-03-29T10:58:18.950 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.949+0000 7f80761ff700 1 Processor -- start 2023-03-29T10:58:18.950 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.950+0000 7f80761ff700 1 -- start start 2023-03-29T10:58:18.951 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.951+0000 7f80761ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f807006a920 0x7f807014e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:18.952 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.951+0000 7f80761ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8070071200 0x7f80700ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:18.952 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.951+0000 7f80761ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f807015a5a0 0x7f807015c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:18.952 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.951+0000 7f80761ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80700acac0 con 0x7f8070071200 2023-03-29T10:58:18.952 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.951+0000 7f80761ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807014b710 con 0x7f807006a920 2023-03-29T10:58:18.952 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.951+0000 7f80761ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80700ccf70 con 0x7f807015a5a0 2023-03-29T10:58:18.952 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.951+0000 7f8067fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8070071200 0x7f80700ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:18.952 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.951+0000 7f8067fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8070071200 0x7f80700ab8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52784/0 (socket says 172.21.15.111:52784) 2023-03-29T10:58:18.952 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.951+0000 7f8067fff700 1 -- 172.21.15.111:0/1348578784 learned_addr learned my addr 172.21.15.111:0/1348578784 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:18.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.951+0000 7f806f7fe700 1 --2- 172.21.15.111:0/1348578784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f807006a920 0x7f807014e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:18.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.951+0000 7f806ffff700 1 --2- 172.21.15.111:0/1348578784 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f807015a5a0 0x7f807015c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:18.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.952+0000 7f8067fff700 1 -- 172.21.15.111:0/1348578784 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f807015a5a0 msgr2=0x7f807015c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:18.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.952+0000 7f8067fff700 1 --2- 172.21.15.111:0/1348578784 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f807015a5a0 0x7f807015c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.952+0000 7f8067fff700 1 -- 172.21.15.111:0/1348578784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f807006a920 msgr2=0x7f807014e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:18.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.952+0000 7f8067fff700 1 --2- 172.21.15.111:0/1348578784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f807006a920 0x7f807014e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:18.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.952+0000 7f8067fff700 1 -- 172.21.15.111:0/1348578784 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8060009550 con 0x7f8070071200 2023-03-29T10:58:18.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.952+0000 7f8067fff700 1 --2- 172.21.15.111:0/1348578784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8070071200 0x7f80700ab8b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f805800c480 tx=0x7f805800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:18.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.952+0000 7f806cff9700 1 -- 172.21.15.111:0/1348578784 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80580052d0 con 0x7f8070071200 2023-03-29T10:58:18.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.952+0000 7f806cff9700 1 -- 172.21.15.111:0/1348578784 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8058005450 con 0x7f8070071200 2023-03-29T10:58:18.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.952+0000 7f806cff9700 1 -- 172.21.15.111:0/1348578784 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8058003910 con 0x7f8070071200 2023-03-29T10:58:18.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.952+0000 7f80761ff700 1 -- 172.21.15.111:0/1348578784 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807006c9a0 con 0x7f8070071200 2023-03-29T10:58:18.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.952+0000 7f80761ff700 1 -- 172.21.15.111:0/1348578784 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807006c6c0 con 0x7f8070071200 2023-03-29T10:58:18.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.953+0000 7f80761ff700 1 -- 172.21.15.111:0/1348578784 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f807006cc80 con 0x7f8070071200 2023-03-29T10:58:18.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.954+0000 7f806cff9700 1 -- 172.21.15.111:0/1348578784 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f8058003a70 con 0x7f8070071200 2023-03-29T10:58:18.955 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.955+0000 7f806cff9700 1 --2- 172.21.15.111:0/1348578784 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8050069de0 0x7f805006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:18.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.955+0000 7f806cff9700 1 -- 172.21.15.111:0/1348578784 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f80580877b0 con 0x7f8070071200 2023-03-29T10:58:18.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.955+0000 7f806f7fe700 1 --2- 172.21.15.111:0/1348578784 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8050069de0 0x7f805006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:18.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.956+0000 7f806f7fe700 1 --2- 172.21.15.111:0/1348578784 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8050069de0 0x7f805006c290 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8060014040 tx=0x7f8060008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:18.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:18.963+0000 7f806cff9700 1 -- 172.21.15.111:0/1348578784 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f8058052d50 con 0x7f8070071200 2023-03-29T10:58:19.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.161+0000 7f80761ff700 1 -- 172.21.15.111:0/1348578784 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f80700cbcf0 con 0x7f8070071200 2023-03-29T10:58:19.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.162+0000 7f806cff9700 1 -- 172.21.15.111:0/1348578784 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8058056370 con 0x7f8070071200 2023-03-29T10:58:19.163 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:19.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.164+0000 7f80761ff700 1 -- 172.21.15.111:0/1348578784 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8050069de0 msgr2=0x7f805006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:19.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.164+0000 7f80761ff700 1 --2- 172.21.15.111:0/1348578784 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8050069de0 0x7f805006c290 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8060014040 tx=0x7f8060008030 comp rx=0 tx=0).stop 2023-03-29T10:58:19.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.164+0000 7f80761ff700 1 -- 172.21.15.111:0/1348578784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8070071200 msgr2=0x7f80700ab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:19.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.164+0000 7f80761ff700 1 --2- 172.21.15.111:0/1348578784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8070071200 0x7f80700ab8b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f805800c480 tx=0x7f805800c790 comp rx=0 tx=0).stop 2023-03-29T10:58:19.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.164+0000 7f80761ff700 1 -- 172.21.15.111:0/1348578784 shutdown_connections 2023-03-29T10:58:19.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.164+0000 7f80761ff700 1 --2- 172.21.15.111:0/1348578784 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8050069de0 0x7f805006c290 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.164+0000 7f80761ff700 1 --2- 172.21.15.111:0/1348578784 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f807015a5a0 0x7f807015c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.164+0000 7f80761ff700 1 --2- 172.21.15.111:0/1348578784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f807006a920 0x7f807014e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.164+0000 7f80761ff700 1 --2- 172.21.15.111:0/1348578784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8070071200 0x7f80700ab8b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.164+0000 7f80761ff700 1 -- 172.21.15.111:0/1348578784 >> 172.21.15.111:0/1348578784 conn(0x7f80700cf720 msgr2=0x7f80700afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:19.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.164+0000 7f80761ff700 1 -- 172.21.15.111:0/1348578784 shutdown_connections 2023-03-29T10:58:19.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.164+0000 7f80761ff700 1 -- 172.21.15.111:0/1348578784 wait complete. 2023-03-29T10:58:19.182 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-29T10:58:19.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.367+0000 7f8642bff700 1 Processor -- start 2023-03-29T10:58:19.368 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.367+0000 7f8642bff700 1 -- start start 2023-03-29T10:58:19.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.368+0000 7f8642bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f863c0d2110 0x7f863c167550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:19.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.368+0000 7f8642bff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f863c161000 con 0x7f863c0b87a0 2023-03-29T10:58:19.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.368+0000 7f8642bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f863c0d0020 con 0x7f863c0cfbf0 2023-03-29T10:58:19.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.368+0000 7f8642bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f863c0d02d0 con 0x7f863c0d2110 2023-03-29T10:58:19.369 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.369+0000 7f864099b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f863c0b87a0 0x7f863c1601e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:39844/0 (socket says 172.21.15.111:39844) 2023-03-29T10:58:19.370 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.369+0000 7f864099b700 1 -- 172.21.15.111:0/1411386187 learned_addr learned my addr 172.21.15.111:0/1411386187 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:19.370 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.369+0000 7f864119c700 1 --2- 172.21.15.111:0/1411386187 >> v2:172.21.15.119:3300/0 conn(0x7f863c0d2110 0x7f863c167550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:19.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.369+0000 7f863b7fe700 1 -- 172.21.15.111:0/1411386187 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626010993 0 0) 0x7f863c161000 con 0x7f863c0b87a0 2023-03-29T10:58:19.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.369+0000 7f863b7fe700 1 -- 172.21.15.111:0/1411386187 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f862c0035b0 con 0x7f863c0b87a0 2023-03-29T10:58:19.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.369+0000 7f863b7fe700 1 -- 172.21.15.111:0/1411386187 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180366190 0 0) 0x7f863c0d0020 con 0x7f863c0cfbf0 2023-03-29T10:58:19.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.369+0000 7f863b7fe700 1 -- 172.21.15.111:0/1411386187 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f863c161000 con 0x7f863c0cfbf0 2023-03-29T10:58:19.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.369+0000 7f863b7fe700 1 -- 172.21.15.111:0/1411386187 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 885872064 0 0) 0x7f862c0035b0 con 0x7f863c0b87a0 2023-03-29T10:58:19.371 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.369+0000 7f863b7fe700 1 -- 172.21.15.111:0/1411386187 >> v2:172.21.15.119:3300/0 conn(0x7f863c0d2110 msgr2=0x7f863c167550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:19.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.369+0000 7f863b7fe700 1 --2- 172.21.15.111:0/1411386187 >> v2:172.21.15.119:3300/0 conn(0x7f863c0d2110 0x7f863c167550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.369+0000 7f863b7fe700 1 -- 172.21.15.111:0/1411386187 >> v1:172.21.15.119:6789/0 conn(0x7f863c0cfbf0 legacy=0x7f863c1608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:19.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.369+0000 7f863b7fe700 1 -- 172.21.15.111:0/1411386187 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863c0cdc60 con 0x7f863c0b87a0 2023-03-29T10:58:19.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.369+0000 7f863b7fe700 1 -- 172.21.15.111:0/1411386187 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f8630001670 con 0x7f863c0b87a0 2023-03-29T10:58:19.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.369+0000 7f863b7fe700 1 -- 172.21.15.111:0/1411386187 >> v1:172.21.15.111:6789/0 conn(0x7f863c0b87a0 legacy=0x7f863c1601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:19.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.369+0000 7f863b7fe700 1 --2- 172.21.15.111:0/1411386187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c003b90 0x7f862c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:19.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.369+0000 7f863b7fe700 1 --2- 172.21.15.111:0/1411386187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f862c004850 0x7f862c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:19.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.369+0000 7f863b7fe700 1 --2- 172.21.15.111:0/1411386187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f862c005200 0x7f862c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:19.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.370+0000 7f863b7fe700 1 -- 172.21.15.111:0/1411386187 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c0d02d0 con 0x7f862c003b90 2023-03-29T10:58:19.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.370+0000 7f863b7fe700 1 -- 172.21.15.111:0/1411386187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c00c110 con 0x7f862c004850 2023-03-29T10:58:19.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.370+0000 7f863b7fe700 1 -- 172.21.15.111:0/1411386187 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862c00c2d0 con 0x7f862c005200 2023-03-29T10:58:19.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.370+0000 7f863bfff700 1 --2- 172.21.15.111:0/1411386187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f862c004850 0x7f862c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:19.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.370+0000 7f864099b700 1 --2- 172.21.15.111:0/1411386187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c003b90 0x7f862c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:19.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.370+0000 7f864119c700 1 --2- 172.21.15.111:0/1411386187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f862c005200 0x7f862c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:19.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.370+0000 7f864099b700 1 -- 172.21.15.111:0/1411386187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f862c005200 msgr2=0x7f862c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:19.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.370+0000 7f864099b700 1 --2- 172.21.15.111:0/1411386187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f862c005200 0x7f862c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.372 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.370+0000 7f864099b700 1 -- 172.21.15.111:0/1411386187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f862c004850 msgr2=0x7f862c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:19.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.370+0000 7f864099b700 1 --2- 172.21.15.111:0/1411386187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f862c004850 0x7f862c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.370+0000 7f864099b700 1 -- 172.21.15.111:0/1411386187 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c0035b0 con 0x7f862c003b90 2023-03-29T10:58:19.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.370+0000 7f864119c700 1 -- 172.21.15.111:0/1411386187 reap_dead start 2023-03-29T10:58:19.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.370+0000 7f864099b700 1 --2- 172.21.15.111:0/1411386187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c003b90 0x7f862c003f80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f863000c210 tx=0x7f862c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344681e4b0da741e server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:19.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.371+0000 7f863b7fe700 1 -- 172.21.15.111:0/1411386187 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863000cd50 con 0x7f862c003b90 2023-03-29T10:58:19.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.371+0000 7f863b7fe700 1 -- 172.21.15.111:0/1411386187 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8630005960 con 0x7f862c003b90 2023-03-29T10:58:19.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.371+0000 7f863b7fe700 1 -- 172.21.15.111:0/1411386187 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8630005d10 con 0x7f862c003b90 2023-03-29T10:58:19.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.371+0000 7f8642bff700 1 -- 172.21.15.111:0/1411386187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c003b90 msgr2=0x7f862c003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:19.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.371+0000 7f8642bff700 1 --2- 172.21.15.111:0/1411386187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c003b90 0x7f862c003f80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f863000c210 tx=0x7f862c005880 comp rx=0 tx=0).stop 2023-03-29T10:58:19.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.371+0000 7f8642bff700 1 -- 172.21.15.111:0/1411386187 shutdown_connections 2023-03-29T10:58:19.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.371+0000 7f8642bff700 1 --2- 172.21.15.111:0/1411386187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c003b90 0x7f862c003f80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.371+0000 7f8642bff700 1 -- 172.21.15.111:0/1411386187 >> 172.21.15.111:0/1411386187 conn(0x7f863c071b10 msgr2=0x7f863c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:19.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.371+0000 7f8642bff700 1 -- 172.21.15.111:0/1411386187 shutdown_connections 2023-03-29T10:58:19.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.371+0000 7f8642bff700 1 -- 172.21.15.111:0/1411386187 wait complete. 2023-03-29T10:58:19.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.373+0000 7f8642bff700 1 Processor -- start 2023-03-29T10:58:19.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.373+0000 7f8642bff700 1 -- start start 2023-03-29T10:58:19.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.374+0000 7f8642bff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c003b90 0x7f863c0cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:19.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.374+0000 7f8642bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f863c0c69c0 0x7f863c151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:19.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.374+0000 7f8642bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f863c159d30 0x7f863c15a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:19.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.374+0000 7f8642bff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c1520e0 con 0x7f862c003b90 2023-03-29T10:58:19.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.374+0000 7f8642bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863c0c7ab0 con 0x7f863c159d30 2023-03-29T10:58:19.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.374+0000 7f8642bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863c15a6e0 con 0x7f863c0c69c0 2023-03-29T10:58:19.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.374+0000 7f864099b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c003b90 0x7f863c0cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:19.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.374+0000 7f864099b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c003b90 0x7f863c0cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52796/0 (socket says 172.21.15.111:52796) 2023-03-29T10:58:19.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.374+0000 7f864099b700 1 -- 172.21.15.111:0/2273205443 learned_addr learned my addr 172.21.15.111:0/2273205443 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:19.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.374+0000 7f863bfff700 1 --2- 172.21.15.111:0/2273205443 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f863c0c69c0 0x7f863c151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:19.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.375+0000 7f864099b700 1 -- 172.21.15.111:0/2273205443 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f863c0c69c0 msgr2=0x7f863c151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:19.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.375+0000 7f864099b700 1 --2- 172.21.15.111:0/2273205443 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f863c0c69c0 0x7f863c151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.375+0000 7f864099b700 1 -- 172.21.15.111:0/2273205443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f863c159d30 msgr2=0x7f863c15a1a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-03-29T10:58:19.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.375+0000 7f864099b700 1 --2- 172.21.15.111:0/2273205443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f863c159d30 0x7f863c15a1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.375+0000 7f864099b700 1 -- 172.21.15.111:0/2273205443 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f863c0cd980 con 0x7f862c003b90 2023-03-29T10:58:19.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.375+0000 7f864099b700 1 --2- 172.21.15.111:0/2273205443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c003b90 0x7f863c0cbfc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f863000c800 tx=0x7f8630001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:19.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.375+0000 7f86397fa700 1 -- 172.21.15.111:0/2273205443 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8630004c80 con 0x7f862c003b90 2023-03-29T10:58:19.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.375+0000 7f86397fa700 1 -- 172.21.15.111:0/2273205443 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8630004de0 con 0x7f862c003b90 2023-03-29T10:58:19.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.375+0000 7f86397fa700 1 -- 172.21.15.111:0/2273205443 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86300072f0 con 0x7f862c003b90 2023-03-29T10:58:19.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.375+0000 7f8642bff700 1 -- 172.21.15.111:0/2273205443 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f863c0cd6a0 con 0x7f862c003b90 2023-03-29T10:58:19.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.375+0000 7f8642bff700 1 -- 172.21.15.111:0/2273205443 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f863c1601e0 con 0x7f862c003b90 2023-03-29T10:58:19.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.376+0000 7f8642bff700 1 -- 172.21.15.111:0/2273205443 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f863c0b52f0 con 0x7f862c003b90 2023-03-29T10:58:19.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.377+0000 7f86397fa700 1 -- 172.21.15.111:0/2273205443 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f8630007450 con 0x7f862c003b90 2023-03-29T10:58:19.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.378+0000 7f86397fa700 1 --2- 172.21.15.111:0/2273205443 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8620069de0 0x7f862006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:19.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.378+0000 7f86397fa700 1 -- 172.21.15.111:0/2273205443 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8630089100 con 0x7f862c003b90 2023-03-29T10:58:19.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.378+0000 7f863bfff700 1 --2- 172.21.15.111:0/2273205443 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8620069de0 0x7f862006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:19.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.379+0000 7f863bfff700 1 --2- 172.21.15.111:0/2273205443 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8620069de0 0x7f862006c290 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f863c072540 tx=0x7f8628009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:19.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.382+0000 7f86397fa700 1 -- 172.21.15.111:0/2273205443 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f8630054670 con 0x7f862c003b90 2023-03-29T10:58:19.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.576+0000 7f8642bff700 1 -- 172.21.15.111:0/2273205443 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f863c152270 con 0x7f862c003b90 2023-03-29T10:58:19.577 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.577+0000 7f86397fa700 1 -- 172.21.15.111:0/2273205443 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f863c152270 con 0x7f862c003b90 2023-03-29T10:58:19.577 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-03-29T10:58:19.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.579+0000 7f8642bff700 1 -- 172.21.15.111:0/2273205443 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8620069de0 msgr2=0x7f862006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:19.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.579+0000 7f8642bff700 1 --2- 172.21.15.111:0/2273205443 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8620069de0 0x7f862006c290 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f863c072540 tx=0x7f8628009040 comp rx=0 tx=0).stop 2023-03-29T10:58:19.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.579+0000 7f8642bff700 1 -- 172.21.15.111:0/2273205443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c003b90 msgr2=0x7f863c0cbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:19.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.579+0000 7f8642bff700 1 --2- 172.21.15.111:0/2273205443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c003b90 0x7f863c0cbfc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f863000c800 tx=0x7f8630001980 comp rx=0 tx=0).stop 2023-03-29T10:58:19.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.579+0000 7f8642bff700 1 -- 172.21.15.111:0/2273205443 shutdown_connections 2023-03-29T10:58:19.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.579+0000 7f8642bff700 1 --2- 172.21.15.111:0/2273205443 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8620069de0 0x7f862006c290 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.579+0000 7f8642bff700 1 --2- 172.21.15.111:0/2273205443 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f862c003b90 0x7f863c0cbfc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.579+0000 7f8642bff700 1 --2- 172.21.15.111:0/2273205443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f863c159d30 0x7f863c15a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.579+0000 7f8642bff700 1 --2- 172.21.15.111:0/2273205443 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f863c0c69c0 0x7f863c151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.579+0000 7f8642bff700 1 -- 172.21.15.111:0/2273205443 >> 172.21.15.111:0/2273205443 conn(0x7f863c071b10 msgr2=0x7f863c0b5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:19.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.579+0000 7f8642bff700 1 -- 172.21.15.111:0/2273205443 shutdown_connections 2023-03-29T10:58:19.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.579+0000 7f8642bff700 1 -- 172.21.15.111:0/2273205443 wait complete. 2023-03-29T10:58:19.597 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-29T10:58:19.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.781+0000 7f9fe8fff700 1 Processor -- start 2023-03-29T10:58:19.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.782+0000 7f9fe8fff700 1 -- start start 2023-03-29T10:58:19.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.783+0000 7f9fe8fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9fe40b87a0 0x7f9fe40cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:19.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.783+0000 7f9fe8fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fe4160d40 0x7f9fe40d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:19.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.783+0000 7f9fe8fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe40cdc60 con 0x7f9fe41601e0 2023-03-29T10:58:19.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.783+0000 7f9fe8fff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe40d02d0 con 0x7f9fe40b87a0 2023-03-29T10:58:19.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.783+0000 7f9fe8fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe40d0cb0 con 0x7f9fe4160d40 2023-03-29T10:58:19.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.783+0000 7f9fe259c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9fe40b87a0 0x7f9fe40cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:19.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.783+0000 7f9fe259c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9fe40b87a0 0x7f9fe40cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52804/0 (socket says 172.21.15.111:52804) 2023-03-29T10:58:19.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.783+0000 7f9fe259c700 1 -- 172.21.15.111:0/3063417199 learned_addr learned my addr 172.21.15.111:0/3063417199 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:19.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.783+0000 7f9fe2d9d700 1 --2- 172.21.15.111:0/3063417199 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fe4160d40 0x7f9fe40d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:19.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.783+0000 7f9fe259c700 1 -- 172.21.15.111:0/3063417199 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fe4160d40 msgr2=0x7f9fe40d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:19.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.783+0000 7f9fe259c700 1 --2- 172.21.15.111:0/3063417199 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fe4160d40 0x7f9fe40d1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.783+0000 7f9fe259c700 1 -- 172.21.15.111:0/3063417199 >> v1:172.21.15.119:6789/0 conn(0x7f9fe41601e0 legacy=0x7f9fe4160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:19.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.783+0000 7f9fe259c700 1 -- 172.21.15.111:0/3063417199 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe40cd6a0 con 0x7f9fe40b87a0 2023-03-29T10:58:19.785 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.784+0000 7f9fe259c700 1 --2- 172.21.15.111:0/3063417199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fe40b87a0 0x7f9fe40cfbf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9fd40098f0 tx=0x7f9fd4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ea06e9ad32d678 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:19.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.784+0000 7f9fe1d9b700 1 -- 172.21.15.111:0/3063417199 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd400b8a0 con 0x7f9fe40b87a0 2023-03-29T10:58:19.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.784+0000 7f9fe1d9b700 1 -- 172.21.15.111:0/3063417199 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd400ba00 con 0x7f9fe40b87a0 2023-03-29T10:58:19.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.784+0000 7f9fe1d9b700 1 -- 172.21.15.111:0/3063417199 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd400bd90 con 0x7f9fe40b87a0 2023-03-29T10:58:19.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.784+0000 7f9fe8fff700 1 -- 172.21.15.111:0/3063417199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fe40b87a0 msgr2=0x7f9fe40cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:19.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.784+0000 7f9fe8fff700 1 --2- 172.21.15.111:0/3063417199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fe40b87a0 0x7f9fe40cfbf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9fd40098f0 tx=0x7f9fd4009c00 comp rx=0 tx=0).stop 2023-03-29T10:58:19.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.784+0000 7f9fe8fff700 1 -- 172.21.15.111:0/3063417199 shutdown_connections 2023-03-29T10:58:19.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.784+0000 7f9fe8fff700 1 --2- 172.21.15.111:0/3063417199 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fe4160d40 0x7f9fe40d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.784+0000 7f9fe8fff700 1 --2- 172.21.15.111:0/3063417199 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fe40b87a0 0x7f9fe40cfbf0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.785+0000 7f9fe8fff700 1 -- 172.21.15.111:0/3063417199 >> 172.21.15.111:0/3063417199 conn(0x7f9fe4071b10 msgr2=0x7f9fe4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:19.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.785+0000 7f9fe8fff700 1 -- 172.21.15.111:0/3063417199 shutdown_connections 2023-03-29T10:58:19.786 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.785+0000 7f9fe8fff700 1 -- 172.21.15.111:0/3063417199 wait complete. 2023-03-29T10:58:19.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.786+0000 7f9fe8fff700 1 Processor -- start 2023-03-29T10:58:19.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.786+0000 7f9fe8fff700 1 -- start start 2023-03-29T10:58:19.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.787+0000 7f9fe8fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fe40b87a0 0x7f9fe41f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:19.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.787+0000 7f9fe8fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fe41601e0 0x7f9fe40c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:19.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.787+0000 7f9fe8fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fe4160d40 0x7f9fe415e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:19.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.787+0000 7f9fe8fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe41f1eb0 con 0x7f9fe41601e0 2023-03-29T10:58:19.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.787+0000 7f9fe8fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe40c8570 con 0x7f9fe40b87a0 2023-03-29T10:58:19.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.787+0000 7f9fe8fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe41f5950 con 0x7f9fe4160d40 2023-03-29T10:58:19.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.787+0000 7f9fdbfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fe41601e0 0x7f9fe40c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:19.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.787+0000 7f9fdbfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fe41601e0 0x7f9fe40c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52820/0 (socket says 172.21.15.111:52820) 2023-03-29T10:58:19.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.787+0000 7f9fdbfff700 1 -- 172.21.15.111:0/2865003874 learned_addr learned my addr 172.21.15.111:0/2865003874 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:19.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.788+0000 7f9fe259c700 1 --2- 172.21.15.111:0/2865003874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fe40b87a0 0x7f9fe41f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:19.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.788+0000 7f9fe2d9d700 1 --2- 172.21.15.111:0/2865003874 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fe4160d40 0x7f9fe415e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:19.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.788+0000 7f9fdbfff700 1 -- 172.21.15.111:0/2865003874 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fe4160d40 msgr2=0x7f9fe415e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:19.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.788+0000 7f9fdbfff700 1 --2- 172.21.15.111:0/2865003874 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fe4160d40 0x7f9fe415e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.788+0000 7f9fdbfff700 1 -- 172.21.15.111:0/2865003874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fe40b87a0 msgr2=0x7f9fe41f5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:19.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.788+0000 7f9fdbfff700 1 --2- 172.21.15.111:0/2865003874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fe40b87a0 0x7f9fe41f5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.788+0000 7f9fdbfff700 1 -- 172.21.15.111:0/2865003874 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd4009550 con 0x7f9fe41601e0 2023-03-29T10:58:19.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.788+0000 7f9fdbfff700 1 --2- 172.21.15.111:0/2865003874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fe41601e0 0x7f9fe40c8030 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00c4e0 tx=0x7f9fcc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:19.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.788+0000 7f9fdaffd700 1 -- 172.21.15.111:0/2865003874 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0052d0 con 0x7f9fe41601e0 2023-03-29T10:58:19.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.788+0000 7f9fdaffd700 1 -- 172.21.15.111:0/2865003874 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc005430 con 0x7f9fe41601e0 2023-03-29T10:58:19.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.788+0000 7f9fdaffd700 1 -- 172.21.15.111:0/2865003874 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0038c0 con 0x7f9fe41601e0 2023-03-29T10:58:19.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.788+0000 7f9fe8fff700 1 -- 172.21.15.111:0/2865003874 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe40d1640 con 0x7f9fe41601e0 2023-03-29T10:58:19.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.789+0000 7f9fe8fff700 1 -- 172.21.15.111:0/2865003874 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe40cd980 con 0x7f9fe41601e0 2023-03-29T10:58:19.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.789+0000 7f9fe8fff700 1 -- 172.21.15.111:0/2865003874 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe40d17d0 con 0x7f9fe41601e0 2023-03-29T10:58:19.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.791+0000 7f9fdaffd700 1 -- 172.21.15.111:0/2865003874 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f9fcc003a20 con 0x7f9fe41601e0 2023-03-29T10:58:19.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.791+0000 7f9fdaffd700 1 --2- 172.21.15.111:0/2865003874 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9fc4069de0 0x7f9fc406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:19.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.791+0000 7f9fdaffd700 1 -- 172.21.15.111:0/2865003874 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9fcc0876c0 con 0x7f9fe41601e0 2023-03-29T10:58:19.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.792+0000 7f9fe259c700 1 --2- 172.21.15.111:0/2865003874 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9fc4069de0 0x7f9fc406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:19.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.792+0000 7f9fe259c700 1 --2- 172.21.15.111:0/2865003874 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9fc4069de0 0x7f9fc406c290 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009c40 tx=0x7f9fd4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:19.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.795+0000 7f9fdaffd700 1 -- 172.21.15.111:0/2865003874 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f9fcc052c30 con 0x7f9fe41601e0 2023-03-29T10:58:19.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.993+0000 7f9fe8fff700 1 -- 172.21.15.111:0/2865003874 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9fe4074430 con 0x7f9fe41601e0 2023-03-29T10:58:19.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.994+0000 7f9fdaffd700 1 -- 172.21.15.111:0/2865003874 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9fcc056250 con 0x7f9fe41601e0 2023-03-29T10:58:19.994 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:19.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.996+0000 7f9fe8fff700 1 -- 172.21.15.111:0/2865003874 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9fc4069de0 msgr2=0x7f9fc406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:19.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.996+0000 7f9fe8fff700 1 --2- 172.21.15.111:0/2865003874 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9fc4069de0 0x7f9fc406c290 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009c40 tx=0x7f9fd4008030 comp rx=0 tx=0).stop 2023-03-29T10:58:19.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.996+0000 7f9fe8fff700 1 -- 172.21.15.111:0/2865003874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fe41601e0 msgr2=0x7f9fe40c8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:19.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.996+0000 7f9fe8fff700 1 --2- 172.21.15.111:0/2865003874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fe41601e0 0x7f9fe40c8030 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00c4e0 tx=0x7f9fcc00c7f0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.996+0000 7f9fe8fff700 1 -- 172.21.15.111:0/2865003874 shutdown_connections 2023-03-29T10:58:19.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.996+0000 7f9fe8fff700 1 --2- 172.21.15.111:0/2865003874 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9fc4069de0 0x7f9fc406c290 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.996+0000 7f9fe8fff700 1 --2- 172.21.15.111:0/2865003874 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9fe4160d40 0x7f9fe415e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.996+0000 7f9fe8fff700 1 --2- 172.21.15.111:0/2865003874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fe40b87a0 0x7f9fe41f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.996+0000 7f9fe8fff700 1 --2- 172.21.15.111:0/2865003874 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9fe41601e0 0x7f9fe40c8030 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:19.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.996+0000 7f9fe8fff700 1 -- 172.21.15.111:0/2865003874 >> 172.21.15.111:0/2865003874 conn(0x7f9fe4071b10 msgr2=0x7f9fe40d1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:19.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.996+0000 7f9fe8fff700 1 -- 172.21.15.111:0/2865003874 shutdown_connections 2023-03-29T10:58:19.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:19.996+0000 7f9fe8fff700 1 -- 172.21.15.111:0/2865003874 wait complete. 2023-03-29T10:58:20.014 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:58:20.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.199+0000 7f6cc2fff700 1 Processor -- start 2023-03-29T10:58:20.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.199+0000 7f6cc2fff700 1 -- start start 2023-03-29T10:58:20.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.200+0000 7f6cc2fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6cbc0cfd40 0x7f6cbc0d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:20.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.200+0000 7f6cc2fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6cbc0b70f0 0x7f6cbc0b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:20.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.200+0000 7f6cc2fff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cbc0cdc10 con 0x7f6cbc0cf970 2023-03-29T10:58:20.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.200+0000 7f6cc2fff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cbc0b6c60 con 0x7f6cbc0cfd40 2023-03-29T10:58:20.201 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.200+0000 7f6cc2fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cbc0cd650 con 0x7f6cbc0b70f0 2023-03-29T10:58:20.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.201+0000 7f6cbbfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6cbc0cfd40 0x7f6cbc0d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:20.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.201+0000 7f6cbbfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f6cbc0cfd40 0x7f6cbc0d01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52836/0 (socket says 172.21.15.111:52836) 2023-03-29T10:58:20.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.201+0000 7f6cbbfff700 1 -- 172.21.15.111:0/1736323197 learned_addr learned my addr 172.21.15.111:0/1736323197 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:20.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.201+0000 7f6cc159c700 1 --2- 172.21.15.111:0/1736323197 >> v2:172.21.15.119:3300/0 conn(0x7f6cbc0b70f0 0x7f6cbc0b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:20.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.201+0000 7f6cbbfff700 1 -- 172.21.15.111:0/1736323197 >> v2:172.21.15.119:3300/0 conn(0x7f6cbc0b70f0 msgr2=0x7f6cbc0b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:20.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.201+0000 7f6cbbfff700 1 --2- 172.21.15.111:0/1736323197 >> v2:172.21.15.119:3300/0 conn(0x7f6cbc0b70f0 0x7f6cbc0b74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:20.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.201+0000 7f6cbbfff700 1 -- 172.21.15.111:0/1736323197 >> v1:172.21.15.111:6789/0 conn(0x7f6cbc0cf970 legacy=0x7f6cbc0cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:20.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.201+0000 7f6cbbfff700 1 -- 172.21.15.111:0/1736323197 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc0cd930 con 0x7f6cbc0cfd40 2023-03-29T10:58:20.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.201+0000 7f6cc159c700 1 --2- 172.21.15.111:0/1736323197 >> v2:172.21.15.119:3300/0 conn(0x7f6cbc0b70f0 0x7f6cbc0b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:20.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.201+0000 7f6cbb7fe700 1 -- 172.21.15.111:0/1736323197 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211165265 0 0) 0x7f6cbc0cdc10 con 0x7f6cbc0cf970 2023-03-29T10:58:20.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.201+0000 7f6cbbfff700 1 --2- 172.21.15.111:0/1736323197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cbc0cfd40 0x7f6cbc0d01c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc161330 tx=0x7f6ca8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ce930c60f955fe server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:20.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.202+0000 7f6cbb7fe700 1 -- 172.21.15.111:0/1736323197 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca800b870 con 0x7f6cbc0cfd40 2023-03-29T10:58:20.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.202+0000 7f6cbb7fe700 1 -- 172.21.15.111:0/1736323197 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca800b9d0 con 0x7f6cbc0cfd40 2023-03-29T10:58:20.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.202+0000 7f6cbb7fe700 1 -- 172.21.15.111:0/1736323197 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca800bd60 con 0x7f6cbc0cfd40 2023-03-29T10:58:20.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.202+0000 7f6cc2fff700 1 -- 172.21.15.111:0/1736323197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cbc0cfd40 msgr2=0x7f6cbc0d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:20.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.202+0000 7f6cc2fff700 1 --2- 172.21.15.111:0/1736323197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cbc0cfd40 0x7f6cbc0d01c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc161330 tx=0x7f6ca8009bd0 comp rx=0 tx=0).stop 2023-03-29T10:58:20.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.202+0000 7f6cc2fff700 1 -- 172.21.15.111:0/1736323197 shutdown_connections 2023-03-29T10:58:20.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.202+0000 7f6cc2fff700 1 --2- 172.21.15.111:0/1736323197 >> v2:172.21.15.119:3300/0 conn(0x7f6cbc0b70f0 0x7f6cbc0b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:20.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.202+0000 7f6cc2fff700 1 --2- 172.21.15.111:0/1736323197 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cbc0cfd40 0x7f6cbc0d01c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:20.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.202+0000 7f6cc2fff700 1 -- 172.21.15.111:0/1736323197 >> 172.21.15.111:0/1736323197 conn(0x7f6cbc071b10 msgr2=0x7f6cbc071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:20.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.202+0000 7f6cc2fff700 1 -- 172.21.15.111:0/1736323197 shutdown_connections 2023-03-29T10:58:20.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.202+0000 7f6cc2fff700 1 -- 172.21.15.111:0/1736323197 wait complete. 2023-03-29T10:58:20.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.204+0000 7f6cc2fff700 1 Processor -- start 2023-03-29T10:58:20.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.204+0000 7f6cc2fff700 1 -- start start 2023-03-29T10:58:20.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.205+0000 7f6cc2fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6cbc0b70f0 0x7f6cbc15af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:20.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.205+0000 7f6cc2fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cbc0cf970 0x7f6cbc15e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:20.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.205+0000 7f6cc2fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cbc0cfd40 0x7f6cbc15cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:20.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.205+0000 7f6cc2fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0cb910 con 0x7f6cbc0cf970 2023-03-29T10:58:20.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.205+0000 7f6cc2fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc15ea50 con 0x7f6cbc0cfd40 2023-03-29T10:58:20.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.205+0000 7f6cc2fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc15ed30 con 0x7f6cbc0b70f0 2023-03-29T10:58:20.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.205+0000 7f6cbbfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cbc0cf970 0x7f6cbc15e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:20.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.205+0000 7f6cbbfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cbc0cf970 0x7f6cbc15e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52850/0 (socket says 172.21.15.111:52850) 2023-03-29T10:58:20.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.205+0000 7f6cbbfff700 1 -- 172.21.15.111:0/1988287563 learned_addr learned my addr 172.21.15.111:0/1988287563 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:20.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.206+0000 7f6cbbfff700 1 -- 172.21.15.111:0/1988287563 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6cbc0b70f0 msgr2=0x7f6cbc15af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:20.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.206+0000 7f6cc159c700 1 --2- 172.21.15.111:0/1988287563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cbc0cfd40 0x7f6cbc15cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:20.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.206+0000 7f6cc0d9b700 1 --2- 172.21.15.111:0/1988287563 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6cbc0b70f0 0x7f6cbc15af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:20.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.206+0000 7f6cbbfff700 1 --2- 172.21.15.111:0/1988287563 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6cbc0b70f0 0x7f6cbc15af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:20.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.206+0000 7f6cbbfff700 1 -- 172.21.15.111:0/1988287563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cbc0cfd40 msgr2=0x7f6cbc15cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:20.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.206+0000 7f6cbbfff700 1 --2- 172.21.15.111:0/1988287563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cbc0cfd40 0x7f6cbc15cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:20.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.206+0000 7f6cbbfff700 1 -- 172.21.15.111:0/1988287563 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca8009550 con 0x7f6cbc0cf970 2023-03-29T10:58:20.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.206+0000 7f6cc159c700 1 --2- 172.21.15.111:0/1988287563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cbc0cfd40 0x7f6cbc15cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:20.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.206+0000 7f6cc0d9b700 1 --2- 172.21.15.111:0/1988287563 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6cbc0b70f0 0x7f6cbc15af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:20.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.206+0000 7f6cbbfff700 1 --2- 172.21.15.111:0/1988287563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cbc0cf970 0x7f6cbc15e070 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc072670 tx=0x7f6ca8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:20.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.206+0000 7f6cb97fa700 1 -- 172.21.15.111:0/1988287563 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8011860 con 0x7f6cbc0cf970 2023-03-29T10:58:20.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.206+0000 7f6cb97fa700 1 -- 172.21.15.111:0/1988287563 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca801f070 con 0x7f6cbc0cf970 2023-03-29T10:58:20.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.206+0000 7f6cc2fff700 1 -- 172.21.15.111:0/1988287563 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cbc0d0700 con 0x7f6cbc0cf970 2023-03-29T10:58:20.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.206+0000 7f6cb97fa700 1 -- 172.21.15.111:0/1988287563 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8011bc0 con 0x7f6cbc0cf970 2023-03-29T10:58:20.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.206+0000 7f6cc2fff700 1 -- 172.21.15.111:0/1988287563 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cbc0b68f0 con 0x7f6cbc0cf970 2023-03-29T10:58:20.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.207+0000 7f6cc2fff700 1 -- 172.21.15.111:0/1988287563 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cbc0d36d0 con 0x7f6cbc0cf970 2023-03-29T10:58:20.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.208+0000 7f6cb97fa700 1 -- 172.21.15.111:0/1988287563 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f6ca8021030 con 0x7f6cbc0cf970 2023-03-29T10:58:20.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.209+0000 7f6cb97fa700 1 --2- 172.21.15.111:0/1988287563 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6ca0069de0 0x7f6ca006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:20.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.209+0000 7f6cb97fa700 1 -- 172.21.15.111:0/1988287563 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6ca80898d0 con 0x7f6cbc0cf970 2023-03-29T10:58:20.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.209+0000 7f6cc0d9b700 1 --2- 172.21.15.111:0/1988287563 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6ca0069de0 0x7f6ca006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:20.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.210+0000 7f6cc0d9b700 1 --2- 172.21.15.111:0/1988287563 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6ca0069de0 0x7f6ca006c290 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0000f60 tx=0x7f6cb0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:20.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.213+0000 7f6cb97fa700 1 -- 172.21.15.111:0/1988287563 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f6ca8054e40 con 0x7f6cbc0cf970 2023-03-29T10:58:20.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.406+0000 7f6cc2fff700 1 -- 172.21.15.111:0/1988287563 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cbc074430 con 0x7f6cbc0cf970 2023-03-29T10:58:20.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.408+0000 7f6cb97fa700 1 -- 172.21.15.111:0/1988287563 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19892 (secure 0 0 0) 0x7f6ca8058460 con 0x7f6cbc0cf970 2023-03-29T10:58:20.409 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:20.409 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":30,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:58:09.476334+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.26","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:58:20.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.410+0000 7f6cc2fff700 1 -- 172.21.15.111:0/1988287563 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6ca0069de0 msgr2=0x7f6ca006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:20.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.410+0000 7f6cc2fff700 1 --2- 172.21.15.111:0/1988287563 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6ca0069de0 0x7f6ca006c290 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0000f60 tx=0x7f6cb0009040 comp rx=0 tx=0).stop 2023-03-29T10:58:20.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.410+0000 7f6cc2fff700 1 -- 172.21.15.111:0/1988287563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cbc0cf970 msgr2=0x7f6cbc15e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:20.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.410+0000 7f6cc2fff700 1 --2- 172.21.15.111:0/1988287563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cbc0cf970 0x7f6cbc15e070 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc072670 tx=0x7f6ca8004750 comp rx=0 tx=0).stop 2023-03-29T10:58:20.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.410+0000 7f6cc2fff700 1 -- 172.21.15.111:0/1988287563 shutdown_connections 2023-03-29T10:58:20.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.410+0000 7f6cc2fff700 1 --2- 172.21.15.111:0/1988287563 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6ca0069de0 0x7f6ca006c290 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:20.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.410+0000 7f6cc2fff700 1 --2- 172.21.15.111:0/1988287563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cbc0cfd40 0x7f6cbc15cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:20.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.410+0000 7f6cc2fff700 1 --2- 172.21.15.111:0/1988287563 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6cbc0b70f0 0x7f6cbc15af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:20.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.410+0000 7f6cc2fff700 1 --2- 172.21.15.111:0/1988287563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6cbc0cf970 0x7f6cbc15e070 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:20.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.410+0000 7f6cc2fff700 1 -- 172.21.15.111:0/1988287563 >> 172.21.15.111:0/1988287563 conn(0x7f6cbc071b10 msgr2=0x7f6cbc0d2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:20.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.410+0000 7f6cc2fff700 1 -- 172.21.15.111:0/1988287563 shutdown_connections 2023-03-29T10:58:20.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.410+0000 7f6cc2fff700 1 -- 172.21.15.111:0/1988287563 wait complete. 2023-03-29T10:58:20.428 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-29T10:57:16.183979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-29T10:57:21.239809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-29T10:57:26.192124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-29T10:57:26.796777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-29T10:57:29.874795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-29T10:58:20.428 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-29T10:58:20.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.615+0000 7f63a77ff700 1 Processor -- start 2023-03-29T10:58:20.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.615+0000 7f63a77ff700 1 -- start start 2023-03-29T10:58:20.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.616+0000 7f63a77ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f63a00cfbf0 0x7f63a01608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:20.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.616+0000 7f63a77ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63a00d1100 0x7f63a00d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:20.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.616+0000 7f63a77ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a00d0020 con 0x7f63a00b87a0 2023-03-29T10:58:20.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.616+0000 7f63a77ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a00d02d0 con 0x7f63a00cfbf0 2023-03-29T10:58:20.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.616+0000 7f63a77ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a00d0cb0 con 0x7f63a00d1100 2023-03-29T10:58:20.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.617+0000 7f63a4d9a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f63a00cfbf0 0x7f63a01608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:20.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.617+0000 7f63a4d9a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f63a00cfbf0 0x7f63a01608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52864/0 (socket says 172.21.15.111:52864) 2023-03-29T10:58:20.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.617+0000 7f63a4d9a700 1 -- 172.21.15.111:0/1706782709 learned_addr learned my addr 172.21.15.111:0/1706782709 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:20.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.617+0000 7f63a559b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f63a00b87a0 0x7f63a01601e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:39856/0 (socket says 172.21.15.111:39856) 2023-03-29T10:58:20.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.617+0000 7f63a5d9c700 1 --2- 172.21.15.111:0/1706782709 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63a00d1100 0x7f63a00d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:20.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.617+0000 7f63a4d9a700 1 -- 172.21.15.111:0/1706782709 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63a00d1100 msgr2=0x7f63a00d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:20.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.617+0000 7f63a4d9a700 1 --2- 172.21.15.111:0/1706782709 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63a00d1100 0x7f63a00d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:20.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.617+0000 7f63a4d9a700 1 -- 172.21.15.111:0/1706782709 >> v1:172.21.15.111:6789/0 conn(0x7f63a00b87a0 legacy=0x7f63a01601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:20.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.617+0000 7f63a4d9a700 1 -- 172.21.15.111:0/1706782709 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a00cdc60 con 0x7f63a00cfbf0 2023-03-29T10:58:20.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.617+0000 7f63a5d9c700 1 --2- 172.21.15.111:0/1706782709 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63a00d1100 0x7f63a00d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:20.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.617+0000 7f63a4d9a700 1 --2- 172.21.15.111:0/1706782709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63a00cfbf0 0x7f63a01608f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f63900098f0 tx=0x7f6390009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61531f6aab1c405c server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:20.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.618+0000 7f6397fff700 1 -- 172.21.15.111:0/1706782709 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639000b8a0 con 0x7f63a00cfbf0 2023-03-29T10:58:20.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.618+0000 7f6397fff700 1 -- 172.21.15.111:0/1706782709 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639000ba00 con 0x7f63a00cfbf0 2023-03-29T10:58:20.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.618+0000 7f6397fff700 1 -- 172.21.15.111:0/1706782709 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639000bd90 con 0x7f63a00cfbf0 2023-03-29T10:58:20.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.618+0000 7f63a77ff700 1 -- 172.21.15.111:0/1706782709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63a00cfbf0 msgr2=0x7f63a01608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:20.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.618+0000 7f63a77ff700 1 --2- 172.21.15.111:0/1706782709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63a00cfbf0 0x7f63a01608f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f63900098f0 tx=0x7f6390009c00 comp rx=0 tx=0).stop 2023-03-29T10:58:20.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.618+0000 7f63a77ff700 1 -- 172.21.15.111:0/1706782709 shutdown_connections 2023-03-29T10:58:20.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.618+0000 7f63a77ff700 1 --2- 172.21.15.111:0/1706782709 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63a00d1100 0x7f63a00d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:20.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.618+0000 7f63a77ff700 1 --2- 172.21.15.111:0/1706782709 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63a00cfbf0 0x7f63a01608f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:20.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.618+0000 7f63a77ff700 1 -- 172.21.15.111:0/1706782709 >> 172.21.15.111:0/1706782709 conn(0x7f63a0071b10 msgr2=0x7f63a0071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:20.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.618+0000 7f63a77ff700 1 -- 172.21.15.111:0/1706782709 shutdown_connections 2023-03-29T10:58:20.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.618+0000 7f63a77ff700 1 -- 172.21.15.111:0/1706782709 wait complete. 2023-03-29T10:58:20.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.620+0000 7f63a77ff700 1 Processor -- start 2023-03-29T10:58:20.621 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.620+0000 7f63a77ff700 1 -- start start 2023-03-29T10:58:20.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.621+0000 7f63a77ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63a00b87a0 0x7f63a01f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:20.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.621+0000 7f63a77ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63a00cfbf0 0x7f63a015dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:20.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.621+0000 7f63a77ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63a00d1100 0x7f63a00c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:20.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.621+0000 7f63a77ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a01f1e10 con 0x7f63a00d1100 2023-03-29T10:58:20.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.621+0000 7f63a77ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a015e740 con 0x7f63a00b87a0 2023-03-29T10:58:20.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.621+0000 7f63a77ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a015ea20 con 0x7f63a00cfbf0 2023-03-29T10:58:20.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.621+0000 7f63a5d9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63a00d1100 0x7f63a00c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:20.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.622+0000 7f63a5d9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63a00d1100 0x7f63a00c69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52874/0 (socket says 172.21.15.111:52874) 2023-03-29T10:58:20.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.622+0000 7f63a5d9c700 1 -- 172.21.15.111:0/2613970152 learned_addr learned my addr 172.21.15.111:0/2613970152 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:20.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.622+0000 7f63a4d9a700 1 --2- 172.21.15.111:0/2613970152 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63a00cfbf0 0x7f63a015dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:20.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.622+0000 7f63a559b700 1 --2- 172.21.15.111:0/2613970152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63a00b87a0 0x7f63a01f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:20.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.622+0000 7f63a5d9c700 1 -- 172.21.15.111:0/2613970152 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63a00cfbf0 msgr2=0x7f63a015dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:20.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.622+0000 7f63a5d9c700 1 --2- 172.21.15.111:0/2613970152 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63a00cfbf0 0x7f63a015dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:20.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.622+0000 7f63a5d9c700 1 -- 172.21.15.111:0/2613970152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63a00b87a0 msgr2=0x7f63a01f18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:20.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.622+0000 7f63a5d9c700 1 --2- 172.21.15.111:0/2613970152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63a00b87a0 0x7f63a01f18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:20.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.622+0000 7f63a5d9c700 1 -- 172.21.15.111:0/2613970152 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6390009550 con 0x7f63a00d1100 2023-03-29T10:58:20.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.622+0000 7f63a4d9a700 1 --2- 172.21.15.111:0/2613970152 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63a00cfbf0 0x7f63a015dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:20.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.622+0000 7f63a5d9c700 1 --2- 172.21.15.111:0/2613970152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63a00d1100 0x7f63a00c69c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f639800cd50 tx=0x7f639800a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:20.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.622+0000 7f6395ffb700 1 -- 172.21.15.111:0/2613970152 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639800f480 con 0x7f63a00d1100 2023-03-29T10:58:20.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.623+0000 7f6395ffb700 1 -- 172.21.15.111:0/2613970152 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639800f600 con 0x7f63a00d1100 2023-03-29T10:58:20.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.623+0000 7f6395ffb700 1 -- 172.21.15.111:0/2613970152 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639800f9b0 con 0x7f63a00d1100 2023-03-29T10:58:20.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.623+0000 7f63a77ff700 1 -- 172.21.15.111:0/2613970152 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a00cd980 con 0x7f63a00d1100 2023-03-29T10:58:20.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.623+0000 7f63a77ff700 1 -- 172.21.15.111:0/2613970152 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a00cd6a0 con 0x7f63a00d1100 2023-03-29T10:58:20.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.624+0000 7f63a77ff700 1 -- 172.21.15.111:0/2613970152 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a0160e30 con 0x7f63a00d1100 2023-03-29T10:58:20.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.625+0000 7f6395ffb700 1 -- 172.21.15.111:0/2613970152 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f639800b040 con 0x7f63a00d1100 2023-03-29T10:58:20.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.625+0000 7f6395ffb700 1 --2- 172.21.15.111:0/2613970152 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6384069de0 0x7f638406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:20.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.625+0000 7f6395ffb700 1 -- 172.21.15.111:0/2613970152 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6398088bc0 con 0x7f63a00d1100 2023-03-29T10:58:20.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.626+0000 7f63a559b700 1 --2- 172.21.15.111:0/2613970152 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6384069de0 0x7f638406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:20.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.626+0000 7f63a559b700 1 --2- 172.21.15.111:0/2613970152 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6384069de0 0x7f638406c290 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f639c000f60 tx=0x7f639c007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:20.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.629+0000 7f6395ffb700 1 -- 172.21.15.111:0/2613970152 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f63980540a0 con 0x7f63a00d1100 2023-03-29T10:58:20.823 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.822+0000 7f63a77ff700 1 -- 172.21.15.111:0/2613970152 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f63a0074430 con 0x7f63a00d1100 2023-03-29T10:58:20.824 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.823+0000 7f6395ffb700 1 -- 172.21.15.111:0/2613970152 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f639801d020 con 0x7f63a00d1100 2023-03-29T10:58:20.824 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-03-29T10:58:20.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.825+0000 7f63a77ff700 1 -- 172.21.15.111:0/2613970152 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6384069de0 msgr2=0x7f638406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:20.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.825+0000 7f63a77ff700 1 --2- 172.21.15.111:0/2613970152 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6384069de0 0x7f638406c290 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f639c000f60 tx=0x7f639c007d30 comp rx=0 tx=0).stop 2023-03-29T10:58:20.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.825+0000 7f63a77ff700 1 -- 172.21.15.111:0/2613970152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63a00d1100 msgr2=0x7f63a00c69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:20.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.825+0000 7f63a77ff700 1 --2- 172.21.15.111:0/2613970152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63a00d1100 0x7f63a00c69c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f639800cd50 tx=0x7f639800a560 comp rx=0 tx=0).stop 2023-03-29T10:58:20.826 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.826+0000 7f63a77ff700 1 -- 172.21.15.111:0/2613970152 shutdown_connections 2023-03-29T10:58:20.827 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.826+0000 7f63a77ff700 1 --2- 172.21.15.111:0/2613970152 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6384069de0 0x7f638406c290 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:20.827 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.826+0000 7f63a77ff700 1 --2- 172.21.15.111:0/2613970152 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f63a00d1100 0x7f63a00c69c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:20.827 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.826+0000 7f63a77ff700 1 --2- 172.21.15.111:0/2613970152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63a00b87a0 0x7f63a01f18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:20.827 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.826+0000 7f63a77ff700 1 --2- 172.21.15.111:0/2613970152 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f63a00cfbf0 0x7f63a015dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:20.827 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.826+0000 7f63a77ff700 1 -- 172.21.15.111:0/2613970152 >> 172.21.15.111:0/2613970152 conn(0x7f63a0071b10 msgr2=0x7f63a00b31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:20.827 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.826+0000 7f63a77ff700 1 -- 172.21.15.111:0/2613970152 shutdown_connections 2023-03-29T10:58:20.827 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:20.826+0000 7f63a77ff700 1 -- 172.21.15.111:0/2613970152 wait complete. 2023-03-29T10:58:20.844 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-29T10:58:21.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.028+0000 7fdc9abff700 1 Processor -- start 2023-03-29T10:58:21.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.028+0000 7fdc9abff700 1 -- start start 2023-03-29T10:58:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.029+0000 7fdc9abff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdc9415a140 0x7fdc940cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.029+0000 7fdc9abff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc94160ba0 0x7fdc940cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.029+0000 7fdc9abff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc940cfe20 0x7fdc940b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.029+0000 7fdc9abff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc940d02a0 con 0x7fdc9415a140 2023-03-29T10:58:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.029+0000 7fdc9abff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc9415d660 con 0x7fdc940cfe20 2023-03-29T10:58:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.029+0000 7fdc9abff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9415cd70 con 0x7fdc94160ba0 2023-03-29T10:58:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.030+0000 7fdc9899b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdc9415a140 0x7fdc940cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.030+0000 7fdc9899b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdc9415a140 0x7fdc940cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52878/0 (socket says 172.21.15.111:52878) 2023-03-29T10:58:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.030+0000 7fdc9899b700 1 -- 172.21.15.111:0/2301690134 learned_addr learned my addr 172.21.15.111:0/2301690134 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.030+0000 7fdc9899b700 1 -- 172.21.15.111:0/2301690134 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc94160ba0 msgr2=0x7fdc940cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:21.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.030+0000 7fdc9919c700 1 --2- 172.21.15.111:0/2301690134 >> v2:172.21.15.119:3300/0 conn(0x7fdc940cfe20 0x7fdc940b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.030+0000 7fdc93fff700 1 --2- 172.21.15.111:0/2301690134 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc94160ba0 0x7fdc940cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.030+0000 7fdc9899b700 1 --2- 172.21.15.111:0/2301690134 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc94160ba0 0x7fdc940cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.030+0000 7fdc9899b700 1 -- 172.21.15.111:0/2301690134 >> v2:172.21.15.119:3300/0 conn(0x7fdc940cfe20 msgr2=0x7fdc940b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.030+0000 7fdc9899b700 1 --2- 172.21.15.111:0/2301690134 >> v2:172.21.15.119:3300/0 conn(0x7fdc940cfe20 0x7fdc940b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.030+0000 7fdc9899b700 1 -- 172.21.15.111:0/2301690134 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc9415d010 con 0x7fdc9415a140 2023-03-29T10:58:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.030+0000 7fdc9919c700 1 --2- 172.21.15.111:0/2301690134 >> v2:172.21.15.119:3300/0 conn(0x7fdc940cfe20 0x7fdc940b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.030+0000 7fdc93fff700 1 --2- 172.21.15.111:0/2301690134 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc94160ba0 0x7fdc940cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:21.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.030+0000 7fdc9899b700 1 --2- 172.21.15.111:0/2301690134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc9415a140 0x7fdc940cd520 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdc880098f0 tx=0x7fdc88009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9165a3e78a43381a server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:21.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.031+0000 7fdc937fe700 1 -- 172.21.15.111:0/2301690134 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8800b860 con 0x7fdc9415a140 2023-03-29T10:58:21.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.031+0000 7fdc937fe700 1 -- 172.21.15.111:0/2301690134 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8800b9c0 con 0x7fdc9415a140 2023-03-29T10:58:21.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.031+0000 7fdc937fe700 1 -- 172.21.15.111:0/2301690134 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8800bd50 con 0x7fdc9415a140 2023-03-29T10:58:21.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.031+0000 7fdc9abff700 1 -- 172.21.15.111:0/2301690134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc9415a140 msgr2=0x7fdc940cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:21.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.031+0000 7fdc9abff700 1 --2- 172.21.15.111:0/2301690134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc9415a140 0x7fdc940cd520 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdc880098f0 tx=0x7fdc88009c00 comp rx=0 tx=0).stop 2023-03-29T10:58:21.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.031+0000 7fdc9abff700 1 -- 172.21.15.111:0/2301690134 shutdown_connections 2023-03-29T10:58:21.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.031+0000 7fdc9abff700 1 --2- 172.21.15.111:0/2301690134 >> v2:172.21.15.119:3300/0 conn(0x7fdc940cfe20 0x7fdc940b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.031+0000 7fdc9abff700 1 --2- 172.21.15.111:0/2301690134 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc9415a140 0x7fdc940cd520 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.031+0000 7fdc9abff700 1 --2- 172.21.15.111:0/2301690134 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc94160ba0 0x7fdc940cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.031+0000 7fdc9abff700 1 -- 172.21.15.111:0/2301690134 >> 172.21.15.111:0/2301690134 conn(0x7fdc94071b10 msgr2=0x7fdc94071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:21.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.031+0000 7fdc9abff700 1 -- 172.21.15.111:0/2301690134 shutdown_connections 2023-03-29T10:58:21.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.031+0000 7fdc9abff700 1 -- 172.21.15.111:0/2301690134 wait complete. 2023-03-29T10:58:21.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.033+0000 7fdc9abff700 1 Processor -- start 2023-03-29T10:58:21.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.033+0000 7fdc9abff700 1 -- start start 2023-03-29T10:58:21.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.034+0000 7fdc9abff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc940cfe20 0x7fdc941f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.034+0000 7fdc9abff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc9415a140 0x7fdc940d2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.034+0000 7fdc9abff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc94160ba0 0x7fdc940d33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.034+0000 7fdc9abff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc941f6140 con 0x7fdc940cfe20 2023-03-29T10:58:21.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.034+0000 7fdc9abff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc940d3930 con 0x7fdc94160ba0 2023-03-29T10:58:21.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.034+0000 7fdc9abff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc940cadd0 con 0x7fdc9415a140 2023-03-29T10:58:21.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.034+0000 7fdc9899b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc940cfe20 0x7fdc941f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.034+0000 7fdc9899b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc940cfe20 0x7fdc941f5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52880/0 (socket says 172.21.15.111:52880) 2023-03-29T10:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.034+0000 7fdc9899b700 1 -- 172.21.15.111:0/4023143464 learned_addr learned my addr 172.21.15.111:0/4023143464 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.034+0000 7fdc93fff700 1 --2- 172.21.15.111:0/4023143464 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc9415a140 0x7fdc940d2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.034+0000 7fdc9919c700 1 --2- 172.21.15.111:0/4023143464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc94160ba0 0x7fdc940d33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.035+0000 7fdc9899b700 1 -- 172.21.15.111:0/4023143464 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc9415a140 msgr2=0x7fdc940d2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.035+0000 7fdc9899b700 1 --2- 172.21.15.111:0/4023143464 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc9415a140 0x7fdc940d2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.035+0000 7fdc9899b700 1 -- 172.21.15.111:0/4023143464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc94160ba0 msgr2=0x7fdc940d33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.035+0000 7fdc9899b700 1 --2- 172.21.15.111:0/4023143464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc94160ba0 0x7fdc940d33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.035+0000 7fdc9899b700 1 -- 172.21.15.111:0/4023143464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc88009550 con 0x7fdc940cfe20 2023-03-29T10:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.035+0000 7fdc9899b700 1 --2- 172.21.15.111:0/4023143464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc940cfe20 0x7fdc941f5c00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc88014040 tx=0x7fdc88004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.035+0000 7fdc917fa700 1 -- 172.21.15.111:0/4023143464 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc880036a0 con 0x7fdc940cfe20 2023-03-29T10:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.035+0000 7fdc917fa700 1 -- 172.21.15.111:0/4023143464 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8801e070 con 0x7fdc940cfe20 2023-03-29T10:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.035+0000 7fdc9abff700 1 -- 172.21.15.111:0/4023143464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9415daf0 con 0x7fdc940cfe20 2023-03-29T10:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.035+0000 7fdc917fa700 1 -- 172.21.15.111:0/4023143464 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc88003a40 con 0x7fdc940cfe20 2023-03-29T10:58:21.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.035+0000 7fdc9abff700 1 -- 172.21.15.111:0/4023143464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9415d2f0 con 0x7fdc940cfe20 2023-03-29T10:58:21.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.036+0000 7fdc9abff700 1 -- 172.21.15.111:0/4023143464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc940cdb00 con 0x7fdc940cfe20 2023-03-29T10:58:21.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.037+0000 7fdc917fa700 1 -- 172.21.15.111:0/4023143464 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fdc88003ba0 con 0x7fdc940cfe20 2023-03-29T10:58:21.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.038+0000 7fdc917fa700 1 --2- 172.21.15.111:0/4023143464 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdc78069de0 0x7fdc7806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.038+0000 7fdc917fa700 1 -- 172.21.15.111:0/4023143464 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdc88020070 con 0x7fdc940cfe20 2023-03-29T10:58:21.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.038+0000 7fdc93fff700 1 --2- 172.21.15.111:0/4023143464 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdc78069de0 0x7fdc7806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.039 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.039+0000 7fdc93fff700 1 --2- 172.21.15.111:0/4023143464 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdc78069de0 0x7fdc7806c290 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdc941e8a00 tx=0x7fdc80008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:21.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.042+0000 7fdc917fa700 1 -- 172.21.15.111:0/4023143464 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fdc88054f30 con 0x7fdc940cfe20 2023-03-29T10:58:21.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.234+0000 7fdc9abff700 1 -- 172.21.15.111:0/4023143464 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdc940b51f0 con 0x7fdc940cfe20 2023-03-29T10:58:21.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.235+0000 7fdc917fa700 1 -- 172.21.15.111:0/4023143464 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdc88010380 con 0x7fdc940cfe20 2023-03-29T10:58:21.236 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 8 2023-03-29T10:58:21.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.237+0000 7fdc9abff700 1 -- 172.21.15.111:0/4023143464 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdc78069de0 msgr2=0x7fdc7806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:21.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.237+0000 7fdc9abff700 1 --2- 172.21.15.111:0/4023143464 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdc78069de0 0x7fdc7806c290 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdc941e8a00 tx=0x7fdc80008040 comp rx=0 tx=0).stop 2023-03-29T10:58:21.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.237+0000 7fdc9abff700 1 -- 172.21.15.111:0/4023143464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc940cfe20 msgr2=0x7fdc941f5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:21.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.237+0000 7fdc9abff700 1 --2- 172.21.15.111:0/4023143464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc940cfe20 0x7fdc941f5c00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc88014040 tx=0x7fdc88004750 comp rx=0 tx=0).stop 2023-03-29T10:58:21.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.237+0000 7fdc9abff700 1 -- 172.21.15.111:0/4023143464 shutdown_connections 2023-03-29T10:58:21.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.237+0000 7fdc9abff700 1 --2- 172.21.15.111:0/4023143464 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdc78069de0 0x7fdc7806c290 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.237+0000 7fdc9abff700 1 --2- 172.21.15.111:0/4023143464 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc940cfe20 0x7fdc941f5c00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.237+0000 7fdc9abff700 1 --2- 172.21.15.111:0/4023143464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc94160ba0 0x7fdc940d33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.237+0000 7fdc9abff700 1 --2- 172.21.15.111:0/4023143464 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc9415a140 0x7fdc940d2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.237+0000 7fdc9abff700 1 -- 172.21.15.111:0/4023143464 >> 172.21.15.111:0/4023143464 conn(0x7fdc94071b10 msgr2=0x7fdc94072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:21.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.237+0000 7fdc9abff700 1 -- 172.21.15.111:0/4023143464 shutdown_connections 2023-03-29T10:58:21.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.237+0000 7fdc9abff700 1 -- 172.21.15.111:0/4023143464 wait complete. 2023-03-29T10:58:21.255 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-29T10:58:21.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.439+0000 7fda2edff700 1 Processor -- start 2023-03-29T10:58:21.440 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.439+0000 7fda2edff700 1 -- start start 2023-03-29T10:58:21.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.440+0000 7fda2edff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fda280adf70 0x7fda280cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.440+0000 7fda2edff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda2815cf10 0x7fda2806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.440+0000 7fda2edff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda280cc9e0 con 0x7fda280cd760 2023-03-29T10:58:21.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.440+0000 7fda2edff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda2815de20 con 0x7fda280adf70 2023-03-29T10:58:21.442 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.440+0000 7fda2edff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2815d860 con 0x7fda2815cf10 2023-03-29T10:58:21.442 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.441+0000 7fda27fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fda280adf70 0x7fda280cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.442 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.441+0000 7fda27fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fda280adf70 0x7fda280cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52894/0 (socket says 172.21.15.111:52894) 2023-03-29T10:58:21.442 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.441+0000 7fda27fff700 1 -- 172.21.15.111:0/550208692 learned_addr learned my addr 172.21.15.111:0/550208692 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:21.442 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.441+0000 7fda2d39c700 1 --2- 172.21.15.111:0/550208692 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda2815cf10 0x7fda2806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.441+0000 7fda27fff700 1 -- 172.21.15.111:0/550208692 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda2815cf10 msgr2=0x7fda2806c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:21.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.441+0000 7fda27fff700 1 --2- 172.21.15.111:0/550208692 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda2815cf10 0x7fda2806c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.441+0000 7fda27fff700 1 -- 172.21.15.111:0/550208692 >> v1:172.21.15.119:6789/0 conn(0x7fda280cd760 legacy=0x7fda280cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:21.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.441+0000 7fda27fff700 1 -- 172.21.15.111:0/550208692 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2815db40 con 0x7fda280adf70 2023-03-29T10:58:21.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.441+0000 7fda2d39c700 1 --2- 172.21.15.111:0/550208692 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda2815cf10 0x7fda2806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:21.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.441+0000 7fda27fff700 1 --2- 172.21.15.111:0/550208692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda280adf70 0x7fda280cce70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fda14006fd0 tx=0x7fda14009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afca51fc5872a9ff server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:21.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.442+0000 7fda277fe700 1 -- 172.21.15.111:0/550208692 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14005e30 con 0x7fda280adf70 2023-03-29T10:58:21.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.442+0000 7fda277fe700 1 -- 172.21.15.111:0/550208692 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda14004030 con 0x7fda280adf70 2023-03-29T10:58:21.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.442+0000 7fda277fe700 1 -- 172.21.15.111:0/550208692 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda140043e0 con 0x7fda280adf70 2023-03-29T10:58:21.443 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.442+0000 7fda2edff700 1 -- 172.21.15.111:0/550208692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda280adf70 msgr2=0x7fda280cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:21.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.442+0000 7fda2edff700 1 --2- 172.21.15.111:0/550208692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda280adf70 0x7fda280cce70 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fda14006fd0 tx=0x7fda14009bd0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.442+0000 7fda2edff700 1 -- 172.21.15.111:0/550208692 shutdown_connections 2023-03-29T10:58:21.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.442+0000 7fda2edff700 1 --2- 172.21.15.111:0/550208692 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda2815cf10 0x7fda2806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.442+0000 7fda2edff700 1 --2- 172.21.15.111:0/550208692 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda280adf70 0x7fda280cce70 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.442+0000 7fda2edff700 1 -- 172.21.15.111:0/550208692 >> 172.21.15.111:0/550208692 conn(0x7fda280cf720 msgr2=0x7fda280cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:21.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.442+0000 7fda2edff700 1 -- 172.21.15.111:0/550208692 shutdown_connections 2023-03-29T10:58:21.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.442+0000 7fda2edff700 1 -- 172.21.15.111:0/550208692 wait complete. 2023-03-29T10:58:21.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.444+0000 7fda2edff700 1 Processor -- start 2023-03-29T10:58:21.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.444+0000 7fda2edff700 1 -- start start 2023-03-29T10:58:21.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.445+0000 7fda2edff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda280adf70 0x7fda2814afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.445+0000 7fda2edff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda280cd760 0x7fda2814b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.445+0000 7fda2edff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda2815cf10 0x7fda281642a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.445+0000 7fda2edff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda28165be0 con 0x7fda280cd760 2023-03-29T10:58:21.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.445+0000 7fda2edff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2814ba20 con 0x7fda280adf70 2023-03-29T10:58:21.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.445+0000 7fda2edff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda28164f60 con 0x7fda2815cf10 2023-03-29T10:58:21.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.445+0000 7fda27fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda280cd760 0x7fda2814b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.445+0000 7fda27fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda280cd760 0x7fda2814b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52904/0 (socket says 172.21.15.111:52904) 2023-03-29T10:58:21.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.445+0000 7fda27fff700 1 -- 172.21.15.111:0/462012202 learned_addr learned my addr 172.21.15.111:0/462012202 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:21.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.445+0000 7fda2cb9b700 1 --2- 172.21.15.111:0/462012202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda280adf70 0x7fda2814afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.445+0000 7fda2d39c700 1 --2- 172.21.15.111:0/462012202 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda2815cf10 0x7fda281642a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.445+0000 7fda27fff700 1 -- 172.21.15.111:0/462012202 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda2815cf10 msgr2=0x7fda281642a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:21.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.445+0000 7fda27fff700 1 --2- 172.21.15.111:0/462012202 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda2815cf10 0x7fda281642a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.446+0000 7fda27fff700 1 -- 172.21.15.111:0/462012202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda280adf70 msgr2=0x7fda2814afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:21.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.446+0000 7fda27fff700 1 --2- 172.21.15.111:0/462012202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda280adf70 0x7fda2814afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.446+0000 7fda27fff700 1 -- 172.21.15.111:0/462012202 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda14009500 con 0x7fda280cd760 2023-03-29T10:58:21.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.446+0000 7fda27fff700 1 --2- 172.21.15.111:0/462012202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda280cd760 0x7fda2814b4e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fda14009130 tx=0x7fda14004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:21.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.446+0000 7fda257fa700 1 -- 172.21.15.111:0/462012202 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14003d90 con 0x7fda280cd760 2023-03-29T10:58:21.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.446+0000 7fda257fa700 1 -- 172.21.15.111:0/462012202 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda14011420 con 0x7fda280cd760 2023-03-29T10:58:21.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.446+0000 7fda257fa700 1 -- 172.21.15.111:0/462012202 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda140117d0 con 0x7fda280cd760 2023-03-29T10:58:21.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.446+0000 7fda2edff700 1 -- 172.21.15.111:0/462012202 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda28071600 con 0x7fda280cd760 2023-03-29T10:58:21.447 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.446+0000 7fda2edff700 1 -- 172.21.15.111:0/462012202 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda280cc670 con 0x7fda280cd760 2023-03-29T10:58:21.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.447+0000 7fda2edff700 1 -- 172.21.15.111:0/462012202 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda280cd3b0 con 0x7fda280cd760 2023-03-29T10:58:21.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.448+0000 7fda257fa700 1 -- 172.21.15.111:0/462012202 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fda14011930 con 0x7fda280cd760 2023-03-29T10:58:21.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.449+0000 7fda257fa700 1 --2- 172.21.15.111:0/462012202 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fda0c069de0 0x7fda0c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.449+0000 7fda257fa700 1 -- 172.21.15.111:0/462012202 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fda140895b0 con 0x7fda280cd760 2023-03-29T10:58:21.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.449+0000 7fda2cb9b700 1 --2- 172.21.15.111:0/462012202 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fda0c069de0 0x7fda0c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.450 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.450+0000 7fda2cb9b700 1 --2- 172.21.15.111:0/462012202 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fda0c069de0 0x7fda0c06c290 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fda1c000f60 tx=0x7fda1c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:21.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.454+0000 7fda257fa700 1 -- 172.21.15.111:0/462012202 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fda14054b20 con 0x7fda280cd760 2023-03-29T10:58:21.623 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T10:58:21.627 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T10:58:21.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.648+0000 7fda2edff700 1 -- 172.21.15.111:0/462012202 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fda280b4a10 con 0x7fda280cd760 2023-03-29T10:58:21.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.649+0000 7fda257fa700 1 -- 172.21.15.111:0/462012202 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7fda14058140 con 0x7fda280cd760 2023-03-29T10:58:21.650 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:21.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.652+0000 7fda2edff700 1 -- 172.21.15.111:0/462012202 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fda0c069de0 msgr2=0x7fda0c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.652+0000 7fda2edff700 1 --2- 172.21.15.111:0/462012202 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fda0c069de0 0x7fda0c06c290 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fda1c000f60 tx=0x7fda1c009040 comp rx=0 tx=0).stop 2023-03-29T10:58:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.652+0000 7fda2edff700 1 -- 172.21.15.111:0/462012202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda280cd760 msgr2=0x7fda2814b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.652+0000 7fda2edff700 1 --2- 172.21.15.111:0/462012202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda280cd760 0x7fda2814b4e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fda14009130 tx=0x7fda14004750 comp rx=0 tx=0).stop 2023-03-29T10:58:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.652+0000 7fda2edff700 1 -- 172.21.15.111:0/462012202 shutdown_connections 2023-03-29T10:58:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.652+0000 7fda2edff700 1 --2- 172.21.15.111:0/462012202 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fda0c069de0 0x7fda0c06c290 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.652+0000 7fda2edff700 1 --2- 172.21.15.111:0/462012202 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda2815cf10 0x7fda281642a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.652+0000 7fda2edff700 1 --2- 172.21.15.111:0/462012202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda280adf70 0x7fda2814afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.652+0000 7fda2edff700 1 --2- 172.21.15.111:0/462012202 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda280cd760 0x7fda2814b4e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.652+0000 7fda2edff700 1 -- 172.21.15.111:0/462012202 >> 172.21.15.111:0/462012202 conn(0x7fda280cf720 msgr2=0x7fda28154460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.652+0000 7fda2edff700 1 -- 172.21.15.111:0/462012202 shutdown_connections 2023-03-29T10:58:21.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.652+0000 7fda2edff700 1 -- 172.21.15.111:0/462012202 wait complete. 2023-03-29T10:58:21.670 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-29T10:58:21.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.898+0000 7fb952fff700 1 Processor -- start 2023-03-29T10:58:21.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.898+0000 7fb952fff700 1 -- start start 2023-03-29T10:58:21.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.900+0000 7fb952fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb94c0cfbf0 0x7fb94c1608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.900+0000 7fb952fff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94c0d0020 con 0x7fb94c0b87a0 2023-03-29T10:58:21.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.900+0000 7fb952fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94c0cdc60 con 0x7fb94c0d1100 2023-03-29T10:58:21.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.900+0000 7fb952fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c0d02d0 con 0x7fb94c0cfbf0 2023-03-29T10:58:21.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.900+0000 7fb950d9b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb94c0b87a0 0x7fb94c1601e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:39864/0 (socket says 172.21.15.111:39864) 2023-03-29T10:58:21.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.900+0000 7fb950d9b700 1 -- 172.21.15.111:0/130209514 learned_addr learned my addr 172.21.15.111:0/130209514 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:21.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.900+0000 7fb94bfff700 1 --2- 172.21.15.111:0/130209514 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb94c0cfbf0 0x7fb94c1608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.900+0000 7fb94b7fe700 1 -- 172.21.15.111:0/130209514 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887385727 0 0) 0x7fb94c0d0020 con 0x7fb94c0b87a0 2023-03-29T10:58:21.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.900+0000 7fb94b7fe700 1 -- 172.21.15.111:0/130209514 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb93c0035b0 con 0x7fb94c0b87a0 2023-03-29T10:58:21.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb94b7fe700 1 -- 172.21.15.111:0/130209514 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2044869858 0 0) 0x7fb93c0035b0 con 0x7fb94c0b87a0 2023-03-29T10:58:21.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb94b7fe700 1 -- 172.21.15.111:0/130209514 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb94c0cfbf0 msgr2=0x7fb94c1608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:21.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb94b7fe700 1 --2- 172.21.15.111:0/130209514 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb94c0cfbf0 0x7fb94c1608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb94b7fe700 1 -- 172.21.15.111:0/130209514 >> v1:172.21.15.119:6789/0 conn(0x7fb94c0d1100 legacy=0x7fb94c0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:21.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb94b7fe700 1 -- 172.21.15.111:0/130209514 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c0cd6a0 con 0x7fb94c0b87a0 2023-03-29T10:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb94b7fe700 1 -- 172.21.15.111:0/130209514 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7fb940001670 con 0x7fb94c0b87a0 2023-03-29T10:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb94b7fe700 1 -- 172.21.15.111:0/130209514 >> v1:172.21.15.111:6789/0 conn(0x7fb94c0b87a0 legacy=0x7fb94c1601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb94b7fe700 1 --2- 172.21.15.111:0/130209514 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb94c0cfbf0 0x7fb93c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb94b7fe700 1 --2- 172.21.15.111:0/130209514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb93c003f70 0x7fb93c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb94b7fe700 1 --2- 172.21.15.111:0/130209514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb93c004d90 0x7fb93c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb94b7fe700 1 -- 172.21.15.111:0/130209514 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c0d02d0 con 0x7fb93c004d90 2023-03-29T10:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb94b7fe700 1 -- 172.21.15.111:0/130209514 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c00ba40 con 0x7fb93c003f70 2023-03-29T10:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb94b7fe700 1 -- 172.21.15.111:0/130209514 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c00bc00 con 0x7fb94c0cfbf0 2023-03-29T10:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb95159c700 1 --2- 172.21.15.111:0/130209514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb93c004d90 0x7fb93c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb950d9b700 1 --2- 172.21.15.111:0/130209514 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb94c0cfbf0 0x7fb93c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb94bfff700 1 --2- 172.21.15.111:0/130209514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb93c003f70 0x7fb93c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb95159c700 1 -- 172.21.15.111:0/130209514 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb94c0cfbf0 msgr2=0x7fb93c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb95159c700 1 --2- 172.21.15.111:0/130209514 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb94c0cfbf0 0x7fb93c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb95159c700 1 -- 172.21.15.111:0/130209514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb93c003f70 msgr2=0x7fb93c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.901+0000 7fb95159c700 1 --2- 172.21.15.111:0/130209514 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb93c003f70 0x7fb93c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.902+0000 7fb95159c700 1 -- 172.21.15.111:0/130209514 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb93c0035b0 con 0x7fb93c004d90 2023-03-29T10:58:21.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.902+0000 7fb95159c700 1 --2- 172.21.15.111:0/130209514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb93c004d90 0x7fb93c005200 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb94400c290 tx=0x7fb93c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aec94b6d87e05c3 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:21.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.902+0000 7fb94b7fe700 1 -- 172.21.15.111:0/130209514 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944001e70 con 0x7fb93c004d90 2023-03-29T10:58:21.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.902+0000 7fb94b7fe700 1 -- 172.21.15.111:0/130209514 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94400ce50 con 0x7fb93c004d90 2023-03-29T10:58:21.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.902+0000 7fb94b7fe700 1 -- 172.21.15.111:0/130209514 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944012660 con 0x7fb93c004d90 2023-03-29T10:58:21.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.902+0000 7fb952fff700 1 -- 172.21.15.111:0/130209514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb93c004d90 msgr2=0x7fb93c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:21.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.902+0000 7fb952fff700 1 --2- 172.21.15.111:0/130209514 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb93c004d90 0x7fb93c005200 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb94400c290 tx=0x7fb93c005880 comp rx=0 tx=0).stop 2023-03-29T10:58:21.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.902+0000 7fb95159c700 1 -- 172.21.15.111:0/130209514 reap_dead start 2023-03-29T10:58:21.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.903+0000 7fb952fff700 1 -- 172.21.15.111:0/130209514 shutdown_connections 2023-03-29T10:58:21.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.903+0000 7fb952fff700 1 -- 172.21.15.111:0/130209514 >> 172.21.15.111:0/130209514 conn(0x7fb94c071b10 msgr2=0x7fb94c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:21.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.903+0000 7fb952fff700 1 -- 172.21.15.111:0/130209514 shutdown_connections 2023-03-29T10:58:21.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.903+0000 7fb952fff700 1 -- 172.21.15.111:0/130209514 wait complete. 2023-03-29T10:58:21.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.905+0000 7fb952fff700 1 Processor -- start 2023-03-29T10:58:21.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.905+0000 7fb952fff700 1 -- start start 2023-03-29T10:58:21.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.906+0000 7fb952fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb93c004d90 0x7fb94c153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.906+0000 7fb952fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb94c0c8860 0x7fb94c0c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.906+0000 7fb952fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb94c0c9f80 0x7fb94c0ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.906+0000 7fb952fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c0c83d0 con 0x7fb94c0c8860 2023-03-29T10:58:21.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.906+0000 7fb952fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c0c77c0 con 0x7fb93c004d90 2023-03-29T10:58:21.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.906+0000 7fb952fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c0c7aa0 con 0x7fb94c0c9f80 2023-03-29T10:58:21.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.907+0000 7fb94bfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb94c0c8860 0x7fb94c0c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.907+0000 7fb94bfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb94c0c8860 0x7fb94c0c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52924/0 (socket says 172.21.15.111:52924) 2023-03-29T10:58:21.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.907+0000 7fb94bfff700 1 -- 172.21.15.111:0/2722189030 learned_addr learned my addr 172.21.15.111:0/2722189030 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:21.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.907+0000 7fb950d9b700 1 --2- 172.21.15.111:0/2722189030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb93c004d90 0x7fb94c153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.907+0000 7fb95159c700 1 --2- 172.21.15.111:0/2722189030 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb94c0c9f80 0x7fb94c0ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.907+0000 7fb94bfff700 1 -- 172.21.15.111:0/2722189030 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb94c0c9f80 msgr2=0x7fb94c0ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:21.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.907+0000 7fb94bfff700 1 --2- 172.21.15.111:0/2722189030 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb94c0c9f80 0x7fb94c0ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.907+0000 7fb94bfff700 1 -- 172.21.15.111:0/2722189030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb93c004d90 msgr2=0x7fb94c153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:21.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.907+0000 7fb94bfff700 1 --2- 172.21.15.111:0/2722189030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb93c004d90 0x7fb94c153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:21.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.907+0000 7fb94bfff700 1 -- 172.21.15.111:0/2722189030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c1608f0 con 0x7fb94c0c8860 2023-03-29T10:58:21.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.907+0000 7fb94bfff700 1 --2- 172.21.15.111:0/2722189030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb94c0c8860 0x7fb94c0c8cd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb938011aa0 tx=0x7fb938011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:21.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.908+0000 7fb9497fa700 1 -- 172.21.15.111:0/2722189030 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb938011ee0 con 0x7fb94c0c8860 2023-03-29T10:58:21.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.908+0000 7fb9497fa700 1 -- 172.21.15.111:0/2722189030 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93800f940 con 0x7fb94c0c8860 2023-03-29T10:58:21.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.908+0000 7fb952fff700 1 -- 172.21.15.111:0/2722189030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94c0cd980 con 0x7fb94c0c8860 2023-03-29T10:58:21.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.908+0000 7fb9497fa700 1 -- 172.21.15.111:0/2722189030 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93800fcf0 con 0x7fb94c0c8860 2023-03-29T10:58:21.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.908+0000 7fb952fff700 1 -- 172.21.15.111:0/2722189030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94c160a80 con 0x7fb94c0c8860 2023-03-29T10:58:21.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.909+0000 7fb952fff700 1 -- 172.21.15.111:0/2722189030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94c0c86b0 con 0x7fb94c0c8860 2023-03-29T10:58:21.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.910+0000 7fb9497fa700 1 -- 172.21.15.111:0/2722189030 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fb938014440 con 0x7fb94c0c8860 2023-03-29T10:58:21.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.911+0000 7fb9497fa700 1 --2- 172.21.15.111:0/2722189030 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb930069de0 0x7fb93006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:21.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.911+0000 7fb9497fa700 1 -- 172.21.15.111:0/2722189030 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb938088a10 con 0x7fb94c0c8860 2023-03-29T10:58:21.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.911+0000 7fb950d9b700 1 --2- 172.21.15.111:0/2722189030 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb930069de0 0x7fb93006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:21.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.912+0000 7fb950d9b700 1 --2- 172.21.15.111:0/2722189030 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb930069de0 0x7fb93006c290 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0b3da0 tx=0x7fb940000f80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:21.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:21.915+0000 7fb9497fa700 1 -- 172.21.15.111:0/2722189030 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fb9380579e0 con 0x7fb94c0c8860 2023-03-29T10:58:22.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.109+0000 7fb952fff700 1 -- 172.21.15.111:0/2722189030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb94c0c8510 con 0x7fb94c0c8860 2023-03-29T10:58:22.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.109+0000 7fb9497fa700 1 -- 172.21.15.111:0/2722189030 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb94c0c8510 con 0x7fb94c0c8860 2023-03-29T10:58:22.110 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-03-29T10:58:22.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.111+0000 7fb952fff700 1 -- 172.21.15.111:0/2722189030 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb930069de0 msgr2=0x7fb93006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:22.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.111+0000 7fb952fff700 1 --2- 172.21.15.111:0/2722189030 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb930069de0 0x7fb93006c290 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0b3da0 tx=0x7fb940000f80 comp rx=0 tx=0).stop 2023-03-29T10:58:22.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.111+0000 7fb952fff700 1 -- 172.21.15.111:0/2722189030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb94c0c8860 msgr2=0x7fb94c0c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:22.112 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.111+0000 7fb952fff700 1 --2- 172.21.15.111:0/2722189030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb94c0c8860 0x7fb94c0c8cd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb938011aa0 tx=0x7fb938011db0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.111+0000 7fb952fff700 1 -- 172.21.15.111:0/2722189030 shutdown_connections 2023-03-29T10:58:22.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.111+0000 7fb952fff700 1 --2- 172.21.15.111:0/2722189030 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb930069de0 0x7fb93006c290 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.111+0000 7fb952fff700 1 --2- 172.21.15.111:0/2722189030 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb94c0c9f80 0x7fb94c0ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.111+0000 7fb952fff700 1 --2- 172.21.15.111:0/2722189030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb93c004d90 0x7fb94c153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.111+0000 7fb952fff700 1 --2- 172.21.15.111:0/2722189030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb94c0c8860 0x7fb94c0c8cd0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.111+0000 7fb952fff700 1 -- 172.21.15.111:0/2722189030 >> 172.21.15.111:0/2722189030 conn(0x7fb94c071b10 msgr2=0x7fb94c0b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:22.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.111+0000 7fb952fff700 1 -- 172.21.15.111:0/2722189030 shutdown_connections 2023-03-29T10:58:22.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.111+0000 7fb952fff700 1 -- 172.21.15.111:0/2722189030 wait complete. 2023-03-29T10:58:22.129 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-29T10:58:22.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.316+0000 7f4781dff700 1 Processor -- start 2023-03-29T10:58:22.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.317+0000 7f4781dff700 1 -- start start 2023-03-29T10:58:22.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.318+0000 7f4781dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f477c0b87a0 0x7f477c0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:22.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.318+0000 7f4781dff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f477c160d40 0x7f477c0d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:22.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.318+0000 7f4781dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477c0cdc60 con 0x7f477c1601e0 2023-03-29T10:58:22.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.318+0000 7f4781dff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477c0d02d0 con 0x7f477c160d40 2023-03-29T10:58:22.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.318+0000 7f4781dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477c0d0cb0 con 0x7f477c0b87a0 2023-03-29T10:58:22.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.318+0000 7f477bfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f477c160d40 0x7f477c0d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:22.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.318+0000 7f477bfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f477c160d40 0x7f477c0d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52940/0 (socket says 172.21.15.111:52940) 2023-03-29T10:58:22.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.318+0000 7f477bfff700 1 -- 172.21.15.111:0/3836725525 learned_addr learned my addr 172.21.15.111:0/3836725525 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:22.321 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.318+0000 7f477b7fe700 1 --2- 172.21.15.111:0/3836725525 >> v2:172.21.15.119:3300/0 conn(0x7f477c0b87a0 0x7f477c0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:22.321 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.319+0000 7f477bfff700 1 -- 172.21.15.111:0/3836725525 >> v2:172.21.15.119:3300/0 conn(0x7f477c0b87a0 msgr2=0x7f477c0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:22.321 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.319+0000 7f477bfff700 1 --2- 172.21.15.111:0/3836725525 >> v2:172.21.15.119:3300/0 conn(0x7f477c0b87a0 0x7f477c0cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.321 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.319+0000 7f477bfff700 1 -- 172.21.15.111:0/3836725525 >> v1:172.21.15.119:6789/0 conn(0x7f477c1601e0 legacy=0x7f477c160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:22.321 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.319+0000 7f477bfff700 1 -- 172.21.15.111:0/3836725525 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c0cd6a0 con 0x7f477c160d40 2023-03-29T10:58:22.321 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.319+0000 7f477b7fe700 1 --2- 172.21.15.111:0/3836725525 >> v2:172.21.15.119:3300/0 conn(0x7f477c0b87a0 0x7f477c0cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:22.321 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.319+0000 7f477bfff700 1 --2- 172.21.15.111:0/3836725525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f477c160d40 0x7f477c0d1100 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f477c0b4ac0 tx=0x7f477400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f6a2b07c8da859 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:22.321 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.319+0000 7f477affd700 1 -- 172.21.15.111:0/3836725525 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477400e070 con 0x7f477c160d40 2023-03-29T10:58:22.321 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.319+0000 7f477affd700 1 -- 172.21.15.111:0/3836725525 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477400b980 con 0x7f477c160d40 2023-03-29T10:58:22.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.319+0000 7f477affd700 1 -- 172.21.15.111:0/3836725525 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477400bd30 con 0x7f477c160d40 2023-03-29T10:58:22.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.320+0000 7f4781dff700 1 -- 172.21.15.111:0/3836725525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f477c160d40 msgr2=0x7f477c0d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:22.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.320+0000 7f4781dff700 1 --2- 172.21.15.111:0/3836725525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f477c160d40 0x7f477c0d1100 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f477c0b4ac0 tx=0x7f477400b330 comp rx=0 tx=0).stop 2023-03-29T10:58:22.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.320+0000 7f4781dff700 1 -- 172.21.15.111:0/3836725525 shutdown_connections 2023-03-29T10:58:22.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.320+0000 7f4781dff700 1 --2- 172.21.15.111:0/3836725525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f477c160d40 0x7f477c0d1100 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.320+0000 7f4781dff700 1 --2- 172.21.15.111:0/3836725525 >> v2:172.21.15.119:3300/0 conn(0x7f477c0b87a0 0x7f477c0cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.320+0000 7f4781dff700 1 -- 172.21.15.111:0/3836725525 >> 172.21.15.111:0/3836725525 conn(0x7f477c071b10 msgr2=0x7f477c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:22.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.320+0000 7f4781dff700 1 -- 172.21.15.111:0/3836725525 shutdown_connections 2023-03-29T10:58:22.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.320+0000 7f4781dff700 1 -- 172.21.15.111:0/3836725525 wait complete. 2023-03-29T10:58:22.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.321+0000 7f4781dff700 1 Processor -- start 2023-03-29T10:58:22.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.322+0000 7f4781dff700 1 -- start start 2023-03-29T10:58:22.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.322+0000 7f4781dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f477c0b87a0 0x7f477c1515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:22.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.322+0000 7f4781dff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f477c1601e0 0x7f477c15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:22.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.322+0000 7f4781dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f477c160d40 0x7f477c0d3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:22.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.322+0000 7f4781dff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c155970 con 0x7f477c1601e0 2023-03-29T10:58:22.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.322+0000 7f4781dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c0c9b20 con 0x7f477c160d40 2023-03-29T10:58:22.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.322+0000 7f4781dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477c15ca70 con 0x7f477c0b87a0 2023-03-29T10:58:22.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.323+0000 7f4773fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f477c1601e0 0x7f477c15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:22.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.323+0000 7f4773fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f477c1601e0 0x7f477c15c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52950/0 (socket says 172.21.15.111:52950) 2023-03-29T10:58:22.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.323+0000 7f4773fff700 1 -- 172.21.15.111:0/138948386 learned_addr learned my addr 172.21.15.111:0/138948386 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:22.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.323+0000 7f4773fff700 1 -- 172.21.15.111:0/138948386 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f477c0b87a0 msgr2=0x7f477c1515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:22.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.323+0000 7f4773fff700 1 --2- 172.21.15.111:0/138948386 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f477c0b87a0 0x7f477c1515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.323+0000 7f4773fff700 1 -- 172.21.15.111:0/138948386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f477c160d40 msgr2=0x7f477c0d3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:22.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.323+0000 7f4773fff700 1 --2- 172.21.15.111:0/138948386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f477c160d40 0x7f477c0d3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.323+0000 7f4773fff700 1 -- 172.21.15.111:0/138948386 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4774009b30 con 0x7f477c1601e0 2023-03-29T10:58:22.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.323+0000 7f4773fff700 1 --2- 172.21.15.111:0/138948386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f477c1601e0 0x7f477c15c3e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f476400c500 tx=0x7f476400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:22.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.323+0000 7f4778ff9700 1 -- 172.21.15.111:0/138948386 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47640052d0 con 0x7f477c1601e0 2023-03-29T10:58:22.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.323+0000 7f4778ff9700 1 -- 172.21.15.111:0/138948386 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4764005430 con 0x7f477c1601e0 2023-03-29T10:58:22.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.323+0000 7f4781dff700 1 -- 172.21.15.111:0/138948386 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477c0d1640 con 0x7f477c1601e0 2023-03-29T10:58:22.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.323+0000 7f4778ff9700 1 -- 172.21.15.111:0/138948386 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47640038c0 con 0x7f477c1601e0 2023-03-29T10:58:22.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.324+0000 7f4781dff700 1 -- 172.21.15.111:0/138948386 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477c0cd980 con 0x7f477c1601e0 2023-03-29T10:58:22.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.325+0000 7f4781dff700 1 -- 172.21.15.111:0/138948386 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f477c0d17d0 con 0x7f477c1601e0 2023-03-29T10:58:22.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.326+0000 7f4778ff9700 1 -- 172.21.15.111:0/138948386 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f4764003a20 con 0x7f477c1601e0 2023-03-29T10:58:22.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.327+0000 7f4778ff9700 1 --2- 172.21.15.111:0/138948386 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f475c069de0 0x7f475c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:22.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.327+0000 7f4778ff9700 1 -- 172.21.15.111:0/138948386 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4764087840 con 0x7f477c1601e0 2023-03-29T10:58:22.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.327+0000 7f477b7fe700 1 --2- 172.21.15.111:0/138948386 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f475c069de0 0x7f475c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:22.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.327+0000 7f477b7fe700 1 --2- 172.21.15.111:0/138948386 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f475c069de0 0x7f475c06c290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f476c006fd0 tx=0x7f476c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:22.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.331+0000 7f4778ff9700 1 -- 172.21.15.111:0/138948386 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f4764052db0 con 0x7f477c1601e0 2023-03-29T10:58:22.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.522+0000 7f4781dff700 1 -- 172.21.15.111:0/138948386 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f477c074430 con 0x7f477c1601e0 2023-03-29T10:58:22.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.523+0000 7f4778ff9700 1 -- 172.21.15.111:0/138948386 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f47640563d0 con 0x7f477c1601e0 2023-03-29T10:58:22.524 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:22.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.525+0000 7f4781dff700 1 -- 172.21.15.111:0/138948386 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f475c069de0 msgr2=0x7f475c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:22.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.525+0000 7f4781dff700 1 --2- 172.21.15.111:0/138948386 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f475c069de0 0x7f475c06c290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f476c006fd0 tx=0x7f476c008040 comp rx=0 tx=0).stop 2023-03-29T10:58:22.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.525+0000 7f4781dff700 1 -- 172.21.15.111:0/138948386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f477c1601e0 msgr2=0x7f477c15c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:22.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.525+0000 7f4781dff700 1 --2- 172.21.15.111:0/138948386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f477c1601e0 0x7f477c15c3e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f476400c500 tx=0x7f476400c8c0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.525+0000 7f4781dff700 1 -- 172.21.15.111:0/138948386 shutdown_connections 2023-03-29T10:58:22.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.525+0000 7f4781dff700 1 --2- 172.21.15.111:0/138948386 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f475c069de0 0x7f475c06c290 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.525+0000 7f4781dff700 1 --2- 172.21.15.111:0/138948386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f477c160d40 0x7f477c0d3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.525+0000 7f4781dff700 1 --2- 172.21.15.111:0/138948386 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f477c0b87a0 0x7f477c1515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.525+0000 7f4781dff700 1 --2- 172.21.15.111:0/138948386 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f477c1601e0 0x7f477c15c3e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.525+0000 7f4781dff700 1 -- 172.21.15.111:0/138948386 >> 172.21.15.111:0/138948386 conn(0x7f477c071b10 msgr2=0x7f477c0b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:22.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.525+0000 7f4781dff700 1 -- 172.21.15.111:0/138948386 shutdown_connections 2023-03-29T10:58:22.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.525+0000 7f4781dff700 1 -- 172.21.15.111:0/138948386 wait complete. 2023-03-29T10:58:22.544 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:58:22.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.726+0000 7f0e299ff700 1 Processor -- start 2023-03-29T10:58:22.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.726+0000 7f0e299ff700 1 -- start start 2023-03-29T10:58:22.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.727+0000 7f0e299ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0e240cf970 0x7f0e240cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:22.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.727+0000 7f0e299ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e240cdc10 con 0x7f0e240cfd40 2023-03-29T10:58:22.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.727+0000 7f0e299ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e240b6c60 con 0x7f0e240b70f0 2023-03-29T10:58:22.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.727+0000 7f0e299ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e240d08d0 con 0x7f0e240cf970 2023-03-29T10:58:22.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.727+0000 7f0e227fc700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f0e240cfd40 0x7f0e240d01c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:39868/0 (socket says 172.21.15.111:39868) 2023-03-29T10:58:22.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.727+0000 7f0e227fc700 1 -- 172.21.15.111:0/3792311553 learned_addr learned my addr 172.21.15.111:0/3792311553 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:22.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.728+0000 7f0e22ffd700 1 --2- 172.21.15.111:0/3792311553 >> v2:172.21.15.119:3300/0 conn(0x7f0e240cf970 0x7f0e240cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:22.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.728+0000 7f0e21ffb700 1 -- 172.21.15.111:0/3792311553 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2935966128 0 0) 0x7f0e240cdc10 con 0x7f0e240cfd40 2023-03-29T10:58:22.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.728+0000 7f0e21ffb700 1 -- 172.21.15.111:0/3792311553 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e0c0035b0 con 0x7f0e240cfd40 2023-03-29T10:58:22.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.728+0000 7f0e22ffd700 1 -- 172.21.15.111:0/3792311553 >> v1:172.21.15.119:6789/0 conn(0x7f0e240b70f0 legacy=0x7f0e240b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:22.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.728+0000 7f0e22ffd700 1 -- 172.21.15.111:0/3792311553 >> v1:172.21.15.111:6789/0 conn(0x7f0e240cfd40 legacy=0x7f0e240d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:22.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.728+0000 7f0e22ffd700 1 -- 172.21.15.111:0/3792311553 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e240cd650 con 0x7f0e240cf970 2023-03-29T10:58:22.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.728+0000 7f0e22ffd700 1 --2- 172.21.15.111:0/3792311553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e240cf970 0x7f0e240cf110 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009840 tx=0x7f0e14009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d801a7a7c948dc9f server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:22.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.729+0000 7f0e21ffb700 1 -- 172.21.15.111:0/3792311553 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14011830 con 0x7f0e240cf970 2023-03-29T10:58:22.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.729+0000 7f0e21ffb700 1 -- 172.21.15.111:0/3792311553 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e14011990 con 0x7f0e240cf970 2023-03-29T10:58:22.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.729+0000 7f0e21ffb700 1 -- 172.21.15.111:0/3792311553 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14011d20 con 0x7f0e240cf970 2023-03-29T10:58:22.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.729+0000 7f0e299ff700 1 -- 172.21.15.111:0/3792311553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e240cf970 msgr2=0x7f0e240cf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:22.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.729+0000 7f0e299ff700 1 --2- 172.21.15.111:0/3792311553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e240cf970 0x7f0e240cf110 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009840 tx=0x7f0e14009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:22.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.729+0000 7f0e299ff700 1 -- 172.21.15.111:0/3792311553 shutdown_connections 2023-03-29T10:58:22.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.729+0000 7f0e299ff700 1 --2- 172.21.15.111:0/3792311553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e240cf970 0x7f0e240cf110 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.729+0000 7f0e299ff700 1 -- 172.21.15.111:0/3792311553 >> 172.21.15.111:0/3792311553 conn(0x7f0e24071b10 msgr2=0x7f0e24071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:22.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.729+0000 7f0e299ff700 1 -- 172.21.15.111:0/3792311553 shutdown_connections 2023-03-29T10:58:22.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.729+0000 7f0e299ff700 1 -- 172.21.15.111:0/3792311553 wait complete. 2023-03-29T10:58:22.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.731+0000 7f0e299ff700 1 Processor -- start 2023-03-29T10:58:22.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.731+0000 7f0e299ff700 1 -- start start 2023-03-29T10:58:22.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.732+0000 7f0e299ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e240b70f0 0x7f0e24156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:22.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.732+0000 7f0e299ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e240cf970 0x7f0e24150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:22.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.732+0000 7f0e299ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e240cfd40 0x7f0e241507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:22.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.732+0000 7f0e299ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e240ca540 con 0x7f0e240b70f0 2023-03-29T10:58:22.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.732+0000 7f0e299ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e24157480 con 0x7f0e240cf970 2023-03-29T10:58:22.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.732+0000 7f0e299ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e24157760 con 0x7f0e240cfd40 2023-03-29T10:58:22.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.732+0000 7f0e22ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e240b70f0 0x7f0e24156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:22.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.732+0000 7f0e22ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e240b70f0 0x7f0e24156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52960/0 (socket says 172.21.15.111:52960) 2023-03-29T10:58:22.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.732+0000 7f0e22ffd700 1 -- 172.21.15.111:0/1720666906 learned_addr learned my addr 172.21.15.111:0/1720666906 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:22.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.733+0000 7f0e227fc700 1 --2- 172.21.15.111:0/1720666906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e240cf970 0x7f0e24150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:22.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.733+0000 7f0e22ffd700 1 -- 172.21.15.111:0/1720666906 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e240cfd40 msgr2=0x7f0e241507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:22.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.733+0000 7f0e22ffd700 1 --2- 172.21.15.111:0/1720666906 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e240cfd40 0x7f0e241507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.733+0000 7f0e22ffd700 1 -- 172.21.15.111:0/1720666906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e240cf970 msgr2=0x7f0e24150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:22.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.733+0000 7f0e22ffd700 1 --2- 172.21.15.111:0/1720666906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e240cf970 0x7f0e24150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.733+0000 7f0e22ffd700 1 -- 172.21.15.111:0/1720666906 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e140094a0 con 0x7f0e240b70f0 2023-03-29T10:58:22.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.733+0000 7f0e227fc700 1 --2- 172.21.15.111:0/1720666906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e240cf970 0x7f0e24150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:22.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.733+0000 7f0e22ffd700 1 --2- 172.21.15.111:0/1720666906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e240b70f0 0x7f0e24156ea0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009b90 tx=0x7f0e14004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:22.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.733+0000 7f0e0bfff700 1 -- 172.21.15.111:0/1720666906 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e140038a0 con 0x7f0e240b70f0 2023-03-29T10:58:22.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.733+0000 7f0e0bfff700 1 -- 172.21.15.111:0/1720666906 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1401e070 con 0x7f0e240b70f0 2023-03-29T10:58:22.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.733+0000 7f0e299ff700 1 -- 172.21.15.111:0/1720666906 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e240b68f0 con 0x7f0e240b70f0 2023-03-29T10:58:22.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.733+0000 7f0e0bfff700 1 -- 172.21.15.111:0/1720666906 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14003c40 con 0x7f0e240b70f0 2023-03-29T10:58:22.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.733+0000 7f0e299ff700 1 -- 172.21.15.111:0/1720666906 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e240cd930 con 0x7f0e240b70f0 2023-03-29T10:58:22.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.735+0000 7f0e299ff700 1 -- 172.21.15.111:0/1720666906 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e240cf650 con 0x7f0e240b70f0 2023-03-29T10:58:22.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.735+0000 7f0e0bfff700 1 -- 172.21.15.111:0/1720666906 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f0e1402d450 con 0x7f0e240b70f0 2023-03-29T10:58:22.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.736+0000 7f0e0bfff700 1 --2- 172.21.15.111:0/1720666906 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0e10069de0 0x7f0e1006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:22.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.736+0000 7f0e0bfff700 1 -- 172.21.15.111:0/1720666906 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0e14089440 con 0x7f0e240b70f0 2023-03-29T10:58:22.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.736+0000 7f0e227fc700 1 --2- 172.21.15.111:0/1720666906 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0e10069de0 0x7f0e1006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:22.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.737+0000 7f0e227fc700 1 --2- 172.21.15.111:0/1720666906 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0e10069de0 0x7f0e1006c290 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0e241f5880 tx=0x7f0e18009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:22.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.740+0000 7f0e0bfff700 1 -- 172.21.15.111:0/1720666906 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f0e14054970 con 0x7f0e240b70f0 2023-03-29T10:58:22.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.939+0000 7f0e299ff700 1 -- 172.21.15.111:0/1720666906 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e240ca030 con 0x7f0e240b70f0 2023-03-29T10:58:22.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.941+0000 7f0e0bfff700 1 -- 172.21.15.111:0/1720666906 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19892 (secure 0 0 0) 0x7f0e240ca030 con 0x7f0e240b70f0 2023-03-29T10:58:22.942 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:22.942 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":30,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:58:09.476334+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.26","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:58:22.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.943+0000 7f0e299ff700 1 -- 172.21.15.111:0/1720666906 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0e10069de0 msgr2=0x7f0e1006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:22.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.943+0000 7f0e299ff700 1 --2- 172.21.15.111:0/1720666906 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0e10069de0 0x7f0e1006c290 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0e241f5880 tx=0x7f0e18009040 comp rx=0 tx=0).stop 2023-03-29T10:58:22.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.943+0000 7f0e299ff700 1 -- 172.21.15.111:0/1720666906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e240b70f0 msgr2=0x7f0e24156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:22.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.943+0000 7f0e299ff700 1 --2- 172.21.15.111:0/1720666906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e240b70f0 0x7f0e24156ea0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009b90 tx=0x7f0e14004750 comp rx=0 tx=0).stop 2023-03-29T10:58:22.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.943+0000 7f0e299ff700 1 -- 172.21.15.111:0/1720666906 shutdown_connections 2023-03-29T10:58:22.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.943+0000 7f0e299ff700 1 --2- 172.21.15.111:0/1720666906 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0e10069de0 0x7f0e1006c290 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.943+0000 7f0e299ff700 1 --2- 172.21.15.111:0/1720666906 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e240b70f0 0x7f0e24156ea0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.943+0000 7f0e299ff700 1 --2- 172.21.15.111:0/1720666906 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e240cfd40 0x7f0e241507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.943+0000 7f0e299ff700 1 --2- 172.21.15.111:0/1720666906 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e240cf970 0x7f0e24150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:22.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.943+0000 7f0e299ff700 1 -- 172.21.15.111:0/1720666906 >> 172.21.15.111:0/1720666906 conn(0x7f0e24071b10 msgr2=0x7f0e24158720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:22.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.943+0000 7f0e299ff700 1 -- 172.21.15.111:0/1720666906 shutdown_connections 2023-03-29T10:58:22.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:22.943+0000 7f0e299ff700 1 -- 172.21.15.111:0/1720666906 wait complete. 2023-03-29T10:58:22.961 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-29T10:57:16.183979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-29T10:57:21.239809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-29T10:57:26.192124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-29T10:57:26.796777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-29T10:57:29.874795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-29T10:58:22.962 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-29T10:58:23.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.146+0000 7fe6e5bff700 1 Processor -- start 2023-03-29T10:58:23.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.146+0000 7fe6e5bff700 1 -- start start 2023-03-29T10:58:23.148 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.147+0000 7fe6e5bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6e00b87a0 0x7fe6e00cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:23.148 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.147+0000 7fe6e5bff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe6e01601e0 0x7fe6e0160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:23.148 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.147+0000 7fe6e5bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe6e0160b70 0x7fe6e00d2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:23.148 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.147+0000 7fe6e5bff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e00d02d0 con 0x7fe6e01601e0 2023-03-29T10:58:23.148 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.147+0000 7fe6e5bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e00d0cb0 con 0x7fe6e0160b70 2023-03-29T10:58:23.148 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.147+0000 7fe6e5bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e00b2730 con 0x7fe6e00b87a0 2023-03-29T10:58:23.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.148+0000 7fe6d6ffd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe6e01601e0 0x7fe6e0160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:23.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.148+0000 7fe6d6ffd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe6e01601e0 0x7fe6e0160630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52974/0 (socket says 172.21.15.111:52974) 2023-03-29T10:58:23.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.148+0000 7fe6d6ffd700 1 -- 172.21.15.111:0/2940665467 learned_addr learned my addr 172.21.15.111:0/2940665467 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:23.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.148+0000 7fe6df7fe700 1 --2- 172.21.15.111:0/2940665467 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6e00b87a0 0x7fe6e00cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:23.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.148+0000 7fe6dffff700 1 --2- 172.21.15.111:0/2940665467 >> v2:172.21.15.119:3300/0 conn(0x7fe6e0160b70 0x7fe6e00d2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:23.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.148+0000 7fe6d6ffd700 1 -- 172.21.15.111:0/2940665467 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6e00b87a0 msgr2=0x7fe6e00cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:23.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.148+0000 7fe6d6ffd700 1 --2- 172.21.15.111:0/2940665467 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6e00b87a0 0x7fe6e00cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.148+0000 7fe6d6ffd700 1 -- 172.21.15.111:0/2940665467 >> v2:172.21.15.119:3300/0 conn(0x7fe6e0160b70 msgr2=0x7fe6e00d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:23.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.148+0000 7fe6d6ffd700 1 --2- 172.21.15.111:0/2940665467 >> v2:172.21.15.119:3300/0 conn(0x7fe6e0160b70 0x7fe6e00d2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.148+0000 7fe6d6ffd700 1 -- 172.21.15.111:0/2940665467 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e0160ff0 con 0x7fe6e01601e0 2023-03-29T10:58:23.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.148+0000 7fe6d6ffd700 1 --2- 172.21.15.111:0/2940665467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6e01601e0 0x7fe6e0160630 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe6e008be30 tx=0x7fe6c8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3841499baedebd server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:23.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.149+0000 7fe6deffd700 1 -- 172.21.15.111:0/2940665467 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8011830 con 0x7fe6e01601e0 2023-03-29T10:58:23.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.149+0000 7fe6deffd700 1 -- 172.21.15.111:0/2940665467 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c8011990 con 0x7fe6e01601e0 2023-03-29T10:58:23.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.149+0000 7fe6deffd700 1 -- 172.21.15.111:0/2940665467 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8011d20 con 0x7fe6e01601e0 2023-03-29T10:58:23.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.149+0000 7fe6e5bff700 1 -- 172.21.15.111:0/2940665467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6e01601e0 msgr2=0x7fe6e0160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:23.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.149+0000 7fe6e5bff700 1 --2- 172.21.15.111:0/2940665467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6e01601e0 0x7fe6e0160630 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe6e008be30 tx=0x7fe6c8009bd0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.149+0000 7fe6e5bff700 1 -- 172.21.15.111:0/2940665467 shutdown_connections 2023-03-29T10:58:23.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.149+0000 7fe6e5bff700 1 --2- 172.21.15.111:0/2940665467 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6e00b87a0 0x7fe6e00cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.149+0000 7fe6e5bff700 1 --2- 172.21.15.111:0/2940665467 >> v2:172.21.15.119:3300/0 conn(0x7fe6e0160b70 0x7fe6e00d2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.149+0000 7fe6e5bff700 1 --2- 172.21.15.111:0/2940665467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6e01601e0 0x7fe6e0160630 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.149+0000 7fe6e5bff700 1 -- 172.21.15.111:0/2940665467 >> 172.21.15.111:0/2940665467 conn(0x7fe6e0071b10 msgr2=0x7fe6e0071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:23.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.149+0000 7fe6e5bff700 1 -- 172.21.15.111:0/2940665467 shutdown_connections 2023-03-29T10:58:23.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.149+0000 7fe6e5bff700 1 -- 172.21.15.111:0/2940665467 wait complete. 2023-03-29T10:58:23.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.151+0000 7fe6e5bff700 1 Processor -- start 2023-03-29T10:58:23.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.151+0000 7fe6e5bff700 1 -- start start 2023-03-29T10:58:23.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.152+0000 7fe6e5bff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6e00b87a0 0x7fe6e0154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:23.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.152+0000 7fe6e5bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6e01601e0 0x7fe6e01503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:23.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.152+0000 7fe6e5bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e0160b70 0x7fe6e00d2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:23.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.152+0000 7fe6e5bff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e015cca0 con 0x7fe6e00b87a0 2023-03-29T10:58:23.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.152+0000 7fe6e5bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e00c9490 con 0x7fe6e0160b70 2023-03-29T10:58:23.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.152+0000 7fe6e5bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e00c9770 con 0x7fe6e01601e0 2023-03-29T10:58:23.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.152+0000 7fe6df7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6e00b87a0 0x7fe6e0154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:23.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.152+0000 7fe6df7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6e00b87a0 0x7fe6e0154e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52980/0 (socket says 172.21.15.111:52980) 2023-03-29T10:58:23.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.152+0000 7fe6df7fe700 1 -- 172.21.15.111:0/2743594646 learned_addr learned my addr 172.21.15.111:0/2743594646 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:23.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.152+0000 7fe6d6ffd700 1 --2- 172.21.15.111:0/2743594646 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6e01601e0 0x7fe6e01503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:23.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.152+0000 7fe6df7fe700 1 -- 172.21.15.111:0/2743594646 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6e01601e0 msgr2=0x7fe6e01503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:23.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.152+0000 7fe6df7fe700 1 --2- 172.21.15.111:0/2743594646 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6e01601e0 0x7fe6e01503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.152+0000 7fe6df7fe700 1 -- 172.21.15.111:0/2743594646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e0160b70 msgr2=0x7fe6e00d2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:23.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.152+0000 7fe6df7fe700 1 --2- 172.21.15.111:0/2743594646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e0160b70 0x7fe6e00d2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.152+0000 7fe6df7fe700 1 -- 172.21.15.111:0/2743594646 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c8009550 con 0x7fe6e00b87a0 2023-03-29T10:58:23.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.152+0000 7fe6d6ffd700 1 --2- 172.21.15.111:0/2743594646 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6e01601e0 0x7fe6e01503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:23.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.153+0000 7fe6df7fe700 1 --2- 172.21.15.111:0/2743594646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6e00b87a0 0x7fe6e0154e30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000eed0 tx=0x7fe6d000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:23.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.153+0000 7fe6dcff9700 1 -- 172.21.15.111:0/2743594646 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0010040 con 0x7fe6e00b87a0 2023-03-29T10:58:23.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.153+0000 7fe6dcff9700 1 -- 172.21.15.111:0/2743594646 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d000bdb0 con 0x7fe6e00b87a0 2023-03-29T10:58:23.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.153+0000 7fe6e5bff700 1 -- 172.21.15.111:0/2743594646 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e00cd6a0 con 0x7fe6e00b87a0 2023-03-29T10:58:23.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.153+0000 7fe6dcff9700 1 -- 172.21.15.111:0/2743594646 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0014620 con 0x7fe6e00b87a0 2023-03-29T10:58:23.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.153+0000 7fe6e5bff700 1 -- 172.21.15.111:0/2743594646 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e00cdc60 con 0x7fe6e00b87a0 2023-03-29T10:58:23.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.154+0000 7fe6e5bff700 1 -- 172.21.15.111:0/2743594646 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e00cd980 con 0x7fe6e00b87a0 2023-03-29T10:58:23.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.155+0000 7fe6dcff9700 1 -- 172.21.15.111:0/2743594646 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fe6d0014780 con 0x7fe6e00b87a0 2023-03-29T10:58:23.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.156+0000 7fe6dcff9700 1 --2- 172.21.15.111:0/2743594646 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe6c0069de0 0x7fe6c006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:23.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.156+0000 7fe6dcff9700 1 -- 172.21.15.111:0/2743594646 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe6d0088900 con 0x7fe6e00b87a0 2023-03-29T10:58:23.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.156+0000 7fe6d6ffd700 1 --2- 172.21.15.111:0/2743594646 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe6c0069de0 0x7fe6c006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:23.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.156+0000 7fe6d6ffd700 1 --2- 172.21.15.111:0/2743594646 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe6c0069de0 0x7fe6c006c290 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe6e00c8510 tx=0x7fe6c8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:23.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.160+0000 7fe6dcff9700 1 -- 172.21.15.111:0/2743594646 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fe6d0053de0 con 0x7fe6e00b87a0 2023-03-29T10:58:23.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.352+0000 7fe6e5bff700 1 -- 172.21.15.111:0/2743594646 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6e0074430 con 0x7fe6e00b87a0 2023-03-29T10:58:23.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.352+0000 7fe6dcff9700 1 -- 172.21.15.111:0/2743594646 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6d0057400 con 0x7fe6e00b87a0 2023-03-29T10:58:23.353 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-03-29T10:58:23.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.354+0000 7fe6e5bff700 1 -- 172.21.15.111:0/2743594646 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe6c0069de0 msgr2=0x7fe6c006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:23.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.354+0000 7fe6e5bff700 1 --2- 172.21.15.111:0/2743594646 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe6c0069de0 0x7fe6c006c290 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe6e00c8510 tx=0x7fe6c8008030 comp rx=0 tx=0).stop 2023-03-29T10:58:23.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.354+0000 7fe6e5bff700 1 -- 172.21.15.111:0/2743594646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6e00b87a0 msgr2=0x7fe6e0154e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:23.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.354+0000 7fe6e5bff700 1 --2- 172.21.15.111:0/2743594646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6e00b87a0 0x7fe6e0154e30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000eed0 tx=0x7fe6d000b390 comp rx=0 tx=0).stop 2023-03-29T10:58:23.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.355+0000 7fe6e5bff700 1 -- 172.21.15.111:0/2743594646 shutdown_connections 2023-03-29T10:58:23.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.355+0000 7fe6e5bff700 1 --2- 172.21.15.111:0/2743594646 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe6c0069de0 0x7fe6c006c290 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.355+0000 7fe6e5bff700 1 --2- 172.21.15.111:0/2743594646 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe6e00b87a0 0x7fe6e0154e30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.355+0000 7fe6e5bff700 1 --2- 172.21.15.111:0/2743594646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e0160b70 0x7fe6e00d2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.355+0000 7fe6e5bff700 1 --2- 172.21.15.111:0/2743594646 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe6e01601e0 0x7fe6e01503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.355+0000 7fe6e5bff700 1 -- 172.21.15.111:0/2743594646 >> 172.21.15.111:0/2743594646 conn(0x7fe6e0071b10 msgr2=0x7fe6e00cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:23.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.355+0000 7fe6e5bff700 1 -- 172.21.15.111:0/2743594646 shutdown_connections 2023-03-29T10:58:23.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.355+0000 7fe6e5bff700 1 -- 172.21.15.111:0/2743594646 wait complete. 2023-03-29T10:58:23.374 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-29T10:58:23.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.560+0000 7efce57ff700 1 Processor -- start 2023-03-29T10:58:23.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.561+0000 7efce57ff700 1 -- start start 2023-03-29T10:58:23.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.562+0000 7efce57ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efce015a5a0 0x7efce006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:23.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.562+0000 7efce57ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7efce006a920 0x7efce0070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:23.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.562+0000 7efce57ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce00cb280 con 0x7efce0071030 2023-03-29T10:58:23.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.562+0000 7efce57ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce015a970 con 0x7efce006a920 2023-03-29T10:58:23.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.562+0000 7efce57ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce006ad90 con 0x7efce015a5a0 2023-03-29T10:58:23.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.562+0000 7efcd7fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7efce006a920 0x7efce0070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:23.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.562+0000 7efcd7fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7efce006a920 0x7efce0070af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52982/0 (socket says 172.21.15.111:52982) 2023-03-29T10:58:23.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.562+0000 7efcdf7fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7efce0071030 0x7efce015dff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:39878/0 (socket says 172.21.15.111:39878) 2023-03-29T10:58:23.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.562+0000 7efcd7fff700 1 -- 172.21.15.111:0/2477298298 learned_addr learned my addr 172.21.15.111:0/2477298298 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:23.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.562+0000 7efcdeffd700 1 --2- 172.21.15.111:0/2477298298 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efce015a5a0 0x7efce006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:23.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.563+0000 7efcd7fff700 1 -- 172.21.15.111:0/2477298298 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efce015a5a0 msgr2=0x7efce006a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:23.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.563+0000 7efcd7fff700 1 --2- 172.21.15.111:0/2477298298 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efce015a5a0 0x7efce006a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.563+0000 7efcd7fff700 1 -- 172.21.15.111:0/2477298298 >> v1:172.21.15.111:6789/0 conn(0x7efce0071030 legacy=0x7efce015dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:23.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.563+0000 7efcd7fff700 1 -- 172.21.15.111:0/2477298298 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce00714b0 con 0x7efce006a920 2023-03-29T10:58:23.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.563+0000 7efcdeffd700 1 --2- 172.21.15.111:0/2477298298 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efce015a5a0 0x7efce006a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:23.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.563+0000 7efcde7fc700 1 -- 172.21.15.111:0/2477298298 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505772981 0 0) 0x7efcd8001490 con 0x7efce0071030 2023-03-29T10:58:23.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.563+0000 7efcd7fff700 1 --2- 172.21.15.111:0/2477298298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce006a920 0x7efce0070af0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efcc80098f0 tx=0x7efcc8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79f0c0a7726aaab server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:23.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.563+0000 7efcde7fc700 1 -- 172.21.15.111:0/2477298298 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc8005e30 con 0x7efce006a920 2023-03-29T10:58:23.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.563+0000 7efcde7fc700 1 -- 172.21.15.111:0/2477298298 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc8004030 con 0x7efce006a920 2023-03-29T10:58:23.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.563+0000 7efcde7fc700 1 -- 172.21.15.111:0/2477298298 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc80043e0 con 0x7efce006a920 2023-03-29T10:58:23.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.564+0000 7efce57ff700 1 -- 172.21.15.111:0/2477298298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce006a920 msgr2=0x7efce0070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:23.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.564+0000 7efce57ff700 1 --2- 172.21.15.111:0/2477298298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce006a920 0x7efce0070af0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efcc80098f0 tx=0x7efcc8009c00 comp rx=0 tx=0).stop 2023-03-29T10:58:23.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.564+0000 7efce57ff700 1 -- 172.21.15.111:0/2477298298 shutdown_connections 2023-03-29T10:58:23.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.564+0000 7efce57ff700 1 --2- 172.21.15.111:0/2477298298 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efce015a5a0 0x7efce006a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.564+0000 7efce57ff700 1 --2- 172.21.15.111:0/2477298298 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce006a920 0x7efce0070af0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.564+0000 7efce57ff700 1 -- 172.21.15.111:0/2477298298 >> 172.21.15.111:0/2477298298 conn(0x7efce00cf720 msgr2=0x7efce00cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:23.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.564+0000 7efce57ff700 1 -- 172.21.15.111:0/2477298298 shutdown_connections 2023-03-29T10:58:23.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.564+0000 7efce57ff700 1 -- 172.21.15.111:0/2477298298 wait complete. 2023-03-29T10:58:23.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.565+0000 7efce57ff700 1 Processor -- start 2023-03-29T10:58:23.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.566+0000 7efce57ff700 1 -- start start 2023-03-29T10:58:23.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.566+0000 7efce57ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce006a920 0x7efce0148da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:23.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.566+0000 7efce57ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efce0071030 0x7efce00ac900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:23.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.566+0000 7efce57ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce015a5a0 0x7efce00ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:23.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.566+0000 7efce57ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce00c67f0 con 0x7efce006a920 2023-03-29T10:58:23.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.566+0000 7efce57ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce01492e0 con 0x7efce015a5a0 2023-03-29T10:58:23.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.566+0000 7efce57ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce00c5b70 con 0x7efce0071030 2023-03-29T10:58:23.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.567+0000 7efcdeffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce006a920 0x7efce0148da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:23.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.567+0000 7efcdeffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce006a920 0x7efce0148da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52998/0 (socket says 172.21.15.111:52998) 2023-03-29T10:58:23.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.567+0000 7efcdeffd700 1 -- 172.21.15.111:0/2361943187 learned_addr learned my addr 172.21.15.111:0/2361943187 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:23.569 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.567+0000 7efcd7fff700 1 --2- 172.21.15.111:0/2361943187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efce0071030 0x7efce00ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:23.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.567+0000 7efcdf7fe700 1 --2- 172.21.15.111:0/2361943187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce015a5a0 0x7efce00ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:23.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.567+0000 7efcdeffd700 1 -- 172.21.15.111:0/2361943187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efce0071030 msgr2=0x7efce00ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:23.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.567+0000 7efcdeffd700 1 --2- 172.21.15.111:0/2361943187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efce0071030 0x7efce00ac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.567+0000 7efcdeffd700 1 -- 172.21.15.111:0/2361943187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce015a5a0 msgr2=0x7efce00ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:23.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.567+0000 7efcdeffd700 1 --2- 172.21.15.111:0/2361943187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce015a5a0 0x7efce00ace40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.567+0000 7efcdeffd700 1 -- 172.21.15.111:0/2361943187 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc8009500 con 0x7efce006a920 2023-03-29T10:58:23.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.567+0000 7efcdf7fe700 1 --2- 172.21.15.111:0/2361943187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce015a5a0 0x7efce00ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:23.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.567+0000 7efcdeffd700 1 --2- 172.21.15.111:0/2361943187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce006a920 0x7efce0148da0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efcd000e810 tx=0x7efcd000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:23.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.568+0000 7efcd77fe700 1 -- 172.21.15.111:0/2361943187 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd000c020 con 0x7efce006a920 2023-03-29T10:58:23.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.568+0000 7efcd77fe700 1 -- 172.21.15.111:0/2361943187 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd0004250 con 0x7efce006a920 2023-03-29T10:58:23.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.568+0000 7efcd77fe700 1 -- 172.21.15.111:0/2361943187 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd00036a0 con 0x7efce006a920 2023-03-29T10:58:23.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.568+0000 7efce57ff700 1 -- 172.21.15.111:0/2361943187 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce00719e0 con 0x7efce006a920 2023-03-29T10:58:23.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.568+0000 7efce57ff700 1 -- 172.21.15.111:0/2361943187 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce0071700 con 0x7efce006a920 2023-03-29T10:58:23.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.569+0000 7efce57ff700 1 -- 172.21.15.111:0/2361943187 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce006c6c0 con 0x7efce006a920 2023-03-29T10:58:23.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.570+0000 7efcd77fe700 1 -- 172.21.15.111:0/2361943187 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7efcd0003800 con 0x7efce006a920 2023-03-29T10:58:23.571 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.571+0000 7efcd77fe700 1 --2- 172.21.15.111:0/2361943187 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7efcc0069de0 0x7efcc006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:23.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.571+0000 7efcd77fe700 1 -- 172.21.15.111:0/2361943187 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7efcd0087e50 con 0x7efce006a920 2023-03-29T10:58:23.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.571+0000 7efcd7fff700 1 --2- 172.21.15.111:0/2361943187 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7efcc0069de0 0x7efcc006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:23.572 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.571+0000 7efcd7fff700 1 --2- 172.21.15.111:0/2361943187 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7efcc0069de0 0x7efcc006c290 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efcc8000c00 tx=0x7efcc8008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:23.575 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.574+0000 7efcd77fe700 1 -- 172.21.15.111:0/2361943187 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7efcd00533d0 con 0x7efce006a920 2023-03-29T10:58:23.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.767+0000 7efce57ff700 1 -- 172.21.15.111:0/2361943187 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efce0074430 con 0x7efce006a920 2023-03-29T10:58:23.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.767+0000 7efcd77fe700 1 -- 172.21.15.111:0/2361943187 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7efcd000f040 con 0x7efce006a920 2023-03-29T10:58:23.768 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 8 2023-03-29T10:58:23.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.769+0000 7efce57ff700 1 -- 172.21.15.111:0/2361943187 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7efcc0069de0 msgr2=0x7efcc006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:23.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.769+0000 7efce57ff700 1 --2- 172.21.15.111:0/2361943187 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7efcc0069de0 0x7efcc006c290 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efcc8000c00 tx=0x7efcc8008070 comp rx=0 tx=0).stop 2023-03-29T10:58:23.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.769+0000 7efce57ff700 1 -- 172.21.15.111:0/2361943187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce006a920 msgr2=0x7efce0148da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:23.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.769+0000 7efce57ff700 1 --2- 172.21.15.111:0/2361943187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce006a920 0x7efce0148da0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efcd000e810 tx=0x7efcd000eb20 comp rx=0 tx=0).stop 2023-03-29T10:58:23.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.770+0000 7efce57ff700 1 -- 172.21.15.111:0/2361943187 shutdown_connections 2023-03-29T10:58:23.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.770+0000 7efce57ff700 1 --2- 172.21.15.111:0/2361943187 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7efcc0069de0 0x7efcc006c290 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.770+0000 7efce57ff700 1 --2- 172.21.15.111:0/2361943187 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7efce006a920 0x7efce0148da0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.770+0000 7efce57ff700 1 --2- 172.21.15.111:0/2361943187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce015a5a0 0x7efce00ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.770+0000 7efce57ff700 1 --2- 172.21.15.111:0/2361943187 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7efce0071030 0x7efce00ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.770+0000 7efce57ff700 1 -- 172.21.15.111:0/2361943187 >> 172.21.15.111:0/2361943187 conn(0x7efce00cf720 msgr2=0x7efce00af540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:23.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.770+0000 7efce57ff700 1 -- 172.21.15.111:0/2361943187 shutdown_connections 2023-03-29T10:58:23.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.770+0000 7efce57ff700 1 -- 172.21.15.111:0/2361943187 wait complete. 2023-03-29T10:58:23.788 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-29T10:58:23.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.976+0000 7f76847ff700 1 Processor -- start 2023-03-29T10:58:23.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.976+0000 7f76847ff700 1 -- start start 2023-03-29T10:58:23.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.977+0000 7f76847ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f767c06aaf0 0x7f767c070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:23.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.977+0000 7f76847ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767c0cb280 con 0x7f767c071030 2023-03-29T10:58:23.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.977+0000 7f76847ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767c0714b0 con 0x7f767c15a5a0 2023-03-29T10:58:23.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.977+0000 7f76847ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767c15a970 con 0x7f767c06aaf0 2023-03-29T10:58:23.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.977+0000 7f7682d9c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f767c071030 0x7f767c06d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:39880/0 (socket says 172.21.15.111:39880) 2023-03-29T10:58:23.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.977+0000 7f7682d9c700 1 -- 172.21.15.111:0/705298284 learned_addr learned my addr 172.21.15.111:0/705298284 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:23.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.978+0000 7f7681d9a700 1 --2- 172.21.15.111:0/705298284 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f767c06aaf0 0x7f767c070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:23.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.978+0000 7f7681599700 1 -- 172.21.15.111:0/705298284 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3823007039 0 0) 0x7f7674001490 con 0x7f767c071030 2023-03-29T10:58:23.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.978+0000 7f7681599700 1 -- 172.21.15.111:0/705298284 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76680035b0 con 0x7f767c071030 2023-03-29T10:58:23.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.978+0000 7f7681599700 1 -- 172.21.15.111:0/705298284 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 843480465 0 0) 0x7f767c0714b0 con 0x7f767c15a5a0 2023-03-29T10:58:23.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.978+0000 7f7681599700 1 -- 172.21.15.111:0/705298284 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7674001490 con 0x7f767c15a5a0 2023-03-29T10:58:23.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.978+0000 7f7681d9a700 1 -- 172.21.15.111:0/705298284 >> v1:172.21.15.119:6789/0 conn(0x7f767c15a5a0 legacy=0x7f767c06a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:23.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.978+0000 7f7681d9a700 1 -- 172.21.15.111:0/705298284 >> v1:172.21.15.111:6789/0 conn(0x7f767c071030 legacy=0x7f767c06d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:23.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.978+0000 7f7681d9a700 1 -- 172.21.15.111:0/705298284 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c071700 con 0x7f767c06aaf0 2023-03-29T10:58:23.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.978+0000 7f7681d9a700 1 --2- 172.21.15.111:0/705298284 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f767c06aaf0 0x7f767c070af0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f766c009850 tx=0x7f766c009b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3bd8603905fe0c8 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:23.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.979+0000 7f7681599700 1 -- 172.21.15.111:0/705298284 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c005e30 con 0x7f767c06aaf0 2023-03-29T10:58:23.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.979+0000 7f7681599700 1 -- 172.21.15.111:0/705298284 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766c006030 con 0x7f767c06aaf0 2023-03-29T10:58:23.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.979+0000 7f7681599700 1 -- 172.21.15.111:0/705298284 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c0063e0 con 0x7f767c06aaf0 2023-03-29T10:58:23.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.979+0000 7f76847ff700 1 -- 172.21.15.111:0/705298284 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f767c06aaf0 msgr2=0x7f767c070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:23.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.979+0000 7f76847ff700 1 --2- 172.21.15.111:0/705298284 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f767c06aaf0 0x7f767c070af0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f766c009850 tx=0x7f766c009b60 comp rx=0 tx=0).stop 2023-03-29T10:58:23.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.979+0000 7f76847ff700 1 -- 172.21.15.111:0/705298284 shutdown_connections 2023-03-29T10:58:23.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.979+0000 7f76847ff700 1 --2- 172.21.15.111:0/705298284 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f767c06aaf0 0x7f767c070af0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.979+0000 7f76847ff700 1 -- 172.21.15.111:0/705298284 >> 172.21.15.111:0/705298284 conn(0x7f767c0cf720 msgr2=0x7f767c0cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:23.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.979+0000 7f76847ff700 1 -- 172.21.15.111:0/705298284 shutdown_connections 2023-03-29T10:58:23.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.979+0000 7f76847ff700 1 -- 172.21.15.111:0/705298284 wait complete. 2023-03-29T10:58:23.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.981+0000 7f76847ff700 1 Processor -- start 2023-03-29T10:58:23.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.981+0000 7f76847ff700 1 -- start start 2023-03-29T10:58:23.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.982+0000 7f76847ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f767c06aaf0 0x7f767c0ac550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:23.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.982+0000 7f76847ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f767c071030 0x7f767c155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:23.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.982+0000 7f76847ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f767c15a5a0 0x7f767c0af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:23.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.982+0000 7f76847ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c1f2360 con 0x7f767c15a5a0 2023-03-29T10:58:23.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.982+0000 7f76847ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767c0cc5d0 con 0x7f767c06aaf0 2023-03-29T10:58:23.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.982+0000 7f76847ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f767c1f2e70 con 0x7f767c071030 2023-03-29T10:58:23.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.982+0000 7f7682d9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f767c15a5a0 0x7f767c0af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:23.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.982+0000 7f7682d9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f767c15a5a0 0x7f767c0af420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53012/0 (socket says 172.21.15.111:53012) 2023-03-29T10:58:23.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.982+0000 7f7682d9c700 1 -- 172.21.15.111:0/3418347540 learned_addr learned my addr 172.21.15.111:0/3418347540 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:23.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.982+0000 7f768259b700 1 --2- 172.21.15.111:0/3418347540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f767c06aaf0 0x7f767c0ac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:23.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.982+0000 7f7681d9a700 1 --2- 172.21.15.111:0/3418347540 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f767c071030 0x7f767c155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:23.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.983+0000 7f7682d9c700 1 -- 172.21.15.111:0/3418347540 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f767c071030 msgr2=0x7f767c155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:23.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.983+0000 7f7682d9c700 1 --2- 172.21.15.111:0/3418347540 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f767c071030 0x7f767c155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.983+0000 7f7682d9c700 1 -- 172.21.15.111:0/3418347540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f767c06aaf0 msgr2=0x7f767c0ac550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:23.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.983+0000 7f7682d9c700 1 --2- 172.21.15.111:0/3418347540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f767c06aaf0 0x7f767c0ac550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:23.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.983+0000 7f7682d9c700 1 -- 172.21.15.111:0/3418347540 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c009430 con 0x7f767c15a5a0 2023-03-29T10:58:23.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.983+0000 7f768259b700 1 --2- 172.21.15.111:0/3418347540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f767c06aaf0 0x7f767c0ac550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:23.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.983+0000 7f7681d9a700 1 --2- 172.21.15.111:0/3418347540 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f767c071030 0x7f767c155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:23.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.983+0000 7f7682d9c700 1 --2- 172.21.15.111:0/3418347540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f767c15a5a0 0x7f767c0af420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7674001f50 tx=0x7f767400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:23.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.983+0000 7f7672ffd700 1 -- 172.21.15.111:0/3418347540 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767400ade0 con 0x7f767c15a5a0 2023-03-29T10:58:23.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.983+0000 7f7672ffd700 1 -- 172.21.15.111:0/3418347540 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7674012400 con 0x7f767c15a5a0 2023-03-29T10:58:23.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.983+0000 7f76847ff700 1 -- 172.21.15.111:0/3418347540 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f767c06c6c0 con 0x7f767c15a5a0 2023-03-29T10:58:23.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.983+0000 7f7672ffd700 1 -- 172.21.15.111:0/3418347540 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7674012770 con 0x7f767c15a5a0 2023-03-29T10:58:23.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.983+0000 7f76847ff700 1 -- 172.21.15.111:0/3418347540 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f767c0719e0 con 0x7f767c15a5a0 2023-03-29T10:58:23.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.984+0000 7f76847ff700 1 -- 172.21.15.111:0/3418347540 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f767c06c9a0 con 0x7f767c15a5a0 2023-03-29T10:58:23.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.985+0000 7f7672ffd700 1 -- 172.21.15.111:0/3418347540 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f767401b3b0 con 0x7f767c15a5a0 2023-03-29T10:58:23.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.986+0000 7f7672ffd700 1 --2- 172.21.15.111:0/3418347540 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7660069de0 0x7f766006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:23.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.986+0000 7f7672ffd700 1 -- 172.21.15.111:0/3418347540 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7674087fe0 con 0x7f767c15a5a0 2023-03-29T10:58:23.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.986+0000 7f768259b700 1 --2- 172.21.15.111:0/3418347540 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7660069de0 0x7f766006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:23.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.987+0000 7f768259b700 1 --2- 172.21.15.111:0/3418347540 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7660069de0 0x7f766006c290 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f76780011e0 tx=0x7f7678009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:23.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:23.990+0000 7f7672ffd700 1 -- 172.21.15.111:0/3418347540 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f767404c160 con 0x7f767c15a5a0 2023-03-29T10:58:24.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.183+0000 7f76847ff700 1 -- 172.21.15.111:0/3418347540 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f767c0cb280 con 0x7f767c15a5a0 2023-03-29T10:58:24.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.183+0000 7f7672ffd700 1 -- 172.21.15.111:0/3418347540 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f767c0cb280 con 0x7f767c15a5a0 2023-03-29T10:58:24.184 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:24.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.185+0000 7f76847ff700 1 -- 172.21.15.111:0/3418347540 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7660069de0 msgr2=0x7f766006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:24.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.185+0000 7f76847ff700 1 --2- 172.21.15.111:0/3418347540 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7660069de0 0x7f766006c290 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f76780011e0 tx=0x7f7678009040 comp rx=0 tx=0).stop 2023-03-29T10:58:24.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.185+0000 7f76847ff700 1 -- 172.21.15.111:0/3418347540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f767c15a5a0 msgr2=0x7f767c0af420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:24.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.185+0000 7f76847ff700 1 --2- 172.21.15.111:0/3418347540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f767c15a5a0 0x7f767c0af420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7674001f50 tx=0x7f767400a340 comp rx=0 tx=0).stop 2023-03-29T10:58:24.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.185+0000 7f76847ff700 1 -- 172.21.15.111:0/3418347540 shutdown_connections 2023-03-29T10:58:24.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.185+0000 7f76847ff700 1 --2- 172.21.15.111:0/3418347540 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f7660069de0 0x7f766006c290 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:24.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.185+0000 7f76847ff700 1 --2- 172.21.15.111:0/3418347540 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f767c15a5a0 0x7f767c0af420 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:24.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.185+0000 7f76847ff700 1 --2- 172.21.15.111:0/3418347540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f767c06aaf0 0x7f767c0ac550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:24.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.185+0000 7f76847ff700 1 --2- 172.21.15.111:0/3418347540 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f767c071030 0x7f767c155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:24.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.185+0000 7f76847ff700 1 -- 172.21.15.111:0/3418347540 >> 172.21.15.111:0/3418347540 conn(0x7f767c0cf720 msgr2=0x7f767c15cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:24.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.185+0000 7f76847ff700 1 -- 172.21.15.111:0/3418347540 shutdown_connections 2023-03-29T10:58:24.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.185+0000 7f76847ff700 1 -- 172.21.15.111:0/3418347540 wait complete. 2023-03-29T10:58:24.203 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-29T10:58:24.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.389+0000 7f3f62bff700 1 Processor -- start 2023-03-29T10:58:24.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.389+0000 7f3f62bff700 1 -- start start 2023-03-29T10:58:24.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.390+0000 7f3f62bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3f5c0b87a0 0x7f3f5c0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:24.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.390+0000 7f3f62bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f5c160d40 0x7f3f5c0d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:24.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.390+0000 7f3f62bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5c0cdc60 con 0x7f3f5c1601e0 2023-03-29T10:58:24.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.390+0000 7f3f62bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5c0d02d0 con 0x7f3f5c0b87a0 2023-03-29T10:58:24.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.390+0000 7f3f62bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0d0cb0 con 0x7f3f5c160d40 2023-03-29T10:58:24.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.391+0000 7f3f6099b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3f5c0b87a0 0x7f3f5c0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:24.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.391+0000 7f3f6119c700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f5c160d40 0x7f3f5c0d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:24.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.391+0000 7f3f5bfff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3f5c1601e0 0x7f3f5c160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.111:37110/0 (socket says 172.21.15.111:37110) 2023-03-29T10:58:24.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.391+0000 7f3f6099b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3f5c0b87a0 0x7f3f5c0cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:38630/0 (socket says 172.21.15.111:38630) 2023-03-29T10:58:24.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.391+0000 7f3f6119c700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f5c160d40 0x7f3f5c0d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:48292/0 (socket says 172.21.15.111:48292) 2023-03-29T10:58:24.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.391+0000 7f3f5bfff700 1 -- 172.21.15.111:0/1850406975 learned_addr learned my addr 172.21.15.111:0/1850406975 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:24.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.391+0000 7f3f6099b700 1 -- 172.21.15.111:0/1850406975 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f5c160d40 msgr2=0x7f3f5c0d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:24.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.391+0000 7f3f6099b700 1 --2- 172.21.15.111:0/1850406975 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f5c160d40 0x7f3f5c0d1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:24.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.391+0000 7f3f6099b700 1 -- 172.21.15.111:0/1850406975 >> v1:172.21.15.119:6789/0 conn(0x7f3f5c1601e0 legacy=0x7f3f5c160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:24.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.391+0000 7f3f6099b700 1 -- 172.21.15.111:0/1850406975 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5c0cd6a0 con 0x7f3f5c0b87a0 2023-03-29T10:58:24.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.391+0000 7f3f6099b700 1 --2- 172.21.15.111:0/1850406975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f5c0b87a0 0x7f3f5c0cfbf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009840 tx=0x7f3f50009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b046ae2f1fb68c2 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:24.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.392+0000 7f3f5b7fe700 1 -- 172.21.15.111:0/1850406975 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f50011830 con 0x7f3f5c0b87a0 2023-03-29T10:58:24.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.392+0000 7f3f5b7fe700 1 -- 172.21.15.111:0/1850406975 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f50011990 con 0x7f3f5c0b87a0 2023-03-29T10:58:24.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.392+0000 7f3f5b7fe700 1 -- 172.21.15.111:0/1850406975 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f50011d20 con 0x7f3f5c0b87a0 2023-03-29T10:58:24.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.392+0000 7f3f62bff700 1 -- 172.21.15.111:0/1850406975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f5c0b87a0 msgr2=0x7f3f5c0cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:24.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.392+0000 7f3f62bff700 1 --2- 172.21.15.111:0/1850406975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f5c0b87a0 0x7f3f5c0cfbf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009840 tx=0x7f3f50009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:24.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.392+0000 7f3f62bff700 1 -- 172.21.15.111:0/1850406975 shutdown_connections 2023-03-29T10:58:24.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.392+0000 7f3f62bff700 1 --2- 172.21.15.111:0/1850406975 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f5c160d40 0x7f3f5c0d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:24.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.392+0000 7f3f62bff700 1 --2- 172.21.15.111:0/1850406975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f5c0b87a0 0x7f3f5c0cfbf0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:24.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.392+0000 7f3f62bff700 1 -- 172.21.15.111:0/1850406975 >> 172.21.15.111:0/1850406975 conn(0x7f3f5c071b10 msgr2=0x7f3f5c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:24.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.392+0000 7f3f62bff700 1 -- 172.21.15.111:0/1850406975 shutdown_connections 2023-03-29T10:58:24.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.392+0000 7f3f62bff700 1 -- 172.21.15.111:0/1850406975 wait complete. 2023-03-29T10:58:24.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.394+0000 7f3f62bff700 1 Processor -- start 2023-03-29T10:58:24.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.394+0000 7f3f62bff700 1 -- start start 2023-03-29T10:58:24.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.395+0000 7f3f62bff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f5c0b87a0 0x7f3f5c14d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:24.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.395+0000 7f3f62bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f5c1601e0 0x7f3f5c15f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:24.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.395+0000 7f3f62bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f5c160d40 0x7f3f5c0b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:24.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.395+0000 7f3f62bff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c15fa70 con 0x7f3f5c0b87a0 2023-03-29T10:58:24.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.395+0000 7f3f62bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0c69e0 con 0x7f3f5c160d40 2023-03-29T10:58:24.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.395+0000 7f3f62bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5c0c6cc0 con 0x7f3f5c1601e0 2023-03-29T10:58:24.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.395+0000 7f3f6099b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f5c0b87a0 0x7f3f5c14d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:24.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.395+0000 7f3f6099b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f5c0b87a0 0x7f3f5c14d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53024/0 (socket says 172.21.15.111:53024) 2023-03-29T10:58:24.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.395+0000 7f3f6099b700 1 -- 172.21.15.111:0/2582847229 learned_addr learned my addr 172.21.15.111:0/2582847229 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:24.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.396+0000 7f3f5bfff700 1 --2- 172.21.15.111:0/2582847229 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f5c1601e0 0x7f3f5c15f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:24.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.396+0000 7f3f6119c700 1 --2- 172.21.15.111:0/2582847229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f5c160d40 0x7f3f5c0b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:24.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.396+0000 7f3f6099b700 1 -- 172.21.15.111:0/2582847229 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f5c1601e0 msgr2=0x7f3f5c15f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:24.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.396+0000 7f3f6099b700 1 --2- 172.21.15.111:0/2582847229 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f5c1601e0 0x7f3f5c15f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:24.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.396+0000 7f3f6099b700 1 -- 172.21.15.111:0/2582847229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f5c160d40 msgr2=0x7f3f5c0b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:24.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.396+0000 7f3f6099b700 1 --2- 172.21.15.111:0/2582847229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f5c160d40 0x7f3f5c0b6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:24.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.396+0000 7f3f6099b700 1 -- 172.21.15.111:0/2582847229 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f500094a0 con 0x7f3f5c0b87a0 2023-03-29T10:58:24.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.396+0000 7f3f5bfff700 1 --2- 172.21.15.111:0/2582847229 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f5c1601e0 0x7f3f5c15f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:24.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.396+0000 7f3f6119c700 1 --2- 172.21.15.111:0/2582847229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f5c160d40 0x7f3f5c0b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:24.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.396+0000 7f3f6099b700 1 --2- 172.21.15.111:0/2582847229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f5c0b87a0 0x7f3f5c14d380 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009fd0 tx=0x7f3f50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:24.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.396+0000 7f3f597fa700 1 -- 172.21.15.111:0/2582847229 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f500036a0 con 0x7f3f5c0b87a0 2023-03-29T10:58:24.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.396+0000 7f3f597fa700 1 -- 172.21.15.111:0/2582847229 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5001f070 con 0x7f3f5c0b87a0 2023-03-29T10:58:24.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.396+0000 7f3f62bff700 1 -- 172.21.15.111:0/2582847229 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f5c0d1640 con 0x7f3f5c0b87a0 2023-03-29T10:58:24.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.396+0000 7f3f597fa700 1 -- 172.21.15.111:0/2582847229 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f50003a60 con 0x7f3f5c0b87a0 2023-03-29T10:58:24.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.397+0000 7f3f62bff700 1 -- 172.21.15.111:0/2582847229 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f5c0cd980 con 0x7f3f5c0b87a0 2023-03-29T10:58:24.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.398+0000 7f3f62bff700 1 -- 172.21.15.111:0/2582847229 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5c0d17d0 con 0x7f3f5c0b87a0 2023-03-29T10:58:24.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.398+0000 7f3f597fa700 1 -- 172.21.15.111:0/2582847229 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f3f50003bc0 con 0x7f3f5c0b87a0 2023-03-29T10:58:24.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.399+0000 7f3f597fa700 1 --2- 172.21.15.111:0/2582847229 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3f40069de0 0x7f3f4006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:24.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.399+0000 7f3f597fa700 1 -- 172.21.15.111:0/2582847229 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3f50021070 con 0x7f3f5c0b87a0 2023-03-29T10:58:24.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.400+0000 7f3f5bfff700 1 --2- 172.21.15.111:0/2582847229 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3f40069de0 0x7f3f4006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:24.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.400+0000 7f3f5bfff700 1 --2- 172.21.15.111:0/2582847229 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3f40069de0 0x7f3f4006c290 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3f48007fd0 tx=0x7f3f48009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:24.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.403+0000 7f3f597fa700 1 -- 172.21.15.111:0/2582847229 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f3f50055280 con 0x7f3f5c0b87a0 2023-03-29T10:58:24.599 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.598+0000 7f3f62bff700 1 -- 172.21.15.111:0/2582847229 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3f5c074430 con 0x7f3f5c0b87a0 2023-03-29T10:58:24.599 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.599+0000 7f3f597fa700 1 -- 172.21.15.111:0/2582847229 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3f500150a0 con 0x7f3f5c0b87a0 2023-03-29T10:58:24.599 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-03-29T10:58:24.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.601+0000 7f3f62bff700 1 -- 172.21.15.111:0/2582847229 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3f40069de0 msgr2=0x7f3f4006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:24.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.601+0000 7f3f62bff700 1 --2- 172.21.15.111:0/2582847229 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3f40069de0 0x7f3f4006c290 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3f48007fd0 tx=0x7f3f48009040 comp rx=0 tx=0).stop 2023-03-29T10:58:24.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.601+0000 7f3f62bff700 1 -- 172.21.15.111:0/2582847229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f5c0b87a0 msgr2=0x7f3f5c14d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:24.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.601+0000 7f3f62bff700 1 --2- 172.21.15.111:0/2582847229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f5c0b87a0 0x7f3f5c14d380 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3f50009fd0 tx=0x7f3f50004750 comp rx=0 tx=0).stop 2023-03-29T10:58:24.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.601+0000 7f3f62bff700 1 -- 172.21.15.111:0/2582847229 shutdown_connections 2023-03-29T10:58:24.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.601+0000 7f3f62bff700 1 --2- 172.21.15.111:0/2582847229 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3f40069de0 0x7f3f4006c290 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:24.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.601+0000 7f3f62bff700 1 --2- 172.21.15.111:0/2582847229 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f5c0b87a0 0x7f3f5c14d380 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:24.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.601+0000 7f3f62bff700 1 --2- 172.21.15.111:0/2582847229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f5c160d40 0x7f3f5c0b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:24.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.601+0000 7f3f62bff700 1 --2- 172.21.15.111:0/2582847229 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f5c1601e0 0x7f3f5c15f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:24.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.601+0000 7f3f62bff700 1 -- 172.21.15.111:0/2582847229 >> 172.21.15.111:0/2582847229 conn(0x7f3f5c071b10 msgr2=0x7f3f5c071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:24.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.601+0000 7f3f62bff700 1 -- 172.21.15.111:0/2582847229 shutdown_connections 2023-03-29T10:58:24.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.601+0000 7f3f62bff700 1 -- 172.21.15.111:0/2582847229 wait complete. 2023-03-29T10:58:24.619 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-29T10:58:24.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.804+0000 7f094e5ff700 1 Processor -- start 2023-03-29T10:58:24.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.804+0000 7f094e5ff700 1 -- start start 2023-03-29T10:58:24.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.805+0000 7f094e5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09480d2110 0x7f0948167550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:24.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.805+0000 7f094e5ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0948161000 con 0x7f09480cfbf0 2023-03-29T10:58:24.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.805+0000 7f094e5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09480d0020 con 0x7f09480b87a0 2023-03-29T10:58:24.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.805+0000 7f094e5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09480d02d0 con 0x7f09480d2110 2023-03-29T10:58:24.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.806+0000 7f093ffff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f09480cfbf0 0x7f09481608f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:39894/0 (socket says 172.21.15.111:39894) 2023-03-29T10:58:24.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.806+0000 7f093ffff700 1 -- 172.21.15.111:0/1579210584 learned_addr learned my addr 172.21.15.111:0/1579210584 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:24.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.806+0000 7f094cb9c700 1 --2- 172.21.15.111:0/1579210584 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09480d2110 0x7f0948167550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:24.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.806+0000 7f09477fe700 1 -- 172.21.15.111:0/1579210584 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507522391 0 0) 0x7f0948161000 con 0x7f09480cfbf0 2023-03-29T10:58:24.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.806+0000 7f09477fe700 1 -- 172.21.15.111:0/1579210584 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09340035b0 con 0x7f09480cfbf0 2023-03-29T10:58:24.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.806+0000 7f09477fe700 1 -- 172.21.15.111:0/1579210584 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 483775080 0 0) 0x7f09480d0020 con 0x7f09480b87a0 2023-03-29T10:58:24.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.806+0000 7f094cb9c700 1 -- 172.21.15.111:0/1579210584 >> v1:172.21.15.119:6789/0 conn(0x7f09480b87a0 legacy=0x7f09481601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:24.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.806+0000 7f094cb9c700 1 -- 172.21.15.111:0/1579210584 >> v1:172.21.15.111:6789/0 conn(0x7f09480cfbf0 legacy=0x7f09481608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:24.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.806+0000 7f094cb9c700 1 -- 172.21.15.111:0/1579210584 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09480cdc60 con 0x7f09480d2110 2023-03-29T10:58:24.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.807+0000 7f094cb9c700 1 --2- 172.21.15.111:0/1579210584 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09480d2110 0x7f0948167550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0940009dc0 tx=0x7f094000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a17e1e98b81f946c server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:24.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.807+0000 7f09477fe700 1 -- 172.21.15.111:0/1579210584 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094000e070 con 0x7f09480d2110 2023-03-29T10:58:24.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.807+0000 7f09477fe700 1 -- 172.21.15.111:0/1579210584 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094000bb20 con 0x7f09480d2110 2023-03-29T10:58:24.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.807+0000 7f09477fe700 1 -- 172.21.15.111:0/1579210584 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094000bed0 con 0x7f09480d2110 2023-03-29T10:58:24.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.807+0000 7f094e5ff700 1 -- 172.21.15.111:0/1579210584 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09480d2110 msgr2=0x7f0948167550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:24.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.807+0000 7f094e5ff700 1 --2- 172.21.15.111:0/1579210584 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09480d2110 0x7f0948167550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0940009dc0 tx=0x7f094000b330 comp rx=0 tx=0).stop 2023-03-29T10:58:24.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.807+0000 7f094e5ff700 1 -- 172.21.15.111:0/1579210584 shutdown_connections 2023-03-29T10:58:24.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.807+0000 7f094e5ff700 1 --2- 172.21.15.111:0/1579210584 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09480d2110 0x7f0948167550 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:24.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.807+0000 7f094e5ff700 1 -- 172.21.15.111:0/1579210584 >> 172.21.15.111:0/1579210584 conn(0x7f0948071b10 msgr2=0x7f0948071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:24.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.807+0000 7f094e5ff700 1 -- 172.21.15.111:0/1579210584 shutdown_connections 2023-03-29T10:58:24.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.807+0000 7f094e5ff700 1 -- 172.21.15.111:0/1579210584 wait complete. 2023-03-29T10:58:24.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.809+0000 7f094e5ff700 1 Processor -- start 2023-03-29T10:58:24.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.809+0000 7f094e5ff700 1 -- start start 2023-03-29T10:58:24.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.810+0000 7f094e5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09480b87a0 0x7f09480cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:24.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.810+0000 7f094e5ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09480cfbf0 0x7f09481552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:24.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.810+0000 7f094e5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09480d2110 0x7f09480c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:24.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.810+0000 7f094e5ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094815df00 con 0x7f09480cfbf0 2023-03-29T10:58:24.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.810+0000 7f094e5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094815e1e0 con 0x7f09480d2110 2023-03-29T10:58:24.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.810+0000 7f094e5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f094815e4c0 con 0x7f09480b87a0 2023-03-29T10:58:24.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.810+0000 7f093ffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09480cfbf0 0x7f09481552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:24.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.810+0000 7f093ffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09480cfbf0 0x7f09481552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53030/0 (socket says 172.21.15.111:53030) 2023-03-29T10:58:24.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.810+0000 7f093ffff700 1 -- 172.21.15.111:0/2725798402 learned_addr learned my addr 172.21.15.111:0/2725798402 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:24.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.810+0000 7f094cb9c700 1 --2- 172.21.15.111:0/2725798402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09480d2110 0x7f09480c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:24.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.810+0000 7f0947fff700 1 --2- 172.21.15.111:0/2725798402 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09480b87a0 0x7f09480cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:24.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.811+0000 7f093ffff700 1 -- 172.21.15.111:0/2725798402 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09480b87a0 msgr2=0x7f09480cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:24.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.811+0000 7f093ffff700 1 --2- 172.21.15.111:0/2725798402 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09480b87a0 0x7f09480cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:24.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.811+0000 7f093ffff700 1 -- 172.21.15.111:0/2725798402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09480d2110 msgr2=0x7f09480c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:24.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.811+0000 7f093ffff700 1 --2- 172.21.15.111:0/2725798402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09480d2110 0x7f09480c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:24.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.811+0000 7f093ffff700 1 -- 172.21.15.111:0/2725798402 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0940009a70 con 0x7f09480cfbf0 2023-03-29T10:58:24.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.811+0000 7f093ffff700 1 --2- 172.21.15.111:0/2725798402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09480cfbf0 0x7f09481552e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f093000c4d0 tx=0x7f093000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:24.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.811+0000 7f09457fa700 1 -- 172.21.15.111:0/2725798402 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930005d70 con 0x7f09480cfbf0 2023-03-29T10:58:24.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.811+0000 7f09457fa700 1 -- 172.21.15.111:0/2725798402 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09300070d0 con 0x7f09480cfbf0 2023-03-29T10:58:24.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.811+0000 7f094e5ff700 1 -- 172.21.15.111:0/2725798402 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09480cd980 con 0x7f09480cfbf0 2023-03-29T10:58:24.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.811+0000 7f09457fa700 1 -- 172.21.15.111:0/2725798402 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930007440 con 0x7f09480cfbf0 2023-03-29T10:58:24.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.811+0000 7f094e5ff700 1 -- 172.21.15.111:0/2725798402 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09480cd6a0 con 0x7f09480cfbf0 2023-03-29T10:58:24.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.812+0000 7f094e5ff700 1 -- 172.21.15.111:0/2725798402 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0948167a90 con 0x7f09480cfbf0 2023-03-29T10:58:24.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.813+0000 7f09457fa700 1 -- 172.21.15.111:0/2725798402 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f09300075a0 con 0x7f09480cfbf0 2023-03-29T10:58:24.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.814+0000 7f09457fa700 1 --2- 172.21.15.111:0/2725798402 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0928069de0 0x7f092806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:24.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.814+0000 7f09457fa700 1 -- 172.21.15.111:0/2725798402 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0930087650 con 0x7f09480cfbf0 2023-03-29T10:58:24.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.814+0000 7f0947fff700 1 --2- 172.21.15.111:0/2725798402 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0928069de0 0x7f092806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:24.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.815+0000 7f0947fff700 1 --2- 172.21.15.111:0/2725798402 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0928069de0 0x7f092806c290 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0938000f60 tx=0x7f0938009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:24.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:24.818+0000 7f09457fa700 1 -- 172.21.15.111:0/2725798402 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f093004c160 con 0x7f09480cfbf0 2023-03-29T10:58:25.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.011+0000 7f094e5ff700 1 -- 172.21.15.111:0/2725798402 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f094815e040 con 0x7f09480cfbf0 2023-03-29T10:58:25.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.011+0000 7f09457fa700 1 -- 172.21.15.111:0/2725798402 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f094815e040 con 0x7f09480cfbf0 2023-03-29T10:58:25.012 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:25.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.013+0000 7f094e5ff700 1 -- 172.21.15.111:0/2725798402 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0928069de0 msgr2=0x7f092806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:25.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.013+0000 7f094e5ff700 1 --2- 172.21.15.111:0/2725798402 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0928069de0 0x7f092806c290 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0938000f60 tx=0x7f0938009040 comp rx=0 tx=0).stop 2023-03-29T10:58:25.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.013+0000 7f094e5ff700 1 -- 172.21.15.111:0/2725798402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09480cfbf0 msgr2=0x7f09481552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:25.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.013+0000 7f094e5ff700 1 --2- 172.21.15.111:0/2725798402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09480cfbf0 0x7f09481552e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f093000c4d0 tx=0x7f093000c7e0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.013+0000 7f094e5ff700 1 -- 172.21.15.111:0/2725798402 shutdown_connections 2023-03-29T10:58:25.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.013+0000 7f094e5ff700 1 --2- 172.21.15.111:0/2725798402 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0928069de0 0x7f092806c290 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.013+0000 7f094e5ff700 1 --2- 172.21.15.111:0/2725798402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09480d2110 0x7f09480c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.013+0000 7f094e5ff700 1 --2- 172.21.15.111:0/2725798402 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f09480b87a0 0x7f09480cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.013+0000 7f094e5ff700 1 --2- 172.21.15.111:0/2725798402 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f09480cfbf0 0x7f09481552e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.013+0000 7f094e5ff700 1 -- 172.21.15.111:0/2725798402 >> 172.21.15.111:0/2725798402 conn(0x7f0948071b10 msgr2=0x7f094815c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:25.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.013+0000 7f094e5ff700 1 -- 172.21.15.111:0/2725798402 shutdown_connections 2023-03-29T10:58:25.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.014+0000 7f094e5ff700 1 -- 172.21.15.111:0/2725798402 wait complete. 2023-03-29T10:58:25.031 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:58:25.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.215+0000 7f203a1ff700 1 Processor -- start 2023-03-29T10:58:25.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.215+0000 7f203a1ff700 1 -- start start 2023-03-29T10:58:25.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.216+0000 7f203a1ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f20340cf970 0x7f20340cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:25.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.216+0000 7f203a1ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f20340b70f0 0x7f20340b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:25.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.216+0000 7f203a1ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20340cdc10 con 0x7f20340cfd40 2023-03-29T10:58:25.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.216+0000 7f203a1ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20340d08d0 con 0x7f20340b70f0 2023-03-29T10:58:25.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.216+0000 7f203a1ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20340b6c60 con 0x7f20340cf970 2023-03-29T10:58:25.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.216+0000 7f2033fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f20340b70f0 0x7f20340b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:25.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.216+0000 7f2033fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f20340b70f0 0x7f20340b74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53036/0 (socket says 172.21.15.111:53036) 2023-03-29T10:58:25.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.217+0000 7f2033fff700 1 -- 172.21.15.111:0/174510009 learned_addr learned my addr 172.21.15.111:0/174510009 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:25.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.217+0000 7f20337fe700 1 --2- 172.21.15.111:0/174510009 >> v2:172.21.15.119:3300/0 conn(0x7f20340cf970 0x7f20340cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:25.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.217+0000 7f2033fff700 1 -- 172.21.15.111:0/174510009 >> v2:172.21.15.119:3300/0 conn(0x7f20340cf970 msgr2=0x7f20340cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:25.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.217+0000 7f2033fff700 1 --2- 172.21.15.111:0/174510009 >> v2:172.21.15.119:3300/0 conn(0x7f20340cf970 0x7f20340cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.217+0000 7f2033fff700 1 -- 172.21.15.111:0/174510009 >> v1:172.21.15.111:6789/0 conn(0x7f20340cfd40 legacy=0x7f20340d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:25.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.217+0000 7f2033fff700 1 -- 172.21.15.111:0/174510009 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20340cd650 con 0x7f20340b70f0 2023-03-29T10:58:25.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.217+0000 7f20327fc700 1 -- 172.21.15.111:0/174510009 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930771367 0 0) 0x7f20340cdc10 con 0x7f20340cfd40 2023-03-29T10:58:25.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.217+0000 7f2033fff700 1 --2- 172.21.15.111:0/174510009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20340b70f0 0x7f20340b74f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f20340d0a10 tx=0x7f202c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed5ee8e42eaf682 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:25.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.217+0000 7f20327fc700 1 -- 172.21.15.111:0/174510009 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c00e070 con 0x7f20340b70f0 2023-03-29T10:58:25.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.217+0000 7f20327fc700 1 -- 172.21.15.111:0/174510009 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c00b9c0 con 0x7f20340b70f0 2023-03-29T10:58:25.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.217+0000 7f20327fc700 1 -- 172.21.15.111:0/174510009 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c00bd70 con 0x7f20340b70f0 2023-03-29T10:58:25.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.218+0000 7f203a1ff700 1 -- 172.21.15.111:0/174510009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20340b70f0 msgr2=0x7f20340b74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:25.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.218+0000 7f203a1ff700 1 --2- 172.21.15.111:0/174510009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20340b70f0 0x7f20340b74f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f20340d0a10 tx=0x7f202c00b330 comp rx=0 tx=0).stop 2023-03-29T10:58:25.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.218+0000 7f203a1ff700 1 -- 172.21.15.111:0/174510009 shutdown_connections 2023-03-29T10:58:25.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.218+0000 7f203a1ff700 1 --2- 172.21.15.111:0/174510009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20340b70f0 0x7f20340b74f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.218+0000 7f203a1ff700 1 --2- 172.21.15.111:0/174510009 >> v2:172.21.15.119:3300/0 conn(0x7f20340cf970 0x7f20340cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.218+0000 7f203a1ff700 1 -- 172.21.15.111:0/174510009 >> 172.21.15.111:0/174510009 conn(0x7f2034071b10 msgr2=0x7f2034071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:25.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.218+0000 7f203a1ff700 1 -- 172.21.15.111:0/174510009 shutdown_connections 2023-03-29T10:58:25.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.218+0000 7f203a1ff700 1 -- 172.21.15.111:0/174510009 wait complete. 2023-03-29T10:58:25.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.220+0000 7f203a1ff700 1 Processor -- start 2023-03-29T10:58:25.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.220+0000 7f203a1ff700 1 -- start start 2023-03-29T10:58:25.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.221+0000 7f203a1ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f20340b70f0 0x7f20340ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:25.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.221+0000 7f203a1ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20340cf970 0x7f20340c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:25.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.221+0000 7f203a1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20340cfd40 0x7f20340cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:25.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.221+0000 7f203a1ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2034153c40 con 0x7f20340cf970 2023-03-29T10:58:25.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.221+0000 7f203a1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20340ca610 con 0x7f20340cfd40 2023-03-29T10:58:25.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.221+0000 7f203a1ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20340ca8f0 con 0x7f20340b70f0 2023-03-29T10:58:25.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.221+0000 7f2032ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20340cf970 0x7f20340c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:25.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.221+0000 7f2032ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20340cf970 0x7f20340c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53044/0 (socket says 172.21.15.111:53044) 2023-03-29T10:58:25.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.221+0000 7f2032ffd700 1 -- 172.21.15.111:0/3814479115 learned_addr learned my addr 172.21.15.111:0/3814479115 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:25.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.221+0000 7f2033fff700 1 --2- 172.21.15.111:0/3814479115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20340cfd40 0x7f20340cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:25.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.221+0000 7f20337fe700 1 --2- 172.21.15.111:0/3814479115 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f20340b70f0 0x7f20340ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:25.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.222+0000 7f2032ffd700 1 -- 172.21.15.111:0/3814479115 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f20340b70f0 msgr2=0x7f20340ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:25.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.222+0000 7f2032ffd700 1 --2- 172.21.15.111:0/3814479115 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f20340b70f0 0x7f20340ca030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.222+0000 7f2032ffd700 1 -- 172.21.15.111:0/3814479115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20340cfd40 msgr2=0x7f20340cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:25.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.222+0000 7f2032ffd700 1 --2- 172.21.15.111:0/3814479115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20340cfd40 0x7f20340cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.222+0000 7f2032ffd700 1 -- 172.21.15.111:0/3814479115 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c009b30 con 0x7f20340cf970 2023-03-29T10:58:25.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.222+0000 7f2033fff700 1 --2- 172.21.15.111:0/3814479115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20340cfd40 0x7f20340cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:25.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.222+0000 7f2032ffd700 1 --2- 172.21.15.111:0/3814479115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20340cf970 0x7f20340c8c10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f202800c550 tx=0x7f202800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:25.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.222+0000 7f201bfff700 1 -- 172.21.15.111:0/3814479115 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20280054e0 con 0x7f20340cf970 2023-03-29T10:58:25.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.222+0000 7f201bfff700 1 -- 172.21.15.111:0/3814479115 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2028003680 con 0x7f20340cf970 2023-03-29T10:58:25.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.222+0000 7f201bfff700 1 -- 172.21.15.111:0/3814479115 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2028003a30 con 0x7f20340cf970 2023-03-29T10:58:25.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.222+0000 7f203a1ff700 1 -- 172.21.15.111:0/3814479115 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20340b68f0 con 0x7f20340cf970 2023-03-29T10:58:25.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.222+0000 7f203a1ff700 1 -- 172.21.15.111:0/3814479115 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20340cd930 con 0x7f20340cf970 2023-03-29T10:58:25.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.223+0000 7f203a1ff700 1 -- 172.21.15.111:0/3814479115 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20340cf650 con 0x7f20340cf970 2023-03-29T10:58:25.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.224+0000 7f201bfff700 1 -- 172.21.15.111:0/3814479115 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f2028003b90 con 0x7f20340cf970 2023-03-29T10:58:25.225 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.225+0000 7f201bfff700 1 --2- 172.21.15.111:0/3814479115 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2020069de0 0x7f202006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:25.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.225+0000 7f201bfff700 1 -- 172.21.15.111:0/3814479115 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f20280877f0 con 0x7f20340cf970 2023-03-29T10:58:25.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.225+0000 7f20337fe700 1 --2- 172.21.15.111:0/3814479115 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2020069de0 0x7f202006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:25.226 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.226+0000 7f20337fe700 1 --2- 172.21.15.111:0/3814479115 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2020069de0 0x7f202006c290 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f202400b9b0 tx=0x7f2024008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:25.229 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.229+0000 7f201bfff700 1 -- 172.21.15.111:0/3814479115 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f2028052cd0 con 0x7f20340cf970 2023-03-29T10:58:25.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.426+0000 7f203a1ff700 1 -- 172.21.15.111:0/3814479115 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20340b8960 con 0x7f20340cf970 2023-03-29T10:58:25.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.427+0000 7f201bfff700 1 -- 172.21.15.111:0/3814479115 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19892 (secure 0 0 0) 0x7f20280562f0 con 0x7f20340cf970 2023-03-29T10:58:25.428 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:25.428 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":30,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:58:09.476334+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.26","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:58:25.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.429+0000 7f203a1ff700 1 -- 172.21.15.111:0/3814479115 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2020069de0 msgr2=0x7f202006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:25.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.429+0000 7f203a1ff700 1 --2- 172.21.15.111:0/3814479115 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2020069de0 0x7f202006c290 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f202400b9b0 tx=0x7f2024008040 comp rx=0 tx=0).stop 2023-03-29T10:58:25.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.429+0000 7f203a1ff700 1 -- 172.21.15.111:0/3814479115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20340cf970 msgr2=0x7f20340c8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:25.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.429+0000 7f203a1ff700 1 --2- 172.21.15.111:0/3814479115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20340cf970 0x7f20340c8c10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f202800c550 tx=0x7f202800c910 comp rx=0 tx=0).stop 2023-03-29T10:58:25.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.429+0000 7f203a1ff700 1 -- 172.21.15.111:0/3814479115 shutdown_connections 2023-03-29T10:58:25.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.429+0000 7f203a1ff700 1 --2- 172.21.15.111:0/3814479115 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2020069de0 0x7f202006c290 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.429+0000 7f203a1ff700 1 --2- 172.21.15.111:0/3814479115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20340cfd40 0x7f20340cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.429+0000 7f203a1ff700 1 --2- 172.21.15.111:0/3814479115 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f20340b70f0 0x7f20340ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.429+0000 7f203a1ff700 1 --2- 172.21.15.111:0/3814479115 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f20340cf970 0x7f20340c8c10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.429+0000 7f203a1ff700 1 -- 172.21.15.111:0/3814479115 >> 172.21.15.111:0/3814479115 conn(0x7f2034071b10 msgr2=0x7f203415aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:25.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.430+0000 7f203a1ff700 1 -- 172.21.15.111:0/3814479115 shutdown_connections 2023-03-29T10:58:25.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.430+0000 7f203a1ff700 1 -- 172.21.15.111:0/3814479115 wait complete. 2023-03-29T10:58:25.448 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-29T10:57:16.183979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-29T10:57:21.239809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-29T10:57:26.192124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-29T10:57:26.796777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-29T10:57:29.874795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-29T10:58:25.448 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-29T10:58:25.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.635+0000 7f1ea15ff700 1 Processor -- start 2023-03-29T10:58:25.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.635+0000 7f1ea15ff700 1 -- start start 2023-03-29T10:58:25.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.636+0000 7f1ea15ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1e9c15a5a0 0x7f1e9c06a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:25.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.636+0000 7f1ea15ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e9c071200 0x7f1e9c06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:25.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.636+0000 7f1ea15ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e9c0cb280 con 0x7f1e9c06a920 2023-03-29T10:58:25.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.636+0000 7f1ea15ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e9c06adc0 con 0x7f1e9c15a5a0 2023-03-29T10:58:25.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.636+0000 7f1ea15ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c15a970 con 0x7f1e9c071200 2023-03-29T10:58:25.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.637+0000 7f1e9b7fe700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e9c071200 0x7f1e9c06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:25.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.637+0000 7f1e9affd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1e9c15a5a0 0x7f1e9c06a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:25.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.637+0000 7f1e9a7fc700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1e9c06a920 0x7f1e9c070af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.111:37134/0 (socket says 172.21.15.111:37134) 2023-03-29T10:58:25.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.637+0000 7f1e9a7fc700 1 -- 172.21.15.111:0/1077736496 learned_addr learned my addr 172.21.15.111:0/1077736496 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:25.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.637+0000 7f1e99ffb700 1 -- 172.21.15.111:0/1077736496 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799900313 0 0) 0x7f1e90001160 con 0x7f1e9c06a920 2023-03-29T10:58:25.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.637+0000 7f1e9b7fe700 1 -- 172.21.15.111:0/1077736496 >> v2:172.21.15.119:3300/0 conn(0x7f1e9c15a5a0 msgr2=0x7f1e9c06a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:25.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.637+0000 7f1e9b7fe700 1 --2- 172.21.15.111:0/1077736496 >> v2:172.21.15.119:3300/0 conn(0x7f1e9c15a5a0 0x7f1e9c06a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.637+0000 7f1e9b7fe700 1 -- 172.21.15.111:0/1077736496 >> v1:172.21.15.119:6789/0 conn(0x7f1e9c06a920 legacy=0x7f1e9c070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:25.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.637+0000 7f1e9b7fe700 1 -- 172.21.15.111:0/1077736496 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c071700 con 0x7f1e9c071200 2023-03-29T10:58:25.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.637+0000 7f1e9b7fe700 1 --2- 172.21.15.111:0/1077736496 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e9c071200 0x7f1e9c06d1a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1e94009d90 tx=0x7f1e9400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a6c305d80083d78 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:25.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.638+0000 7f1e99ffb700 1 -- 172.21.15.111:0/1077736496 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9400e070 con 0x7f1e9c071200 2023-03-29T10:58:25.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.638+0000 7f1e99ffb700 1 -- 172.21.15.111:0/1077736496 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9400bb20 con 0x7f1e9c071200 2023-03-29T10:58:25.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.638+0000 7f1e99ffb700 1 -- 172.21.15.111:0/1077736496 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e94012430 con 0x7f1e9c071200 2023-03-29T10:58:25.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.638+0000 7f1ea15ff700 1 -- 172.21.15.111:0/1077736496 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e9c071200 msgr2=0x7f1e9c06d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:25.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.638+0000 7f1ea15ff700 1 --2- 172.21.15.111:0/1077736496 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e9c071200 0x7f1e9c06d1a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1e94009d90 tx=0x7f1e9400b330 comp rx=0 tx=0).stop 2023-03-29T10:58:25.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.638+0000 7f1ea15ff700 1 -- 172.21.15.111:0/1077736496 shutdown_connections 2023-03-29T10:58:25.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.638+0000 7f1ea15ff700 1 --2- 172.21.15.111:0/1077736496 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e9c071200 0x7f1e9c06d1a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.638+0000 7f1ea15ff700 1 --2- 172.21.15.111:0/1077736496 >> v2:172.21.15.119:3300/0 conn(0x7f1e9c15a5a0 0x7f1e9c06a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.638+0000 7f1ea15ff700 1 -- 172.21.15.111:0/1077736496 >> 172.21.15.111:0/1077736496 conn(0x7f1e9c0cf720 msgr2=0x7f1e9c0cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:25.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.638+0000 7f1ea15ff700 1 -- 172.21.15.111:0/1077736496 shutdown_connections 2023-03-29T10:58:25.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.638+0000 7f1ea15ff700 1 -- 172.21.15.111:0/1077736496 wait complete. 2023-03-29T10:58:25.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.640+0000 7f1ea15ff700 1 Processor -- start 2023-03-29T10:58:25.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.640+0000 7f1ea15ff700 1 -- start start 2023-03-29T10:58:25.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.641+0000 7f1ea15ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e9c06a920 0x7f1e9c0c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:25.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.641+0000 7f1ea15ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e9c071200 0x7f1e9c0b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:25.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.641+0000 7f1ea15ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e9c15a5a0 0x7f1e9c1527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:25.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.641+0000 7f1ea15ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c0af530 con 0x7f1e9c071200 2023-03-29T10:58:25.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.641+0000 7f1ea15ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c0b4720 con 0x7f1e9c06a920 2023-03-29T10:58:25.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.641+0000 7f1ea15ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c1538b0 con 0x7f1e9c15a5a0 2023-03-29T10:58:25.644 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.641+0000 7f1e9a7fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e9c071200 0x7f1e9c0b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:25.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.641+0000 7f1e9a7fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e9c071200 0x7f1e9c0b4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53054/0 (socket says 172.21.15.111:53054) 2023-03-29T10:58:25.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.641+0000 7f1e9a7fc700 1 -- 172.21.15.111:0/3700010449 learned_addr learned my addr 172.21.15.111:0/3700010449 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:25.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.641+0000 7f1e9b7fe700 1 --2- 172.21.15.111:0/3700010449 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e9c15a5a0 0x7f1e9c1527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:25.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.641+0000 7f1e9affd700 1 --2- 172.21.15.111:0/3700010449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e9c06a920 0x7f1e9c0c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:25.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.642+0000 7f1e9a7fc700 1 -- 172.21.15.111:0/3700010449 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e9c15a5a0 msgr2=0x7f1e9c1527b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:25.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.642+0000 7f1e9a7fc700 1 --2- 172.21.15.111:0/3700010449 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e9c15a5a0 0x7f1e9c1527b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.642+0000 7f1e9a7fc700 1 -- 172.21.15.111:0/3700010449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e9c06a920 msgr2=0x7f1e9c0c5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:25.645 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.642+0000 7f1e9a7fc700 1 --2- 172.21.15.111:0/3700010449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e9c06a920 0x7f1e9c0c5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.642+0000 7f1e9a7fc700 1 -- 172.21.15.111:0/3700010449 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e94009a70 con 0x7f1e9c071200 2023-03-29T10:58:25.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.642+0000 7f1e9a7fc700 1 --2- 172.21.15.111:0/3700010449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e9c071200 0x7f1e9c0b4020 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000c7e0 tx=0x7f1e9000caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:25.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.642+0000 7f1e837fe700 1 -- 172.21.15.111:0/3700010449 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90004cd0 con 0x7f1e9c071200 2023-03-29T10:58:25.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.642+0000 7f1e837fe700 1 -- 172.21.15.111:0/3700010449 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e90004e30 con 0x7f1e9c071200 2023-03-29T10:58:25.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.642+0000 7f1ea15ff700 1 -- 172.21.15.111:0/3700010449 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9c06c6c0 con 0x7f1e9c071200 2023-03-29T10:58:25.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.642+0000 7f1e837fe700 1 -- 172.21.15.111:0/3700010449 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e90007390 con 0x7f1e9c071200 2023-03-29T10:58:25.646 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.642+0000 7f1ea15ff700 1 -- 172.21.15.111:0/3700010449 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9c0719e0 con 0x7f1e9c071200 2023-03-29T10:58:25.647 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.643+0000 7f1ea15ff700 1 -- 172.21.15.111:0/3700010449 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e9c06c9a0 con 0x7f1e9c071200 2023-03-29T10:58:25.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.644+0000 7f1e837fe700 1 -- 172.21.15.111:0/3700010449 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f1e900074f0 con 0x7f1e9c071200 2023-03-29T10:58:25.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.645+0000 7f1e837fe700 1 --2- 172.21.15.111:0/3700010449 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1e88069de0 0x7f1e8806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:25.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.645+0000 7f1e837fe700 1 -- 172.21.15.111:0/3700010449 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1e90087dc0 con 0x7f1e9c071200 2023-03-29T10:58:25.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.645+0000 7f1e9affd700 1 --2- 172.21.15.111:0/3700010449 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1e88069de0 0x7f1e8806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:25.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.646+0000 7f1e9affd700 1 --2- 172.21.15.111:0/3700010449 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1e88069de0 0x7f1e8806c290 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c009b40 tx=0x7f1e8c009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:25.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.649+0000 7f1e837fe700 1 -- 172.21.15.111:0/3700010449 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f1e900533d0 con 0x7f1e9c071200 2023-03-29T10:58:25.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.841+0000 7f1ea15ff700 1 -- 172.21.15.111:0/3700010449 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e9c0cb280 con 0x7f1e9c071200 2023-03-29T10:58:25.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.842+0000 7f1e837fe700 1 -- 172.21.15.111:0/3700010449 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e9c0cb280 con 0x7f1e9c071200 2023-03-29T10:58:25.842 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-03-29T10:58:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.844+0000 7f1ea15ff700 1 -- 172.21.15.111:0/3700010449 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1e88069de0 msgr2=0x7f1e8806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.844+0000 7f1ea15ff700 1 --2- 172.21.15.111:0/3700010449 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1e88069de0 0x7f1e8806c290 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c009b40 tx=0x7f1e8c009210 comp rx=0 tx=0).stop 2023-03-29T10:58:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.844+0000 7f1ea15ff700 1 -- 172.21.15.111:0/3700010449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e9c071200 msgr2=0x7f1e9c0b4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.844+0000 7f1ea15ff700 1 --2- 172.21.15.111:0/3700010449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e9c071200 0x7f1e9c0b4020 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1e9000c7e0 tx=0x7f1e9000caf0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.844+0000 7f1ea15ff700 1 -- 172.21.15.111:0/3700010449 shutdown_connections 2023-03-29T10:58:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.844+0000 7f1ea15ff700 1 --2- 172.21.15.111:0/3700010449 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1e88069de0 0x7f1e8806c290 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.844+0000 7f1ea15ff700 1 --2- 172.21.15.111:0/3700010449 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e9c15a5a0 0x7f1e9c1527b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.844+0000 7f1ea15ff700 1 --2- 172.21.15.111:0/3700010449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e9c06a920 0x7f1e9c0c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.844+0000 7f1ea15ff700 1 --2- 172.21.15.111:0/3700010449 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e9c071200 0x7f1e9c0b4020 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.844+0000 7f1ea15ff700 1 -- 172.21.15.111:0/3700010449 >> 172.21.15.111:0/3700010449 conn(0x7f1e9c0cf720 msgr2=0x7f1e9c070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.844+0000 7f1ea15ff700 1 -- 172.21.15.111:0/3700010449 shutdown_connections 2023-03-29T10:58:25.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:25.844+0000 7f1ea15ff700 1 -- 172.21.15.111:0/3700010449 wait complete. 2023-03-29T10:58:25.862 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-29T10:58:26.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.048+0000 7f1e4bfff700 1 Processor -- start 2023-03-29T10:58:26.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.048+0000 7f1e4bfff700 1 -- start start 2023-03-29T10:58:26.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.049+0000 7f1e4bfff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e4415c590 0x7f1e44160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.049+0000 7f1e4bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1e440cff50 0x7f1e440d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.049+0000 7f1e4bfff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e440cd520 con 0x7f1e440cf010 2023-03-29T10:58:26.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.049+0000 7f1e4bfff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e440d2640 con 0x7f1e440cff50 2023-03-29T10:58:26.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.049+0000 7f1e4bfff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e441610d0 con 0x7f1e4415c590 2023-03-29T10:58:26.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.050+0000 7f1e4959a700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1e440cf010 0x7f1e440cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:39912/0 (socket says 172.21.15.111:39912) 2023-03-29T10:58:26.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.050+0000 7f1e4959a700 1 -- 172.21.15.111:0/1686938050 learned_addr learned my addr 172.21.15.111:0/1686938050 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:26.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.050+0000 7f1e49d9b700 1 --2- 172.21.15.111:0/1686938050 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e4415c590 0x7f1e44160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.050+0000 7f1e48d99700 1 -- 172.21.15.111:0/1686938050 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559061949 0 0) 0x7f1e440cd520 con 0x7f1e440cf010 2023-03-29T10:58:26.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.050+0000 7f1e4a59c700 1 --2- 172.21.15.111:0/1686938050 >> v2:172.21.15.119:3300/0 conn(0x7f1e440cff50 0x7f1e440d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.050+0000 7f1e48d99700 1 -- 172.21.15.111:0/1686938050 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e300035b0 con 0x7f1e440cf010 2023-03-29T10:58:26.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.050+0000 7f1e48d99700 1 -- 172.21.15.111:0/1686938050 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1489691798 0 0) 0x7f1e300035b0 con 0x7f1e440cf010 2023-03-29T10:58:26.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.050+0000 7f1e48d99700 1 -- 172.21.15.111:0/1686938050 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e4415c590 msgr2=0x7f1e44160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:26.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.050+0000 7f1e48d99700 1 --2- 172.21.15.111:0/1686938050 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e4415c590 0x7f1e44160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.050+0000 7f1e48d99700 1 -- 172.21.15.111:0/1686938050 >> v2:172.21.15.119:3300/0 conn(0x7f1e440cff50 msgr2=0x7f1e440d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:26.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.050+0000 7f1e48d99700 1 --2- 172.21.15.111:0/1686938050 >> v2:172.21.15.119:3300/0 conn(0x7f1e440cff50 0x7f1e440d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.050+0000 7f1e48d99700 1 -- 172.21.15.111:0/1686938050 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e440b23c0 con 0x7f1e440cf010 2023-03-29T10:58:26.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.050+0000 7f1e48d99700 1 -- 172.21.15.111:0/1686938050 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f1e34001670 con 0x7f1e440cf010 2023-03-29T10:58:26.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.050+0000 7f1e48d99700 1 -- 172.21.15.111:0/1686938050 >> v1:172.21.15.111:6789/0 conn(0x7f1e440cf010 legacy=0x7f1e440cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:26.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.051+0000 7f1e48d99700 1 --2- 172.21.15.111:0/1686938050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e30003a00 0x7f1e30003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.051+0000 7f1e48d99700 1 --2- 172.21.15.111:0/1686938050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e30004430 0x7f1e300048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.051+0000 7f1e48d99700 1 --2- 172.21.15.111:0/1686938050 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e4415c590 0x7f1e30004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.051+0000 7f1e48d99700 1 -- 172.21.15.111:0/1686938050 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440d2640 con 0x7f1e30003a00 2023-03-29T10:58:26.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.051+0000 7f1e48d99700 1 -- 172.21.15.111:0/1686938050 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e441610d0 con 0x7f1e30004430 2023-03-29T10:58:26.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.051+0000 7f1e48d99700 1 -- 172.21.15.111:0/1686938050 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3000ba10 con 0x7f1e4415c590 2023-03-29T10:58:26.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.051+0000 7f1e49d9b700 1 --2- 172.21.15.111:0/1686938050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e30003a00 0x7f1e30003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.051+0000 7f1e4959a700 1 --2- 172.21.15.111:0/1686938050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e30004430 0x7f1e300048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.051+0000 7f1e4a59c700 1 --2- 172.21.15.111:0/1686938050 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e4415c590 0x7f1e30004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.051+0000 7f1e49d9b700 1 -- 172.21.15.111:0/1686938050 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e4415c590 msgr2=0x7f1e30004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:26.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.051+0000 7f1e49d9b700 1 --2- 172.21.15.111:0/1686938050 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e4415c590 0x7f1e30004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.051+0000 7f1e49d9b700 1 -- 172.21.15.111:0/1686938050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e30004430 msgr2=0x7f1e300048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:26.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.051+0000 7f1e49d9b700 1 --2- 172.21.15.111:0/1686938050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e30004430 0x7f1e300048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.051+0000 7f1e49d9b700 1 -- 172.21.15.111:0/1686938050 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e300035b0 con 0x7f1e30003a00 2023-03-29T10:58:26.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.051+0000 7f1e4959a700 1 --2- 172.21.15.111:0/1686938050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e30004430 0x7f1e300048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:26.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.051+0000 7f1e4a59c700 1 --2- 172.21.15.111:0/1686938050 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e4415c590 0x7f1e30004fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:26.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.051+0000 7f1e49d9b700 1 --2- 172.21.15.111:0/1686938050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e30003a00 0x7f1e30003e10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009b60 tx=0x7f1e30005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630596a4fbb5a883 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:26.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.052+0000 7f1e48d99700 1 -- 172.21.15.111:0/1686938050 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4000e6c0 con 0x7f1e30003a00 2023-03-29T10:58:26.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.052+0000 7f1e48d99700 1 -- 172.21.15.111:0/1686938050 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e4000e820 con 0x7f1e30003a00 2023-03-29T10:58:26.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.052+0000 7f1e48d99700 1 -- 172.21.15.111:0/1686938050 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e4000eb90 con 0x7f1e30003a00 2023-03-29T10:58:26.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.052+0000 7f1e4bfff700 1 -- 172.21.15.111:0/1686938050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e30003a00 msgr2=0x7f1e30003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:26.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.052+0000 7f1e4bfff700 1 --2- 172.21.15.111:0/1686938050 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e30003a00 0x7f1e30003e10 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1e40009b60 tx=0x7f1e30005880 comp rx=0 tx=0).stop 2023-03-29T10:58:26.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.052+0000 7f1e4a59c700 1 -- 172.21.15.111:0/1686938050 reap_dead start 2023-03-29T10:58:26.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.052+0000 7f1e4bfff700 1 -- 172.21.15.111:0/1686938050 shutdown_connections 2023-03-29T10:58:26.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.052+0000 7f1e4bfff700 1 -- 172.21.15.111:0/1686938050 >> 172.21.15.111:0/1686938050 conn(0x7f1e44071b10 msgr2=0x7f1e44071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:26.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.052+0000 7f1e4bfff700 1 -- 172.21.15.111:0/1686938050 shutdown_connections 2023-03-29T10:58:26.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.052+0000 7f1e4bfff700 1 -- 172.21.15.111:0/1686938050 wait complete. 2023-03-29T10:58:26.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.054+0000 7f1e4bfff700 1 Processor -- start 2023-03-29T10:58:26.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.054+0000 7f1e4bfff700 1 -- start start 2023-03-29T10:58:26.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.055+0000 7f1e4bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e30003a00 0x7f1e441f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.055+0000 7f1e4bfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e440b1120 0x7f1e440cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.055+0000 7f1e4bfff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e440cc510 0x7f1e441f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.055+0000 7f1e4bfff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4415f9d0 con 0x7f1e440b1120 2023-03-29T10:58:26.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.055+0000 7f1e4bfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440cc9b0 con 0x7f1e30003a00 2023-03-29T10:58:26.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.055+0000 7f1e4bfff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e441f30e0 con 0x7f1e440cc510 2023-03-29T10:58:26.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.055+0000 7f1e4959a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e440b1120 0x7f1e440cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.055+0000 7f1e4959a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e440b1120 0x7f1e440cbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53062/0 (socket says 172.21.15.111:53062) 2023-03-29T10:58:26.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.055+0000 7f1e4959a700 1 -- 172.21.15.111:0/2416414039 learned_addr learned my addr 172.21.15.111:0/2416414039 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:26.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.055+0000 7f1e49d9b700 1 --2- 172.21.15.111:0/2416414039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e30003a00 0x7f1e441f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.055+0000 7f1e4a59c700 1 --2- 172.21.15.111:0/2416414039 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e440cc510 0x7f1e441f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.055+0000 7f1e4959a700 1 -- 172.21.15.111:0/2416414039 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e440cc510 msgr2=0x7f1e441f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:26.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.056+0000 7f1e4959a700 1 --2- 172.21.15.111:0/2416414039 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e440cc510 0x7f1e441f5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.056+0000 7f1e4959a700 1 -- 172.21.15.111:0/2416414039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e30003a00 msgr2=0x7f1e441f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:26.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.056+0000 7f1e4959a700 1 --2- 172.21.15.111:0/2416414039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e30003a00 0x7f1e441f2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.056+0000 7f1e4959a700 1 -- 172.21.15.111:0/2416414039 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e440cd770 con 0x7f1e440b1120 2023-03-29T10:58:26.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.056+0000 7f1e4959a700 1 --2- 172.21.15.111:0/2416414039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e440b1120 0x7f1e440cbfd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400eb20 tx=0x7f1e3400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:26.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.056+0000 7f1e3a7fc700 1 -- 172.21.15.111:0/2416414039 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e34005e00 con 0x7f1e440b1120 2023-03-29T10:58:26.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.056+0000 7f1e3a7fc700 1 -- 172.21.15.111:0/2416414039 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3400c7e0 con 0x7f1e440b1120 2023-03-29T10:58:26.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.056+0000 7f1e4bfff700 1 -- 172.21.15.111:0/2416414039 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e440cf560 con 0x7f1e440b1120 2023-03-29T10:58:26.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.056+0000 7f1e3a7fc700 1 -- 172.21.15.111:0/2416414039 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3400cb70 con 0x7f1e440b1120 2023-03-29T10:58:26.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.056+0000 7f1e4bfff700 1 -- 172.21.15.111:0/2416414039 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4415c960 con 0x7f1e440b1120 2023-03-29T10:58:26.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.057+0000 7f1e4bfff700 1 -- 172.21.15.111:0/2416414039 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e440b51f0 con 0x7f1e440b1120 2023-03-29T10:58:26.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.058+0000 7f1e3a7fc700 1 -- 172.21.15.111:0/2416414039 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f1e3400ccd0 con 0x7f1e440b1120 2023-03-29T10:58:26.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.059+0000 7f1e3a7fc700 1 --2- 172.21.15.111:0/2416414039 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1e28069de0 0x7f1e2806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.059+0000 7f1e3a7fc700 1 -- 172.21.15.111:0/2416414039 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1e34088240 con 0x7f1e440b1120 2023-03-29T10:58:26.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.059+0000 7f1e49d9b700 1 --2- 172.21.15.111:0/2416414039 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1e28069de0 0x7f1e2806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.060+0000 7f1e49d9b700 1 --2- 172.21.15.111:0/2416414039 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1e28069de0 0x7f1e2806c290 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1e400090b0 tx=0x7f1e40008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:26.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.063+0000 7f1e3a7fc700 1 -- 172.21.15.111:0/2416414039 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f1e34057150 con 0x7f1e440b1120 2023-03-29T10:58:26.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.258+0000 7f1e4bfff700 1 -- 172.21.15.111:0/2416414039 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1e440caa60 con 0x7f1e440b1120 2023-03-29T10:58:26.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.259+0000 7f1e3a7fc700 1 -- 172.21.15.111:0/2416414039 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1e34056d40 con 0x7f1e440b1120 2023-03-29T10:58:26.260 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 8 2023-03-29T10:58:26.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.261+0000 7f1e4bfff700 1 -- 172.21.15.111:0/2416414039 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1e28069de0 msgr2=0x7f1e2806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:26.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.261+0000 7f1e4bfff700 1 --2- 172.21.15.111:0/2416414039 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1e28069de0 0x7f1e2806c290 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1e400090b0 tx=0x7f1e40008030 comp rx=0 tx=0).stop 2023-03-29T10:58:26.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.261+0000 7f1e4bfff700 1 -- 172.21.15.111:0/2416414039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e440b1120 msgr2=0x7f1e440cbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:26.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.261+0000 7f1e4bfff700 1 --2- 172.21.15.111:0/2416414039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e440b1120 0x7f1e440cbfd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1e3400eb20 tx=0x7f1e3400eee0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.261+0000 7f1e4bfff700 1 -- 172.21.15.111:0/2416414039 shutdown_connections 2023-03-29T10:58:26.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.261+0000 7f1e4bfff700 1 --2- 172.21.15.111:0/2416414039 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1e28069de0 0x7f1e2806c290 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.261+0000 7f1e4bfff700 1 --2- 172.21.15.111:0/2416414039 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1e440cc510 0x7f1e441f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.261+0000 7f1e4bfff700 1 --2- 172.21.15.111:0/2416414039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e30003a00 0x7f1e441f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.262+0000 7f1e4bfff700 1 --2- 172.21.15.111:0/2416414039 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1e440b1120 0x7f1e440cbfd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.262+0000 7f1e4bfff700 1 -- 172.21.15.111:0/2416414039 >> 172.21.15.111:0/2416414039 conn(0x7f1e44071b10 msgr2=0x7f1e440b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:26.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.262+0000 7f1e4bfff700 1 -- 172.21.15.111:0/2416414039 shutdown_connections 2023-03-29T10:58:26.263 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.262+0000 7f1e4bfff700 1 -- 172.21.15.111:0/2416414039 wait complete. 2023-03-29T10:58:26.279 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-29T10:58:26.467 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.466+0000 7f70fb3ff700 1 Processor -- start 2023-03-29T10:58:26.467 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.466+0000 7f70fb3ff700 1 -- start start 2023-03-29T10:58:26.468 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.467+0000 7f70fb3ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f70f40b87a0 0x7f70f40cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.468 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.467+0000 7f70fb3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70f41601e0 0x7f70f4160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.468 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.467+0000 7f70fb3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f4160ff0 con 0x7f70f4160b70 2023-03-29T10:58:26.468 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.467+0000 7f70fb3ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f40d02d0 con 0x7f70f40b87a0 2023-03-29T10:58:26.468 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.467+0000 7f70fb3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f40d0cb0 con 0x7f70f41601e0 2023-03-29T10:58:26.469 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.467+0000 7f70f919b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f70f40b87a0 0x7f70f40cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.469 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.468+0000 7f70f919b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f70f40b87a0 0x7f70f40cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53064/0 (socket says 172.21.15.111:53064) 2023-03-29T10:58:26.469 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.468+0000 7f70f919b700 1 -- 172.21.15.111:0/1451770509 learned_addr learned my addr 172.21.15.111:0/1451770509 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:26.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.468+0000 7f70f919b700 1 -- 172.21.15.111:0/1451770509 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70f41601e0 msgr2=0x7f70f4160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:26.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.468+0000 7f70f919b700 1 --2- 172.21.15.111:0/1451770509 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70f41601e0 0x7f70f4160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.468+0000 7f70f919b700 1 -- 172.21.15.111:0/1451770509 >> v1:172.21.15.119:6789/0 conn(0x7f70f4160b70 legacy=0x7f70f40d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:26.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.468+0000 7f70f919b700 1 -- 172.21.15.111:0/1451770509 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f40cdc60 con 0x7f70f40b87a0 2023-03-29T10:58:26.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.468+0000 7f70f919b700 1 --2- 172.21.15.111:0/1451770509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70f40b87a0 0x7f70f40cfbf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f70f00098f0 tx=0x7f70f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85bb5e438990a4 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:26.470 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.468+0000 7f70ebfff700 1 -- 172.21.15.111:0/1451770509 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0004230 con 0x7f70f40b87a0 2023-03-29T10:58:26.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.468+0000 7f70ebfff700 1 -- 172.21.15.111:0/1451770509 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f0004390 con 0x7f70f40b87a0 2023-03-29T10:58:26.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.468+0000 7f70ebfff700 1 -- 172.21.15.111:0/1451770509 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f00036a0 con 0x7f70f40b87a0 2023-03-29T10:58:26.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.469+0000 7f70fb3ff700 1 -- 172.21.15.111:0/1451770509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70f40b87a0 msgr2=0x7f70f40cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:26.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.469+0000 7f70fb3ff700 1 --2- 172.21.15.111:0/1451770509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70f40b87a0 0x7f70f40cfbf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f70f00098f0 tx=0x7f70f0009c00 comp rx=0 tx=0).stop 2023-03-29T10:58:26.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.469+0000 7f70fb3ff700 1 -- 172.21.15.111:0/1451770509 shutdown_connections 2023-03-29T10:58:26.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.469+0000 7f70fb3ff700 1 --2- 172.21.15.111:0/1451770509 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70f40b87a0 0x7f70f40cfbf0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.469+0000 7f70fb3ff700 1 --2- 172.21.15.111:0/1451770509 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70f41601e0 0x7f70f4160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.469+0000 7f70fb3ff700 1 -- 172.21.15.111:0/1451770509 >> 172.21.15.111:0/1451770509 conn(0x7f70f4071b10 msgr2=0x7f70f4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:26.471 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.469+0000 7f70fb3ff700 1 -- 172.21.15.111:0/1451770509 shutdown_connections 2023-03-29T10:58:26.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.469+0000 7f70fb3ff700 1 -- 172.21.15.111:0/1451770509 wait complete. 2023-03-29T10:58:26.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.471+0000 7f70fb3ff700 1 Processor -- start 2023-03-29T10:58:26.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.471+0000 7f70fb3ff700 1 -- start start 2023-03-29T10:58:26.472 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.472+0000 7f70fb3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f40b87a0 0x7f70f415d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.473 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.472+0000 7f70fb3ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70f41601e0 0x7f70f40cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.473 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.472+0000 7f70fb3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70f4160b70 0x7f70f415ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.473 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.472+0000 7f70fb3ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f415ac70 con 0x7f70f41601e0 2023-03-29T10:58:26.473 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.472+0000 7f70fb3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f40c8410 con 0x7f70f40b87a0 2023-03-29T10:58:26.473 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.472+0000 7f70fb3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f415d370 con 0x7f70f4160b70 2023-03-29T10:58:26.474 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.472+0000 7f70f899a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70f41601e0 0x7f70f40cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.474 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.472+0000 7f70f899a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70f41601e0 0x7f70f40cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53068/0 (socket says 172.21.15.111:53068) 2023-03-29T10:58:26.474 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.472+0000 7f70f899a700 1 -- 172.21.15.111:0/3495497525 learned_addr learned my addr 172.21.15.111:0/3495497525 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:26.474 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.472+0000 7f70f919b700 1 --2- 172.21.15.111:0/3495497525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f40b87a0 0x7f70f415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.472+0000 7f70f899a700 1 -- 172.21.15.111:0/3495497525 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70f4160b70 msgr2=0x7f70f415ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:26.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.472+0000 7f70f999c700 1 --2- 172.21.15.111:0/3495497525 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70f4160b70 0x7f70f415ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.472+0000 7f70f899a700 1 --2- 172.21.15.111:0/3495497525 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70f4160b70 0x7f70f415ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.472+0000 7f70f899a700 1 -- 172.21.15.111:0/3495497525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f40b87a0 msgr2=0x7f70f415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:26.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.472+0000 7f70f899a700 1 --2- 172.21.15.111:0/3495497525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f40b87a0 0x7f70f415d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.472+0000 7f70f899a700 1 -- 172.21.15.111:0/3495497525 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f0009550 con 0x7f70f41601e0 2023-03-29T10:58:26.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.472+0000 7f70f919b700 1 --2- 172.21.15.111:0/3495497525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f40b87a0 0x7f70f415d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:26.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.472+0000 7f70f999c700 1 --2- 172.21.15.111:0/3495497525 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70f4160b70 0x7f70f415ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:26.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.473+0000 7f70f899a700 1 --2- 172.21.15.111:0/3495497525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70f41601e0 0x7f70f40cc5d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f70e400b560 tx=0x7f70e400b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:26.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.473+0000 7f70e9ffb700 1 -- 172.21.15.111:0/3495497525 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e4011830 con 0x7f70f41601e0 2023-03-29T10:58:26.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.473+0000 7f70e9ffb700 1 -- 172.21.15.111:0/3495497525 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e4011990 con 0x7f70f41601e0 2023-03-29T10:58:26.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.473+0000 7f70fb3ff700 1 -- 172.21.15.111:0/3495497525 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f40cd980 con 0x7f70f41601e0 2023-03-29T10:58:26.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.473+0000 7f70e9ffb700 1 -- 172.21.15.111:0/3495497525 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e4011d40 con 0x7f70f41601e0 2023-03-29T10:58:26.475 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.473+0000 7f70fb3ff700 1 -- 172.21.15.111:0/3495497525 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f40cd6a0 con 0x7f70f41601e0 2023-03-29T10:58:26.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.474+0000 7f70fb3ff700 1 -- 172.21.15.111:0/3495497525 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f40d2110 con 0x7f70f41601e0 2023-03-29T10:58:26.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.475+0000 7f70e9ffb700 1 -- 172.21.15.111:0/3495497525 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f70e401a400 con 0x7f70f41601e0 2023-03-29T10:58:26.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.476+0000 7f70e9ffb700 1 --2- 172.21.15.111:0/3495497525 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f70d8069de0 0x7f70d806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.476+0000 7f70e9ffb700 1 -- 172.21.15.111:0/3495497525 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f70e4088330 con 0x7f70f41601e0 2023-03-29T10:58:26.476 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.476+0000 7f70f919b700 1 --2- 172.21.15.111:0/3495497525 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f70d8069de0 0x7f70d806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.477 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.477+0000 7f70f919b700 1 --2- 172.21.15.111:0/3495497525 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f70d8069de0 0x7f70d806c290 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009c40 tx=0x7f70f0008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:26.480 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.480+0000 7f70e9ffb700 1 -- 172.21.15.111:0/3495497525 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f70e4053840 con 0x7f70f41601e0 2023-03-29T10:58:26.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.681+0000 7f70fb3ff700 1 -- 172.21.15.111:0/3495497525 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f70f40c8550 con 0x7f70f41601e0 2023-03-29T10:58:26.682 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.681+0000 7f70e9ffb700 1 -- 172.21.15.111:0/3495497525 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f70e4056e60 con 0x7f70f41601e0 2023-03-29T10:58:26.682 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:26.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.684+0000 7f70fb3ff700 1 -- 172.21.15.111:0/3495497525 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f70d8069de0 msgr2=0x7f70d806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:26.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.684+0000 7f70fb3ff700 1 --2- 172.21.15.111:0/3495497525 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f70d8069de0 0x7f70d806c290 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f70f0009c40 tx=0x7f70f0008030 comp rx=0 tx=0).stop 2023-03-29T10:58:26.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.684+0000 7f70fb3ff700 1 -- 172.21.15.111:0/3495497525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70f41601e0 msgr2=0x7f70f40cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:26.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.684+0000 7f70fb3ff700 1 --2- 172.21.15.111:0/3495497525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70f41601e0 0x7f70f40cc5d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f70e400b560 tx=0x7f70e400b920 comp rx=0 tx=0).stop 2023-03-29T10:58:26.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.684+0000 7f70fb3ff700 1 -- 172.21.15.111:0/3495497525 shutdown_connections 2023-03-29T10:58:26.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.684+0000 7f70fb3ff700 1 --2- 172.21.15.111:0/3495497525 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f70d8069de0 0x7f70d806c290 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.684+0000 7f70fb3ff700 1 --2- 172.21.15.111:0/3495497525 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f70f4160b70 0x7f70f415ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.684+0000 7f70fb3ff700 1 --2- 172.21.15.111:0/3495497525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70f40b87a0 0x7f70f415d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.684+0000 7f70fb3ff700 1 --2- 172.21.15.111:0/3495497525 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f70f41601e0 0x7f70f40cc5d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.684+0000 7f70fb3ff700 1 -- 172.21.15.111:0/3495497525 >> 172.21.15.111:0/3495497525 conn(0x7f70f4071b10 msgr2=0x7f70f40b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:26.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.684+0000 7f70fb3ff700 1 -- 172.21.15.111:0/3495497525 shutdown_connections 2023-03-29T10:58:26.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.684+0000 7f70fb3ff700 1 -- 172.21.15.111:0/3495497525 wait complete. 2023-03-29T10:58:26.701 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-29T10:58:26.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.886+0000 7f2f6e1ff700 1 Processor -- start 2023-03-29T10:58:26.887 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.886+0000 7f2f6e1ff700 1 -- start start 2023-03-29T10:58:26.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.888+0000 7f2f6e1ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2f6815c590 0x7f2f68160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.888+0000 7f2f6e1ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f680cf010 0x7f2f680cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.888 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.888+0000 7f2f6e1ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f680cd520 con 0x7f2f680cfd80 2023-03-29T10:58:26.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.888+0000 7f2f6e1ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f680d2640 con 0x7f2f6815c590 2023-03-29T10:58:26.889 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.888+0000 7f2f6e1ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f681610d0 con 0x7f2f680cf010 2023-03-29T10:58:26.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.888+0000 7f2f66ffd700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f680cf010 0x7f2f680cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.888+0000 7f2f67fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2f680cfd80 0x7f2f680d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.111:37150/0 (socket says 172.21.15.111:37150) 2023-03-29T10:58:26.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.888+0000 7f2f67fff700 1 -- 172.21.15.111:0/1786715319 learned_addr learned my addr 172.21.15.111:0/1786715319 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:26.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.888+0000 7f2f677fe700 1 --2- 172.21.15.111:0/1786715319 >> v2:172.21.15.119:3300/0 conn(0x7f2f6815c590 0x7f2f68160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.889+0000 7f2f667fc700 1 -- 172.21.15.111:0/1786715319 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1865592839 0 0) 0x7f2f680cd520 con 0x7f2f680cfd80 2023-03-29T10:58:26.890 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.889+0000 7f2f667fc700 1 -- 172.21.15.111:0/1786715319 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f500035b0 con 0x7f2f680cfd80 2023-03-29T10:58:26.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.889+0000 7f2f66ffd700 1 -- 172.21.15.111:0/1786715319 >> v2:172.21.15.119:3300/0 conn(0x7f2f6815c590 msgr2=0x7f2f68160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:26.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.889+0000 7f2f66ffd700 1 --2- 172.21.15.111:0/1786715319 >> v2:172.21.15.119:3300/0 conn(0x7f2f6815c590 0x7f2f68160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.889+0000 7f2f66ffd700 1 -- 172.21.15.111:0/1786715319 >> v1:172.21.15.119:6789/0 conn(0x7f2f680cfd80 legacy=0x7f2f680d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:26.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.889+0000 7f2f66ffd700 1 -- 172.21.15.111:0/1786715319 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f680d0200 con 0x7f2f680cf010 2023-03-29T10:58:26.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.889+0000 7f2f677fe700 1 --2- 172.21.15.111:0/1786715319 >> v2:172.21.15.119:3300/0 conn(0x7f2f6815c590 0x7f2f68160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:58:26.891 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.889+0000 7f2f66ffd700 1 --2- 172.21.15.111:0/1786715319 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f680cf010 0x7f2f680cf840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c009840 tx=0x7f2f5c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=135c1254b5ff2115 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:26.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.889+0000 7f2f667fc700 1 -- 172.21.15.111:0/1786715319 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c011830 con 0x7f2f680cf010 2023-03-29T10:58:26.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.889+0000 7f2f667fc700 1 -- 172.21.15.111:0/1786715319 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5c011990 con 0x7f2f680cf010 2023-03-29T10:58:26.892 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.889+0000 7f2f667fc700 1 -- 172.21.15.111:0/1786715319 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c011d20 con 0x7f2f680cf010 2023-03-29T10:58:26.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.890+0000 7f2f6e1ff700 1 -- 172.21.15.111:0/1786715319 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f680cf010 msgr2=0x7f2f680cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:26.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.890+0000 7f2f6e1ff700 1 --2- 172.21.15.111:0/1786715319 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f680cf010 0x7f2f680cf840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c009840 tx=0x7f2f5c009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:26.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.890+0000 7f2f6e1ff700 1 -- 172.21.15.111:0/1786715319 shutdown_connections 2023-03-29T10:58:26.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.890+0000 7f2f6e1ff700 1 --2- 172.21.15.111:0/1786715319 >> v2:172.21.15.119:3300/0 conn(0x7f2f6815c590 0x7f2f68160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.890+0000 7f2f6e1ff700 1 --2- 172.21.15.111:0/1786715319 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f680cf010 0x7f2f680cf840 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.890+0000 7f2f6e1ff700 1 -- 172.21.15.111:0/1786715319 >> 172.21.15.111:0/1786715319 conn(0x7f2f68071b10 msgr2=0x7f2f68071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:26.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.890+0000 7f2f6e1ff700 1 -- 172.21.15.111:0/1786715319 shutdown_connections 2023-03-29T10:58:26.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.890+0000 7f2f6e1ff700 1 -- 172.21.15.111:0/1786715319 wait complete. 2023-03-29T10:58:26.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.892+0000 7f2f6e1ff700 1 Processor -- start 2023-03-29T10:58:26.893 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.892+0000 7f2f6e1ff700 1 -- start start 2023-03-29T10:58:26.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.893+0000 7f2f6e1ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f680cf010 0x7f2f680c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.893+0000 7f2f6e1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f680cfd80 0x7f2f680c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.893+0000 7f2f6e1ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f6815c590 0x7f2f680b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.893+0000 7f2f6e1ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6814ec60 con 0x7f2f6815c590 2023-03-29T10:58:26.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.893+0000 7f2f6e1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f680b5d00 con 0x7f2f680cfd80 2023-03-29T10:58:26.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.893+0000 7f2f6e1ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f680b6d10 con 0x7f2f680cf010 2023-03-29T10:58:26.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.893+0000 7f2f67fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f6815c590 0x7f2f680b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.893+0000 7f2f67fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f6815c590 0x7f2f680b7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53082/0 (socket says 172.21.15.111:53082) 2023-03-29T10:58:26.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.893+0000 7f2f67fff700 1 -- 172.21.15.111:0/3710300596 learned_addr learned my addr 172.21.15.111:0/3710300596 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:26.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.893+0000 7f2f677fe700 1 --2- 172.21.15.111:0/3710300596 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f680cf010 0x7f2f680c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.893+0000 7f2f66ffd700 1 --2- 172.21.15.111:0/3710300596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f680cfd80 0x7f2f680c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.894+0000 7f2f677fe700 1 -- 172.21.15.111:0/3710300596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f680cfd80 msgr2=0x7f2f680c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:26.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.894+0000 7f2f677fe700 1 --2- 172.21.15.111:0/3710300596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f680cfd80 0x7f2f680c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.895 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.894+0000 7f2f677fe700 1 -- 172.21.15.111:0/3710300596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f6815c590 msgr2=0x7f2f680b7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:26.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.894+0000 7f2f677fe700 1 --2- 172.21.15.111:0/3710300596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f6815c590 0x7f2f680b7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:26.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.894+0000 7f2f677fe700 1 -- 172.21.15.111:0/3710300596 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f580094a0 con 0x7f2f680cf010 2023-03-29T10:58:26.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.894+0000 7f2f67fff700 1 --2- 172.21.15.111:0/3710300596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f6815c590 0x7f2f680b7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:58:26.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.894+0000 7f2f677fe700 1 --2- 172.21.15.111:0/3710300596 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f680cf010 0x7f2f680c8240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800e900 tx=0x7f2f5800ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:26.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.894+0000 7f2f4ffff700 1 -- 172.21.15.111:0/3710300596 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f58013680 con 0x7f2f680cf010 2023-03-29T10:58:26.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.894+0000 7f2f4ffff700 1 -- 172.21.15.111:0/3710300596 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f580137e0 con 0x7f2f680cf010 2023-03-29T10:58:26.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.894+0000 7f2f6e1ff700 1 -- 172.21.15.111:0/3710300596 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f5c0094a0 con 0x7f2f680cf010 2023-03-29T10:58:26.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.894+0000 7f2f4ffff700 1 -- 172.21.15.111:0/3710300596 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f58013b70 con 0x7f2f680cf010 2023-03-29T10:58:26.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.894+0000 7f2f6e1ff700 1 -- 172.21.15.111:0/3710300596 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f680cf560 con 0x7f2f680cf010 2023-03-29T10:58:26.896 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.895+0000 7f2f6e1ff700 1 -- 172.21.15.111:0/3710300596 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f680b23c0 con 0x7f2f680cf010 2023-03-29T10:58:26.897 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.896+0000 7f2f4ffff700 1 -- 172.21.15.111:0/3710300596 <== mon.2 v2:172.21.15.119:3301/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f2f58013cd0 con 0x7f2f680cf010 2023-03-29T10:58:26.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.897+0000 7f2f4ffff700 1 --2- 172.21.15.111:0/3710300596 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2f54069de0 0x7f2f5406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:26.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.897+0000 7f2f4ffff700 1 -- 172.21.15.111:0/3710300596 <== mon.2 v2:172.21.15.119:3301/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2f58088a20 con 0x7f2f680cf010 2023-03-29T10:58:26.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.897+0000 7f2f66ffd700 1 --2- 172.21.15.111:0/3710300596 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2f54069de0 0x7f2f5406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:26.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.898+0000 7f2f66ffd700 1 --2- 172.21.15.111:0/3710300596 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2f54069de0 0x7f2f5406c290 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c000c00 tx=0x7f2f5c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:26.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:26.901+0000 7f2f4ffff700 1 -- 172.21.15.111:0/3710300596 <== mon.2 v2:172.21.15.119:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f2f58053f20 con 0x7f2f680cf010 2023-03-29T10:58:27.094 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.094+0000 7f2f6e1ff700 1 -- 172.21.15.111:0/3710300596 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f680cd770 con 0x7f2f680cf010 2023-03-29T10:58:27.095 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.094+0000 7f2f4ffff700 1 -- 172.21.15.111:0/3710300596 <== mon.2 v2:172.21.15.119:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f58057540 con 0x7f2f680cf010 2023-03-29T10:58:27.095 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-03-29T10:58:27.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.097+0000 7f2f6e1ff700 1 -- 172.21.15.111:0/3710300596 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2f54069de0 msgr2=0x7f2f5406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:27.097 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.097+0000 7f2f6e1ff700 1 --2- 172.21.15.111:0/3710300596 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2f54069de0 0x7f2f5406c290 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c000c00 tx=0x7f2f5c008030 comp rx=0 tx=0).stop 2023-03-29T10:58:27.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.097+0000 7f2f6e1ff700 1 -- 172.21.15.111:0/3710300596 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f680cf010 msgr2=0x7f2f680c8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:27.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.097+0000 7f2f6e1ff700 1 --2- 172.21.15.111:0/3710300596 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f680cf010 0x7f2f680c8240 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800e900 tx=0x7f2f5800ec10 comp rx=0 tx=0).stop 2023-03-29T10:58:27.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.097+0000 7f2f6e1ff700 1 -- 172.21.15.111:0/3710300596 shutdown_connections 2023-03-29T10:58:27.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.097+0000 7f2f6e1ff700 1 --2- 172.21.15.111:0/3710300596 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2f54069de0 0x7f2f5406c290 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:27.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.097+0000 7f2f6e1ff700 1 --2- 172.21.15.111:0/3710300596 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f2f6815c590 0x7f2f680b7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:27.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.097+0000 7f2f6e1ff700 1 --2- 172.21.15.111:0/3710300596 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f2f680cf010 0x7f2f680c8240 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:27.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.097+0000 7f2f6e1ff700 1 --2- 172.21.15.111:0/3710300596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f680cfd80 0x7f2f680c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:27.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.097+0000 7f2f6e1ff700 1 -- 172.21.15.111:0/3710300596 >> 172.21.15.111:0/3710300596 conn(0x7f2f68071b10 msgr2=0x7f2f680b4760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:27.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.097+0000 7f2f6e1ff700 1 -- 172.21.15.111:0/3710300596 shutdown_connections 2023-03-29T10:58:27.098 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.097+0000 7f2f6e1ff700 1 -- 172.21.15.111:0/3710300596 wait complete. 2023-03-29T10:58:27.115 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-29T10:58:27.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.299+0000 7f56dc3ff700 1 Processor -- start 2023-03-29T10:58:27.300 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.299+0000 7f56dc3ff700 1 -- start start 2023-03-29T10:58:27.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.300+0000 7f56dc3ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f56d40cfbf0 0x7f56d41608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:27.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.300+0000 7f56dc3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d40d1100 0x7f56d40d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:27.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.300+0000 7f56dc3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d40d0020 con 0x7f56d40b87a0 2023-03-29T10:58:27.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.300+0000 7f56dc3ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d40d02d0 con 0x7f56d40cfbf0 2023-03-29T10:58:27.301 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.300+0000 7f56dc3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d40d0cb0 con 0x7f56d40d1100 2023-03-29T10:58:27.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.301+0000 7f56d999a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f56d40cfbf0 0x7f56d41608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:27.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.301+0000 7f56d999a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f56d40cfbf0 0x7f56d41608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53088/0 (socket says 172.21.15.111:53088) 2023-03-29T10:58:27.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.301+0000 7f56d999a700 1 -- 172.21.15.111:0/3873997594 learned_addr learned my addr 172.21.15.111:0/3873997594 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:27.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.301+0000 7f56da99c700 1 --2- 172.21.15.111:0/3873997594 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d40d1100 0x7f56d40d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:27.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.301+0000 7f56d999a700 1 -- 172.21.15.111:0/3873997594 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d40d1100 msgr2=0x7f56d40d1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:27.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.301+0000 7f56d999a700 1 --2- 172.21.15.111:0/3873997594 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d40d1100 0x7f56d40d1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:27.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.301+0000 7f56d999a700 1 -- 172.21.15.111:0/3873997594 >> v1:172.21.15.119:6789/0 conn(0x7f56d40b87a0 legacy=0x7f56d41601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:27.302 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.301+0000 7f56d999a700 1 -- 172.21.15.111:0/3873997594 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d40cdc60 con 0x7f56d40cfbf0 2023-03-29T10:58:27.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.301+0000 7f56d999a700 1 --2- 172.21.15.111:0/3873997594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56d40cfbf0 0x7f56d41608f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f56c40098f0 tx=0x7f56c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17a15a8c6628117 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:27.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.302+0000 7f56d9199700 1 -- 172.21.15.111:0/3873997594 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4004230 con 0x7f56d40cfbf0 2023-03-29T10:58:27.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.302+0000 7f56d9199700 1 -- 172.21.15.111:0/3873997594 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c4004390 con 0x7f56d40cfbf0 2023-03-29T10:58:27.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.302+0000 7f56d9199700 1 -- 172.21.15.111:0/3873997594 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c40036a0 con 0x7f56d40cfbf0 2023-03-29T10:58:27.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.302+0000 7f56dc3ff700 1 -- 172.21.15.111:0/3873997594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56d40cfbf0 msgr2=0x7f56d41608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:27.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.302+0000 7f56dc3ff700 1 --2- 172.21.15.111:0/3873997594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56d40cfbf0 0x7f56d41608f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f56c40098f0 tx=0x7f56c4009c00 comp rx=0 tx=0).stop 2023-03-29T10:58:27.303 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.302+0000 7f56dc3ff700 1 -- 172.21.15.111:0/3873997594 shutdown_connections 2023-03-29T10:58:27.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.302+0000 7f56dc3ff700 1 --2- 172.21.15.111:0/3873997594 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d40d1100 0x7f56d40d1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:27.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.302+0000 7f56dc3ff700 1 --2- 172.21.15.111:0/3873997594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56d40cfbf0 0x7f56d41608f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:27.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.302+0000 7f56dc3ff700 1 -- 172.21.15.111:0/3873997594 >> 172.21.15.111:0/3873997594 conn(0x7f56d4071b10 msgr2=0x7f56d4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:27.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.302+0000 7f56dc3ff700 1 -- 172.21.15.111:0/3873997594 shutdown_connections 2023-03-29T10:58:27.304 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.302+0000 7f56dc3ff700 1 -- 172.21.15.111:0/3873997594 wait complete. 2023-03-29T10:58:27.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.305+0000 7f56dc3ff700 1 Processor -- start 2023-03-29T10:58:27.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.305+0000 7f56dc3ff700 1 -- start start 2023-03-29T10:58:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.306+0000 7f56dc3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d40b87a0 0x7f56d414d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.306+0000 7f56dc3ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56d40cfbf0 0x7f56d40c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.306+0000 7f56dc3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d40d1100 0x7f56d415f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.306+0000 7f56dc3ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d414d8a0 con 0x7f56d40cfbf0 2023-03-29T10:58:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.306+0000 7f56dc3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d415e410 con 0x7f56d40b87a0 2023-03-29T10:58:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.306+0000 7f56dc3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d415e6f0 con 0x7f56d40d1100 2023-03-29T10:58:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.306+0000 7f56d999a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56d40cfbf0 0x7f56d40c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.306+0000 7f56d999a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56d40cfbf0 0x7f56d40c96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53096/0 (socket says 172.21.15.111:53096) 2023-03-29T10:58:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.306+0000 7f56d999a700 1 -- 172.21.15.111:0/4000920280 learned_addr learned my addr 172.21.15.111:0/4000920280 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.306+0000 7f56da19b700 1 --2- 172.21.15.111:0/4000920280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d40b87a0 0x7f56d414d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:27.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.306+0000 7f56da99c700 1 --2- 172.21.15.111:0/4000920280 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d40d1100 0x7f56d415f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:27.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.307+0000 7f56d999a700 1 -- 172.21.15.111:0/4000920280 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d40d1100 msgr2=0x7f56d415f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:27.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.307+0000 7f56d999a700 1 --2- 172.21.15.111:0/4000920280 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d40d1100 0x7f56d415f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:27.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.307+0000 7f56d999a700 1 -- 172.21.15.111:0/4000920280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d40b87a0 msgr2=0x7f56d414d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:27.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.307+0000 7f56d999a700 1 --2- 172.21.15.111:0/4000920280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d40b87a0 0x7f56d414d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:27.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.307+0000 7f56d999a700 1 -- 172.21.15.111:0/4000920280 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c4009550 con 0x7f56d40cfbf0 2023-03-29T10:58:27.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.307+0000 7f56da99c700 1 --2- 172.21.15.111:0/4000920280 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d40d1100 0x7f56d415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:27.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.307+0000 7f56d999a700 1 --2- 172.21.15.111:0/4000920280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56d40cfbf0 0x7f56d40c96a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009130 tx=0x7f56c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:27.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.307+0000 7f56caffd700 1 -- 172.21.15.111:0/4000920280 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c4003b50 con 0x7f56d40cfbf0 2023-03-29T10:58:27.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.307+0000 7f56caffd700 1 -- 172.21.15.111:0/4000920280 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c401f070 con 0x7f56d40cfbf0 2023-03-29T10:58:27.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.307+0000 7f56dc3ff700 1 -- 172.21.15.111:0/4000920280 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d40cd980 con 0x7f56d40cfbf0 2023-03-29T10:58:27.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.307+0000 7f56caffd700 1 -- 172.21.15.111:0/4000920280 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c400b660 con 0x7f56d40cfbf0 2023-03-29T10:58:27.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.307+0000 7f56dc3ff700 1 -- 172.21.15.111:0/4000920280 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d40cd6a0 con 0x7f56d40cfbf0 2023-03-29T10:58:27.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.308+0000 7f56dc3ff700 1 -- 172.21.15.111:0/4000920280 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d4160e30 con 0x7f56d40cfbf0 2023-03-29T10:58:27.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.309+0000 7f56caffd700 1 -- 172.21.15.111:0/4000920280 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f56c4021030 con 0x7f56d40cfbf0 2023-03-29T10:58:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.310+0000 7f56caffd700 1 --2- 172.21.15.111:0/4000920280 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f56b8069de0 0x7f56b806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.310+0000 7f56caffd700 1 -- 172.21.15.111:0/4000920280 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f56c40898f0 con 0x7f56d40cfbf0 2023-03-29T10:58:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.310+0000 7f56da19b700 1 --2- 172.21.15.111:0/4000920280 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f56b8069de0 0x7f56b806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:27.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.311+0000 7f56da19b700 1 --2- 172.21.15.111:0/4000920280 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f56b8069de0 0x7f56b806c290 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f56d0007fd0 tx=0x7f56d0007ec0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:27.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.315+0000 7f56caffd700 1 -- 172.21.15.111:0/4000920280 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f56c4054e00 con 0x7f56d40cfbf0 2023-03-29T10:58:27.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.505+0000 7f56dc3ff700 1 -- 172.21.15.111:0/4000920280 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f56d4160e30 con 0x7f56d40cfbf0 2023-03-29T10:58:27.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.506+0000 7f56caffd700 1 -- 172.21.15.111:0/4000920280 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f56c4058420 con 0x7f56d40cfbf0 2023-03-29T10:58:27.507 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:27.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.508+0000 7f56dc3ff700 1 -- 172.21.15.111:0/4000920280 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f56b8069de0 msgr2=0x7f56b806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:27.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.508+0000 7f56dc3ff700 1 --2- 172.21.15.111:0/4000920280 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f56b8069de0 0x7f56b806c290 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f56d0007fd0 tx=0x7f56d0007ec0 comp rx=0 tx=0).stop 2023-03-29T10:58:27.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.508+0000 7f56dc3ff700 1 -- 172.21.15.111:0/4000920280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56d40cfbf0 msgr2=0x7f56d40c96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:27.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.508+0000 7f56dc3ff700 1 --2- 172.21.15.111:0/4000920280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56d40cfbf0 0x7f56d40c96a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f56c4009130 tx=0x7f56c4004750 comp rx=0 tx=0).stop 2023-03-29T10:58:27.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.508+0000 7f56dc3ff700 1 -- 172.21.15.111:0/4000920280 shutdown_connections 2023-03-29T10:58:27.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.508+0000 7f56dc3ff700 1 --2- 172.21.15.111:0/4000920280 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f56b8069de0 0x7f56b806c290 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:27.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.508+0000 7f56dc3ff700 1 --2- 172.21.15.111:0/4000920280 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f56d40d1100 0x7f56d415f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:27.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.508+0000 7f56dc3ff700 1 --2- 172.21.15.111:0/4000920280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56d40b87a0 0x7f56d414d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:27.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.508+0000 7f56dc3ff700 1 --2- 172.21.15.111:0/4000920280 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f56d40cfbf0 0x7f56d40c96a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:27.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.508+0000 7f56dc3ff700 1 -- 172.21.15.111:0/4000920280 >> 172.21.15.111:0/4000920280 conn(0x7f56d4071b10 msgr2=0x7f56d40d3670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:27.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.508+0000 7f56dc3ff700 1 -- 172.21.15.111:0/4000920280 shutdown_connections 2023-03-29T10:58:27.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.508+0000 7f56dc3ff700 1 -- 172.21.15.111:0/4000920280 wait complete. 2023-03-29T10:58:27.538 INFO:tasks.cephfs_test_runner:Starting test: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2023-03-29T10:58:27.539 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2023-03-29T10:58:27.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.721+0000 7f13e65ff700 1 Processor -- start 2023-03-29T10:58:27.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.721+0000 7f13e65ff700 1 -- start start 2023-03-29T10:58:27.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.722+0000 7f13e65ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13e00b2440 0x7f13e00cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:27.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.722+0000 7f13e65ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f13e00d2ea0 0x7f13e00d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:27.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.722+0000 7f13e65ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e00d3860 con 0x7f13e015db70 2023-03-29T10:58:27.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.722+0000 7f13e65ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e00b2810 con 0x7f13e00d2ea0 2023-03-29T10:58:27.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.722+0000 7f13e65ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e015ce30 con 0x7f13e00b2440 2023-03-29T10:58:27.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.723+0000 7f13dffff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13e00b2440 0x7f13e00cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:27.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.723+0000 7f13e4b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f13e00d2ea0 0x7f13e00d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:27.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.723+0000 7f13dffff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13e00b2440 0x7f13e00cd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:48442/0 (socket says 172.21.15.111:48442) 2023-03-29T10:58:27.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.723+0000 7f13dffff700 1 -- 172.21.15.111:0/400424851 learned_addr learned my addr 172.21.15.111:0/400424851 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:27.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.723+0000 7f13df7fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f13e015db70 0x7f13e0160e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.111:37178/0 (socket says 172.21.15.111:37178) 2023-03-29T10:58:27.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.723+0000 7f13e4b9c700 1 -- 172.21.15.111:0/400424851 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13e00b2440 msgr2=0x7f13e00cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:27.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.723+0000 7f13e4b9c700 1 --2- 172.21.15.111:0/400424851 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13e00b2440 0x7f13e00cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:27.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.723+0000 7f13e4b9c700 1 -- 172.21.15.111:0/400424851 >> v1:172.21.15.119:6789/0 conn(0x7f13e015db70 legacy=0x7f13e0160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:27.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.723+0000 7f13e4b9c700 1 -- 172.21.15.111:0/400424851 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e00cdee0 con 0x7f13e00d2ea0 2023-03-29T10:58:27.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.723+0000 7f13dffff700 1 --2- 172.21.15.111:0/400424851 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13e00b2440 0x7f13e00cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:58:27.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.723+0000 7f13e4b9c700 1 --2- 172.21.15.111:0/400424851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13e00d2ea0 0x7f13e00d3320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009d90 tx=0x7f13d800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a25f121795d3b40 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:27.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.724+0000 7f13deffd700 1 -- 172.21.15.111:0/400424851 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d800e070 con 0x7f13e00d2ea0 2023-03-29T10:58:27.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.724+0000 7f13deffd700 1 -- 172.21.15.111:0/400424851 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d800bb20 con 0x7f13e00d2ea0 2023-03-29T10:58:27.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.724+0000 7f13deffd700 1 -- 172.21.15.111:0/400424851 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d8012430 con 0x7f13e00d2ea0 2023-03-29T10:58:27.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.724+0000 7f13e65ff700 1 -- 172.21.15.111:0/400424851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13e00d2ea0 msgr2=0x7f13e00d3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:27.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.724+0000 7f13e65ff700 1 --2- 172.21.15.111:0/400424851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13e00d2ea0 0x7f13e00d3320 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009d90 tx=0x7f13d800b330 comp rx=0 tx=0).stop 2023-03-29T10:58:27.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.724+0000 7f13e65ff700 1 -- 172.21.15.111:0/400424851 shutdown_connections 2023-03-29T10:58:27.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.724+0000 7f13e65ff700 1 --2- 172.21.15.111:0/400424851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13e00d2ea0 0x7f13e00d3320 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:27.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.724+0000 7f13e65ff700 1 --2- 172.21.15.111:0/400424851 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13e00b2440 0x7f13e00cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:27.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.724+0000 7f13e65ff700 1 -- 172.21.15.111:0/400424851 >> 172.21.15.111:0/400424851 conn(0x7f13e0071b10 msgr2=0x7f13e0071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:27.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.724+0000 7f13e65ff700 1 -- 172.21.15.111:0/400424851 shutdown_connections 2023-03-29T10:58:27.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.724+0000 7f13e65ff700 1 -- 172.21.15.111:0/400424851 wait complete. 2023-03-29T10:58:27.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.726+0000 7f13e65ff700 1 Processor -- start 2023-03-29T10:58:27.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.726+0000 7f13e65ff700 1 -- start start 2023-03-29T10:58:27.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.727+0000 7f13e65ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13e00b2440 0x7f13e01f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:27.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.727+0000 7f13e65ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13e00d2ea0 0x7f13e01f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:27.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.727+0000 7f13e65ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13e015db70 0x7f13e00aa250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:27.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.727+0000 7f13e65ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e00caf40 con 0x7f13e00b2440 2023-03-29T10:58:27.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.727+0000 7f13e65ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e01f59f0 con 0x7f13e015db70 2023-03-29T10:58:27.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.727+0000 7f13e65ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e00cb190 con 0x7f13e00d2ea0 2023-03-29T10:58:27.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.727+0000 7f13dffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13e00b2440 0x7f13e01f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:27.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.727+0000 7f13dffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13e00b2440 0x7f13e01f26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:53104/0 (socket says 172.21.15.111:53104) 2023-03-29T10:58:27.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.727+0000 7f13dffff700 1 -- 172.21.15.111:0/2765824093 learned_addr learned my addr 172.21.15.111:0/2765824093 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:27.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.727+0000 7f13df7fe700 1 --2- 172.21.15.111:0/2765824093 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13e00d2ea0 0x7f13e01f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:27.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.727+0000 7f13e4b9c700 1 --2- 172.21.15.111:0/2765824093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13e015db70 0x7f13e00aa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:27.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.728+0000 7f13dffff700 1 -- 172.21.15.111:0/2765824093 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13e00d2ea0 msgr2=0x7f13e01f54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:27.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.728+0000 7f13dffff700 1 --2- 172.21.15.111:0/2765824093 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13e00d2ea0 0x7f13e01f54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:27.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.728+0000 7f13dffff700 1 -- 172.21.15.111:0/2765824093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13e015db70 msgr2=0x7f13e00aa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:27.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.728+0000 7f13dffff700 1 --2- 172.21.15.111:0/2765824093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13e015db70 0x7f13e00aa250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:27.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.728+0000 7f13dffff700 1 -- 172.21.15.111:0/2765824093 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d0009550 con 0x7f13e00b2440 2023-03-29T10:58:27.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.728+0000 7f13dffff700 1 --2- 172.21.15.111:0/2765824093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13e00b2440 0x7f13e01f26d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f13d000ebe0 tx=0x7f13d000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:27.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.728+0000 7f13dcff9700 1 -- 172.21.15.111:0/2765824093 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d000c850 con 0x7f13e00b2440 2023-03-29T10:58:27.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.728+0000 7f13dcff9700 1 -- 172.21.15.111:0/2765824093 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d000c9b0 con 0x7f13e00b2440 2023-03-29T10:58:27.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.728+0000 7f13e65ff700 1 -- 172.21.15.111:0/2765824093 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d8009a70 con 0x7f13e00b2440 2023-03-29T10:58:27.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.728+0000 7f13dcff9700 1 -- 172.21.15.111:0/2765824093 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d000cd40 con 0x7f13e00b2440 2023-03-29T10:58:27.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.728+0000 7f13e65ff700 1 -- 172.21.15.111:0/2765824093 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e015d090 con 0x7f13e00b2440 2023-03-29T10:58:27.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.729+0000 7f13e65ff700 1 -- 172.21.15.111:0/2765824093 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e00cdc00 con 0x7f13e00b2440 2023-03-29T10:58:27.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.730+0000 7f13dcff9700 1 -- 172.21.15.111:0/2765824093 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f13d0013470 con 0x7f13e00b2440 2023-03-29T10:58:27.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.731+0000 7f13dcff9700 1 --2- 172.21.15.111:0/2765824093 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f13cc069de0 0x7f13cc06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:27.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.731+0000 7f13dcff9700 1 -- 172.21.15.111:0/2765824093 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f13d0088480 con 0x7f13e00b2440 2023-03-29T10:58:27.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.731+0000 7f13df7fe700 1 --2- 172.21.15.111:0/2765824093 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f13cc069de0 0x7f13cc06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:27.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.732+0000 7f13df7fe700 1 --2- 172.21.15.111:0/2765824093 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f13cc069de0 0x7f13cc06c290 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f13e00d32e0 tx=0x7f13d4007ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:27.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:27.735+0000 7f13dcff9700 1 -- 172.21.15.111:0/2765824093 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f13d0053960 con 0x7f13e00b2440 2023-03-29T10:58:28.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.010+0000 7f13e65ff700 1 -- 172.21.15.111:0/2765824093 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) v1 -- 0x7f13e015d370 con 0x7f13e00b2440 2023-03-29T10:58:28.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.560+0000 7f13dcff9700 1 -- 172.21.15.111:0/2765824093 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v79) v1 ==== 147+0+0 (secure 0 0 0) 0x7f13d0056f80 con 0x7f13e00b2440 2023-03-29T10:58:28.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.563+0000 7f13e65ff700 1 -- 172.21.15.111:0/2765824093 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f13cc069de0 msgr2=0x7f13cc06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:28.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.563+0000 7f13e65ff700 1 --2- 172.21.15.111:0/2765824093 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f13cc069de0 0x7f13cc06c290 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f13e00d32e0 tx=0x7f13d4007ed0 comp rx=0 tx=0).stop 2023-03-29T10:58:28.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.563+0000 7f13e65ff700 1 -- 172.21.15.111:0/2765824093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13e00b2440 msgr2=0x7f13e01f26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:28.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.563+0000 7f13e65ff700 1 --2- 172.21.15.111:0/2765824093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13e00b2440 0x7f13e01f26d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f13d000ebe0 tx=0x7f13d000eef0 comp rx=0 tx=0).stop 2023-03-29T10:58:28.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.563+0000 7f13e65ff700 1 -- 172.21.15.111:0/2765824093 shutdown_connections 2023-03-29T10:58:28.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.563+0000 7f13e65ff700 1 --2- 172.21.15.111:0/2765824093 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f13cc069de0 0x7f13cc06c290 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:28.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.563+0000 7f13e65ff700 1 --2- 172.21.15.111:0/2765824093 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f13e00b2440 0x7f13e01f26d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:28.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.563+0000 7f13e65ff700 1 --2- 172.21.15.111:0/2765824093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13e015db70 0x7f13e00aa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:28.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.563+0000 7f13e65ff700 1 --2- 172.21.15.111:0/2765824093 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f13e00d2ea0 0x7f13e01f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:28.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.563+0000 7f13e65ff700 1 -- 172.21.15.111:0/2765824093 >> 172.21.15.111:0/2765824093 conn(0x7f13e0071b10 msgr2=0x7f13e0071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:28.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.563+0000 7f13e65ff700 1 -- 172.21.15.111:0/2765824093 shutdown_connections 2023-03-29T10:58:28.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.563+0000 7f13e65ff700 1 -- 172.21.15.111:0/2765824093 wait complete. 2023-03-29T10:58:28.584 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-03-29T10:58:28.774 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.773+0000 7f86e99ff700 1 Processor -- start 2023-03-29T10:58:28.774 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.773+0000 7f86e99ff700 1 -- start start 2023-03-29T10:58:28.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.774+0000 7f86e99ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f86e415a140 0x7f86e40cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:28.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.774+0000 7f86e99ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f86e4160ba0 0x7f86e40cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:28.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.774+0000 7f86e99ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86e40cfe20 0x7f86e40b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:28.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.774+0000 7f86e99ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e40d02a0 con 0x7f86e415a140 2023-03-29T10:58:28.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.774+0000 7f86e99ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e415d660 con 0x7f86e4160ba0 2023-03-29T10:58:28.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.774+0000 7f86e99ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e415cd70 con 0x7f86e40cfe20 2023-03-29T10:58:28.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.774+0000 7f86e2ffd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f86e415a140 0x7f86e40cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:28.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.775+0000 7f86e2ffd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f86e415a140 0x7f86e40cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44668/0 (socket says 172.21.15.111:44668) 2023-03-29T10:58:28.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.775+0000 7f86e2ffd700 1 -- 172.21.15.111:0/3725351131 learned_addr learned my addr 172.21.15.111:0/3725351131 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:28.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.775+0000 7f86e37fe700 1 --2- 172.21.15.111:0/3725351131 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86e40cfe20 0x7f86e40b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:28.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.775+0000 7f86e27fc700 1 --2- 172.21.15.111:0/3725351131 >> v2:172.21.15.119:3300/0 conn(0x7f86e4160ba0 0x7f86e40cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:28.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.775+0000 7f86e2ffd700 1 -- 172.21.15.111:0/3725351131 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86e40cfe20 msgr2=0x7f86e40b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:28.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.775+0000 7f86e2ffd700 1 --2- 172.21.15.111:0/3725351131 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86e40cfe20 0x7f86e40b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:28.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.775+0000 7f86e2ffd700 1 -- 172.21.15.111:0/3725351131 >> v2:172.21.15.119:3300/0 conn(0x7f86e4160ba0 msgr2=0x7f86e40cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:28.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.775+0000 7f86e2ffd700 1 --2- 172.21.15.111:0/3725351131 >> v2:172.21.15.119:3300/0 conn(0x7f86e4160ba0 0x7f86e40cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:28.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.775+0000 7f86e2ffd700 1 -- 172.21.15.111:0/3725351131 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e415d010 con 0x7f86e415a140 2023-03-29T10:58:28.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.775+0000 7f86e2ffd700 1 --2- 172.21.15.111:0/3725351131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e415a140 0x7f86e40cd520 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f86e40d07b0 tx=0x7f86d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51be8f6824790682 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:28.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.775+0000 7f86e1ffb700 1 -- 172.21.15.111:0/3725351131 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d400b870 con 0x7f86e415a140 2023-03-29T10:58:28.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.775+0000 7f86e1ffb700 1 -- 172.21.15.111:0/3725351131 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d400b9d0 con 0x7f86e415a140 2023-03-29T10:58:28.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.775+0000 7f86e1ffb700 1 -- 172.21.15.111:0/3725351131 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d400bd60 con 0x7f86e415a140 2023-03-29T10:58:28.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.776+0000 7f86e99ff700 1 -- 172.21.15.111:0/3725351131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e415a140 msgr2=0x7f86e40cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:28.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.776+0000 7f86e99ff700 1 --2- 172.21.15.111:0/3725351131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e415a140 0x7f86e40cd520 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f86e40d07b0 tx=0x7f86d4009bd0 comp rx=0 tx=0).stop 2023-03-29T10:58:28.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.776+0000 7f86e99ff700 1 -- 172.21.15.111:0/3725351131 shutdown_connections 2023-03-29T10:58:28.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.776+0000 7f86e99ff700 1 --2- 172.21.15.111:0/3725351131 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86e40cfe20 0x7f86e40b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:28.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.776+0000 7f86e99ff700 1 --2- 172.21.15.111:0/3725351131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e415a140 0x7f86e40cd520 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:28.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.776+0000 7f86e99ff700 1 --2- 172.21.15.111:0/3725351131 >> v2:172.21.15.119:3300/0 conn(0x7f86e4160ba0 0x7f86e40cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:28.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.776+0000 7f86e99ff700 1 -- 172.21.15.111:0/3725351131 >> 172.21.15.111:0/3725351131 conn(0x7f86e4071b10 msgr2=0x7f86e4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:28.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.776+0000 7f86e99ff700 1 -- 172.21.15.111:0/3725351131 shutdown_connections 2023-03-29T10:58:28.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.776+0000 7f86e99ff700 1 -- 172.21.15.111:0/3725351131 wait complete. 2023-03-29T10:58:28.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.778+0000 7f86e99ff700 1 Processor -- start 2023-03-29T10:58:28.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.778+0000 7f86e99ff700 1 -- start start 2023-03-29T10:58:28.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.779+0000 7f86e99ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86e40cfe20 0x7f86e41491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:28.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.779+0000 7f86e99ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e415a140 0x7f86e40b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:28.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.779+0000 7f86e99ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86e4160ba0 0x7f86e41e5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:28.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.779+0000 7f86e99ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e4147de0 con 0x7f86e415a140 2023-03-29T10:58:28.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.779+0000 7f86e99ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e41471d0 con 0x7f86e4160ba0 2023-03-29T10:58:28.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.779+0000 7f86e99ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e41474b0 con 0x7f86e40cfe20 2023-03-29T10:58:28.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.779+0000 7f86e27fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e415a140 0x7f86e40b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:28.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.779+0000 7f86e27fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e415a140 0x7f86e40b6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44670/0 (socket says 172.21.15.111:44670) 2023-03-29T10:58:28.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.779+0000 7f86e27fc700 1 -- 172.21.15.111:0/204557499 learned_addr learned my addr 172.21.15.111:0/204557499 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:28.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.779+0000 7f86e37fe700 1 --2- 172.21.15.111:0/204557499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86e4160ba0 0x7f86e41e5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:28.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.779+0000 7f86e2ffd700 1 --2- 172.21.15.111:0/204557499 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86e40cfe20 0x7f86e41491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:28.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.780+0000 7f86e27fc700 1 -- 172.21.15.111:0/204557499 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86e40cfe20 msgr2=0x7f86e41491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:28.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.780+0000 7f86e27fc700 1 --2- 172.21.15.111:0/204557499 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86e40cfe20 0x7f86e41491a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:28.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.780+0000 7f86e27fc700 1 -- 172.21.15.111:0/204557499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86e4160ba0 msgr2=0x7f86e41e5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:28.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.780+0000 7f86e27fc700 1 --2- 172.21.15.111:0/204557499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86e4160ba0 0x7f86e41e5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:28.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.780+0000 7f86e27fc700 1 -- 172.21.15.111:0/204557499 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d4009550 con 0x7f86e415a140 2023-03-29T10:58:28.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.780+0000 7f86e37fe700 1 --2- 172.21.15.111:0/204557499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86e4160ba0 0x7f86e41e5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:28.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.780+0000 7f86e27fc700 1 --2- 172.21.15.111:0/204557499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e415a140 0x7f86e40b6690 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f86d800e810 tx=0x7f86d800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:28.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.780+0000 7f86cbfff700 1 -- 172.21.15.111:0/204557499 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8013610 con 0x7f86e415a140 2023-03-29T10:58:28.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.780+0000 7f86cbfff700 1 -- 172.21.15.111:0/204557499 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d8013770 con 0x7f86e415a140 2023-03-29T10:58:28.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.780+0000 7f86cbfff700 1 -- 172.21.15.111:0/204557499 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8013b20 con 0x7f86e415a140 2023-03-29T10:58:28.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.780+0000 7f86e99ff700 1 -- 172.21.15.111:0/204557499 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e415daf0 con 0x7f86e415a140 2023-03-29T10:58:28.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.780+0000 7f86e99ff700 1 -- 172.21.15.111:0/204557499 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e415d2f0 con 0x7f86e415a140 2023-03-29T10:58:28.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.781+0000 7f86e99ff700 1 -- 172.21.15.111:0/204557499 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e40cdb00 con 0x7f86e415a140 2023-03-29T10:58:28.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.782+0000 7f86cbfff700 1 -- 172.21.15.111:0/204557499 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f86d8013c80 con 0x7f86e415a140 2023-03-29T10:58:28.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.783+0000 7f86cbfff700 1 --2- 172.21.15.111:0/204557499 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f86d0069de0 0x7f86d006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:28.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.783+0000 7f86cbfff700 1 -- 172.21.15.111:0/204557499 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f86d8088060 con 0x7f86e415a140 2023-03-29T10:58:28.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.783+0000 7f86e2ffd700 1 --2- 172.21.15.111:0/204557499 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f86d0069de0 0x7f86d006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:28.784 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.784+0000 7f86e2ffd700 1 --2- 172.21.15.111:0/204557499 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f86d0069de0 0x7f86d006c290 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f86d4000c00 tx=0x7f86d4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:28.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.787+0000 7f86cbfff700 1 -- 172.21.15.111:0/204557499 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f86d8056f80 con 0x7f86e415a140 2023-03-29T10:58:28.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.980+0000 7f86e99ff700 1 -- 172.21.15.111:0/204557499 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f86e4147f20 con 0x7f86e415a140 2023-03-29T10:58:28.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.986+0000 7f86cbfff700 1 -- 172.21.15.111:0/204557499 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f86d805c0c0 con 0x7f86e415a140 2023-03-29T10:58:28.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.988+0000 7f86e99ff700 1 -- 172.21.15.111:0/204557499 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f86d0069de0 msgr2=0x7f86d006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:28.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.988+0000 7f86e99ff700 1 --2- 172.21.15.111:0/204557499 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f86d0069de0 0x7f86d006c290 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f86d4000c00 tx=0x7f86d4008030 comp rx=0 tx=0).stop 2023-03-29T10:58:28.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.988+0000 7f86e99ff700 1 -- 172.21.15.111:0/204557499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e415a140 msgr2=0x7f86e40b6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:28.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.988+0000 7f86e99ff700 1 --2- 172.21.15.111:0/204557499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e415a140 0x7f86e40b6690 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f86d800e810 tx=0x7f86d800eb20 comp rx=0 tx=0).stop 2023-03-29T10:58:28.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.988+0000 7f86e99ff700 1 -- 172.21.15.111:0/204557499 shutdown_connections 2023-03-29T10:58:28.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.988+0000 7f86e99ff700 1 --2- 172.21.15.111:0/204557499 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f86d0069de0 0x7f86d006c290 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:28.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.988+0000 7f86e99ff700 1 --2- 172.21.15.111:0/204557499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86e4160ba0 0x7f86e41e5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:28.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.988+0000 7f86e99ff700 1 --2- 172.21.15.111:0/204557499 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f86e40cfe20 0x7f86e41491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:28.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.988+0000 7f86e99ff700 1 --2- 172.21.15.111:0/204557499 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f86e415a140 0x7f86e40b6690 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:28.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.988+0000 7f86e99ff700 1 -- 172.21.15.111:0/204557499 >> 172.21.15.111:0/204557499 conn(0x7f86e4071b10 msgr2=0x7f86e41593f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:28.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.988+0000 7f86e99ff700 1 -- 172.21.15.111:0/204557499 shutdown_connections 2023-03-29T10:58:28.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:28.988+0000 7f86e99ff700 1 -- 172.21.15.111:0/204557499 wait complete. 2023-03-29T10:58:29.005 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-03-29T10:58:29.005 DEBUG:teuthology.orchestra.run.smithi111:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-03-29T10:58:29.100 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-03-29T10:58:29.212 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-03-29T10:58:29.213 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:58:29.213 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-29T10:58:29.254 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:58:29.254 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-29T10:58:29.334 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:58:29.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.558+0000 7f06ad9ff700 1 Processor -- start 2023-03-29T10:58:29.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.558+0000 7f06ad9ff700 1 -- start start 2023-03-29T10:58:29.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.559+0000 7f06ad9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f06a806c300 0x7f06a80d00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:29.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.559+0000 7f06ad9ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f06a8160f40 0x7f06a80d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:29.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.559+0000 7f06ad9ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a80d26f0 con 0x7f06a80b2430 2023-03-29T10:58:29.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.559+0000 7f06ad9ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a80cd750 con 0x7f06a8160f40 2023-03-29T10:58:29.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.559+0000 7f06ad9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a80cda30 con 0x7f06a806c300 2023-03-29T10:58:29.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.559+0000 7f06a6ffd700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f06a80b2430 0x7f06a806bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43510/0 (socket says 172.21.15.111:43510) 2023-03-29T10:58:29.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.559+0000 7f06a6ffd700 1 -- 172.21.15.111:0/2939244282 learned_addr learned my addr 172.21.15.111:0/2939244282 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:29.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.559+0000 7f06a77fe700 1 --2- 172.21.15.111:0/2939244282 >> v2:172.21.15.111:3300/0 conn(0x7f06a8160f40 0x7f06a80d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:29.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.559+0000 7f06a67fc700 1 --2- 172.21.15.111:0/2939244282 >> v2:172.21.15.119:3300/0 conn(0x7f06a806c300 0x7f06a80d00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:29.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.560+0000 7f06a77fe700 1 -- 172.21.15.111:0/2939244282 >> v2:172.21.15.119:3300/0 conn(0x7f06a806c300 msgr2=0x7f06a80d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:29.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.560+0000 7f06a77fe700 1 --2- 172.21.15.111:0/2939244282 >> v2:172.21.15.119:3300/0 conn(0x7f06a806c300 0x7f06a80d00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:29.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.560+0000 7f06a77fe700 1 -- 172.21.15.111:0/2939244282 >> v1:172.21.15.111:6789/0 conn(0x7f06a80b2430 legacy=0x7f06a806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:29.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.560+0000 7f06a77fe700 1 -- 172.21.15.111:0/2939244282 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a80d2940 con 0x7f06a8160f40 2023-03-29T10:58:29.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.560+0000 7f06a67fc700 1 --2- 172.21.15.111:0/2939244282 >> v2:172.21.15.119:3300/0 conn(0x7f06a806c300 0x7f06a80d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:58:29.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.560+0000 7f06a77fe700 1 --2- 172.21.15.111:0/2939244282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06a8160f40 0x7f06a80d0d70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f06a80b4b70 tx=0x7f06a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaba86354c9aa6a2 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:29.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.561+0000 7f06a5ffb700 1 -- 172.21.15.111:0/2939244282 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a000e070 con 0x7f06a8160f40 2023-03-29T10:58:29.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.561+0000 7f06a5ffb700 1 -- 172.21.15.111:0/2939244282 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a000bb20 con 0x7f06a8160f40 2023-03-29T10:58:29.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.561+0000 7f06a5ffb700 1 -- 172.21.15.111:0/2939244282 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0012430 con 0x7f06a8160f40 2023-03-29T10:58:29.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.562+0000 7f06ad9ff700 1 -- 172.21.15.111:0/2939244282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06a8160f40 msgr2=0x7f06a80d0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:29.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.562+0000 7f06ad9ff700 1 --2- 172.21.15.111:0/2939244282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06a8160f40 0x7f06a80d0d70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f06a80b4b70 tx=0x7f06a000b330 comp rx=0 tx=0).stop 2023-03-29T10:58:29.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.562+0000 7f06ad9ff700 1 -- 172.21.15.111:0/2939244282 shutdown_connections 2023-03-29T10:58:29.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.562+0000 7f06ad9ff700 1 --2- 172.21.15.111:0/2939244282 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06a8160f40 0x7f06a80d0d70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:29.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.562+0000 7f06ad9ff700 1 --2- 172.21.15.111:0/2939244282 >> v2:172.21.15.119:3300/0 conn(0x7f06a806c300 0x7f06a80d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:29.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.562+0000 7f06ad9ff700 1 -- 172.21.15.111:0/2939244282 >> 172.21.15.111:0/2939244282 conn(0x7f06a8071b10 msgr2=0x7f06a8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:29.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.562+0000 7f06ad9ff700 1 -- 172.21.15.111:0/2939244282 shutdown_connections 2023-03-29T10:58:29.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.562+0000 7f06ad9ff700 1 -- 172.21.15.111:0/2939244282 wait complete. 2023-03-29T10:58:29.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.564+0000 7f06ad9ff700 1 Processor -- start 2023-03-29T10:58:29.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.564+0000 7f06ad9ff700 1 -- start start 2023-03-29T10:58:29.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.565+0000 7f06ad9ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06a806c300 0x7f06a815d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:29.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.565+0000 7f06ad9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06a80b2430 0x7f06a81f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:29.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.565+0000 7f06ad9ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06a8160f40 0x7f06a8154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:29.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.565+0000 7f06ad9ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a81f2400 con 0x7f06a806c300 2023-03-29T10:58:29.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.565+0000 7f06ad9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a81f1db0 con 0x7f06a80b2430 2023-03-29T10:58:29.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.565+0000 7f06ad9ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a815cf50 con 0x7f06a8160f40 2023-03-29T10:58:29.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.565+0000 7f06a6ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06a806c300 0x7f06a815d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:29.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.565+0000 7f06a6ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06a806c300 0x7f06a815d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44688/0 (socket says 172.21.15.111:44688) 2023-03-29T10:58:29.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.565+0000 7f06a6ffd700 1 -- 172.21.15.111:0/856546532 learned_addr learned my addr 172.21.15.111:0/856546532 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:29.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.565+0000 7f06a6ffd700 1 -- 172.21.15.111:0/856546532 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06a8160f40 msgr2=0x7f06a8154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:29.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.565+0000 7f06a6ffd700 1 --2- 172.21.15.111:0/856546532 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06a8160f40 0x7f06a8154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:29.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.565+0000 7f06a6ffd700 1 -- 172.21.15.111:0/856546532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06a80b2430 msgr2=0x7f06a81f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:29.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.565+0000 7f06a67fc700 1 --2- 172.21.15.111:0/856546532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06a80b2430 0x7f06a81f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:29.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.565+0000 7f06a6ffd700 1 --2- 172.21.15.111:0/856546532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06a80b2430 0x7f06a81f2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:29.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.565+0000 7f06a6ffd700 1 -- 172.21.15.111:0/856546532 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069c0094a0 con 0x7f06a806c300 2023-03-29T10:58:29.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.565+0000 7f06a67fc700 1 --2- 172.21.15.111:0/856546532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06a80b2430 0x7f06a81f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:29.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.566+0000 7f06a6ffd700 1 --2- 172.21.15.111:0/856546532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06a806c300 0x7f06a815d770 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f069800c520 tx=0x7f069800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:29.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.566+0000 7f068ffff700 1 -- 172.21.15.111:0/856546532 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0698005d70 con 0x7f06a806c300 2023-03-29T10:58:29.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.566+0000 7f068ffff700 1 -- 172.21.15.111:0/856546532 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06980070d0 con 0x7f06a806c300 2023-03-29T10:58:29.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.566+0000 7f06ad9ff700 1 -- 172.21.15.111:0/856546532 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a0009a70 con 0x7f06a806c300 2023-03-29T10:58:29.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.566+0000 7f068ffff700 1 -- 172.21.15.111:0/856546532 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0698007440 con 0x7f06a806c300 2023-03-29T10:58:29.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.566+0000 7f06ad9ff700 1 -- 172.21.15.111:0/856546532 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a80d05f0 con 0x7f06a806c300 2023-03-29T10:58:29.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.567+0000 7f06ad9ff700 1 -- 172.21.15.111:0/856546532 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06a80d2c20 con 0x7f06a806c300 2023-03-29T10:58:29.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.568+0000 7f068ffff700 1 -- 172.21.15.111:0/856546532 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f06980075a0 con 0x7f06a806c300 2023-03-29T10:58:29.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.569+0000 7f068ffff700 1 --2- 172.21.15.111:0/856546532 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0694069de0 0x7f069406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:29.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.569+0000 7f068ffff700 1 -- 172.21.15.111:0/856546532 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0698087700 con 0x7f06a806c300 2023-03-29T10:58:29.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.569+0000 7f06a67fc700 1 --2- 172.21.15.111:0/856546532 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0694069de0 0x7f069406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:29.570 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.570+0000 7f06a67fc700 1 --2- 172.21.15.111:0/856546532 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0694069de0 0x7f069406c290 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f06a806d5d0 tx=0x7f069c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:29.574 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.574+0000 7f068ffff700 1 -- 172.21.15.111:0/856546532 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f069804c160 con 0x7f06a806c300 2023-03-29T10:58:29.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.766+0000 7f06ad9ff700 1 -- 172.21.15.111:0/856546532 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06a80d0780 con 0x7f06a806c300 2023-03-29T10:58:29.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.768+0000 7f068ffff700 1 -- 172.21.15.111:0/856546532 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19892 (secure 0 0 0) 0x7f0698052d20 con 0x7f06a806c300 2023-03-29T10:58:29.769 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:29.769 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":30,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:58:09.476334+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.26","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:58:29.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.770+0000 7f06ad9ff700 1 -- 172.21.15.111:0/856546532 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0694069de0 msgr2=0x7f069406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:29.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.770+0000 7f06ad9ff700 1 --2- 172.21.15.111:0/856546532 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0694069de0 0x7f069406c290 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f06a806d5d0 tx=0x7f069c008040 comp rx=0 tx=0).stop 2023-03-29T10:58:29.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.770+0000 7f06ad9ff700 1 -- 172.21.15.111:0/856546532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06a806c300 msgr2=0x7f06a815d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:29.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.770+0000 7f06ad9ff700 1 --2- 172.21.15.111:0/856546532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06a806c300 0x7f06a815d770 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f069800c520 tx=0x7f069800c8e0 comp rx=0 tx=0).stop 2023-03-29T10:58:29.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.770+0000 7f06ad9ff700 1 -- 172.21.15.111:0/856546532 shutdown_connections 2023-03-29T10:58:29.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.770+0000 7f06ad9ff700 1 --2- 172.21.15.111:0/856546532 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0694069de0 0x7f069406c290 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:29.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.770+0000 7f06ad9ff700 1 --2- 172.21.15.111:0/856546532 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06a806c300 0x7f06a815d770 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:29.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.770+0000 7f06ad9ff700 1 --2- 172.21.15.111:0/856546532 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06a8160f40 0x7f06a8154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:29.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.770+0000 7f06ad9ff700 1 --2- 172.21.15.111:0/856546532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06a80b2430 0x7f06a81f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:29.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.770+0000 7f06ad9ff700 1 -- 172.21.15.111:0/856546532 >> 172.21.15.111:0/856546532 conn(0x7f06a8071b10 msgr2=0x7f06a80b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:29.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.770+0000 7f06ad9ff700 1 -- 172.21.15.111:0/856546532 shutdown_connections 2023-03-29T10:58:29.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.770+0000 7f06ad9ff700 1 -- 172.21.15.111:0/856546532 wait complete. 2023-03-29T10:58:29.788 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-29T10:57:16.183979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-29T10:57:21.239809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-29T10:57:26.192124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-29T10:57:26.796777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-29T10:57:29.874795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-29T10:58:29.789 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-29T10:58:29.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.972+0000 7fc6711ff700 1 Processor -- start 2023-03-29T10:58:29.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.972+0000 7fc6711ff700 1 -- start start 2023-03-29T10:58:29.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.973+0000 7fc6711ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc66c1589d0 0x7fc66c0cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:29.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.973+0000 7fc6711ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc66c06cdd0 0x7fc66c06a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:29.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.973+0000 7fc6711ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66c0ca890 con 0x7fc66c0cd690 2023-03-29T10:58:29.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.973+0000 7fc6711ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc66c0ca5b0 con 0x7fc66c06cdd0 2023-03-29T10:58:29.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.973+0000 7fc6711ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc66c0cc670 con 0x7fc66c1589d0 2023-03-29T10:58:29.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.974+0000 7fc66b59e700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fc66c06cdd0 0x7fc66c06a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:29.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.974+0000 7fc663fff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fc66c0cd690 0x7fc66c06c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43514/0 (socket says 172.21.15.111:43514) 2023-03-29T10:58:29.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.974+0000 7fc663fff700 1 -- 172.21.15.111:0/1074897344 learned_addr learned my addr 172.21.15.111:0/1074897344 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:29.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.974+0000 7fc66ad9d700 1 --2- 172.21.15.111:0/1074897344 >> v2:172.21.15.119:3300/0 conn(0x7fc66c1589d0 0x7fc66c0cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:29.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.974+0000 7fc66b59e700 1 -- 172.21.15.111:0/1074897344 >> v2:172.21.15.119:3300/0 conn(0x7fc66c1589d0 msgr2=0x7fc66c0cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:29.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.974+0000 7fc66b59e700 1 --2- 172.21.15.111:0/1074897344 >> v2:172.21.15.119:3300/0 conn(0x7fc66c1589d0 0x7fc66c0cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:29.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.974+0000 7fc66b59e700 1 -- 172.21.15.111:0/1074897344 >> v1:172.21.15.111:6789/0 conn(0x7fc66c0cd690 legacy=0x7fc66c06c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:29.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.974+0000 7fc66b59e700 1 -- 172.21.15.111:0/1074897344 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66c0cc950 con 0x7fc66c06cdd0 2023-03-29T10:58:29.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.974+0000 7fc66ad9d700 1 --2- 172.21.15.111:0/1074897344 >> v2:172.21.15.119:3300/0 conn(0x7fc66c1589d0 0x7fc66c0cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:29.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.974+0000 7fc66a59c700 1 -- 172.21.15.111:0/1074897344 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127149813 0 0) 0x7fc66c0ca890 con 0x7fc66c0cd690 2023-03-29T10:58:29.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.974+0000 7fc66b59e700 1 --2- 172.21.15.111:0/1074897344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc66c06cdd0 0x7fc66c06a3e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc66c067990 tx=0x7fc66400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c3ee008b6fc7f2 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:29.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.974+0000 7fc66a59c700 1 -- 172.21.15.111:0/1074897344 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66400e070 con 0x7fc66c06cdd0 2023-03-29T10:58:29.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.975+0000 7fc66a59c700 1 -- 172.21.15.111:0/1074897344 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66400bbd0 con 0x7fc66c06cdd0 2023-03-29T10:58:29.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.975+0000 7fc66a59c700 1 -- 172.21.15.111:0/1074897344 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc664012430 con 0x7fc66c06cdd0 2023-03-29T10:58:29.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.975+0000 7fc6711ff700 1 -- 172.21.15.111:0/1074897344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc66c06cdd0 msgr2=0x7fc66c06a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:29.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.975+0000 7fc6711ff700 1 --2- 172.21.15.111:0/1074897344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc66c06cdd0 0x7fc66c06a3e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc66c067990 tx=0x7fc66400b330 comp rx=0 tx=0).stop 2023-03-29T10:58:29.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.975+0000 7fc6711ff700 1 -- 172.21.15.111:0/1074897344 shutdown_connections 2023-03-29T10:58:29.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.975+0000 7fc6711ff700 1 --2- 172.21.15.111:0/1074897344 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc66c06cdd0 0x7fc66c06a3e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:29.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.975+0000 7fc6711ff700 1 --2- 172.21.15.111:0/1074897344 >> v2:172.21.15.119:3300/0 conn(0x7fc66c1589d0 0x7fc66c0cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:29.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.975+0000 7fc6711ff700 1 -- 172.21.15.111:0/1074897344 >> 172.21.15.111:0/1074897344 conn(0x7fc66c0cf720 msgr2=0x7fc66c0cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:29.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.975+0000 7fc6711ff700 1 -- 172.21.15.111:0/1074897344 shutdown_connections 2023-03-29T10:58:29.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.975+0000 7fc6711ff700 1 -- 172.21.15.111:0/1074897344 wait complete. 2023-03-29T10:58:29.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.977+0000 7fc6711ff700 1 Processor -- start 2023-03-29T10:58:29.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.977+0000 7fc6711ff700 1 -- start start 2023-03-29T10:58:29.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.978+0000 7fc6711ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66c06cdd0 0x7fc66c0c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:29.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.978+0000 7fc6711ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc66c0cd690 0x7fc66c0b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:29.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.978+0000 7fc6711ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc66c1589d0 0x7fc66c0c2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:29.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.978+0000 7fc6711ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c0c7060 con 0x7fc66c0cd690 2023-03-29T10:58:29.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.978+0000 7fc6711ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c0c7300 con 0x7fc66c06cdd0 2023-03-29T10:58:29.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.978+0000 7fc6711ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c0c75e0 con 0x7fc66c1589d0 2023-03-29T10:58:29.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.978+0000 7fc663fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc66c0cd690 0x7fc66c0b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:29.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.978+0000 7fc663fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc66c0cd690 0x7fc66c0b07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44696/0 (socket says 172.21.15.111:44696) 2023-03-29T10:58:29.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.978+0000 7fc663fff700 1 -- 172.21.15.111:0/775737583 learned_addr learned my addr 172.21.15.111:0/775737583 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:29.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.978+0000 7fc663fff700 1 -- 172.21.15.111:0/775737583 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc66c1589d0 msgr2=0x7fc66c0c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:29.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.978+0000 7fc663fff700 1 --2- 172.21.15.111:0/775737583 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc66c1589d0 0x7fc66c0c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:29.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.979+0000 7fc663fff700 1 -- 172.21.15.111:0/775737583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66c06cdd0 msgr2=0x7fc66c0c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:29.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.979+0000 7fc66ad9d700 1 --2- 172.21.15.111:0/775737583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66c06cdd0 0x7fc66c0c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:29.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.979+0000 7fc663fff700 1 --2- 172.21.15.111:0/775737583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66c06cdd0 0x7fc66c0c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:29.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.979+0000 7fc663fff700 1 -- 172.21.15.111:0/775737583 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc664009a70 con 0x7fc66c0cd690 2023-03-29T10:58:29.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.979+0000 7fc66ad9d700 1 --2- 172.21.15.111:0/775737583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66c06cdd0 0x7fc66c0c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:29.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.979+0000 7fc663fff700 1 --2- 172.21.15.111:0/775737583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc66c0cd690 0x7fc66c0b07d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc6540010e0 tx=0x7fc65400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:29.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.979+0000 7fc6637fe700 1 -- 172.21.15.111:0/775737583 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6540074e0 con 0x7fc66c0cd690 2023-03-29T10:58:29.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.979+0000 7fc6637fe700 1 -- 172.21.15.111:0/775737583 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc654007640 con 0x7fc66c0cd690 2023-03-29T10:58:29.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.979+0000 7fc6637fe700 1 -- 172.21.15.111:0/775737583 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc654005a70 con 0x7fc66c0cd690 2023-03-29T10:58:29.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.979+0000 7fc6711ff700 1 -- 172.21.15.111:0/775737583 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc66c158da0 con 0x7fc66c0cd690 2023-03-29T10:58:29.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.979+0000 7fc6711ff700 1 -- 172.21.15.111:0/775737583 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66c0ccc30 con 0x7fc66c0cd690 2023-03-29T10:58:29.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.980+0000 7fc6711ff700 1 -- 172.21.15.111:0/775737583 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66c158f30 con 0x7fc66c0cd690 2023-03-29T10:58:29.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.981+0000 7fc6637fe700 1 -- 172.21.15.111:0/775737583 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fc654005bd0 con 0x7fc66c0cd690 2023-03-29T10:58:29.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.982+0000 7fc6637fe700 1 --2- 172.21.15.111:0/775737583 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc64c069de0 0x7fc64c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:29.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.982+0000 7fc6637fe700 1 -- 172.21.15.111:0/775737583 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc654087900 con 0x7fc66c0cd690 2023-03-29T10:58:29.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.982+0000 7fc66ad9d700 1 --2- 172.21.15.111:0/775737583 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc64c069de0 0x7fc64c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:29.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.983+0000 7fc66ad9d700 1 --2- 172.21.15.111:0/775737583 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc64c069de0 0x7fc64c06c290 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009820 tx=0x7fc65c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:29.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:29.986+0000 7fc6637fe700 1 -- 172.21.15.111:0/775737583 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fc654052de0 con 0x7fc66c0cd690 2023-03-29T10:58:30.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.180+0000 7fc6711ff700 1 -- 172.21.15.111:0/775737583 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc66c0b4a10 con 0x7fc66c0cd690 2023-03-29T10:58:30.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.180+0000 7fc6637fe700 1 -- 172.21.15.111:0/775737583 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc6540170a0 con 0x7fc66c0cd690 2023-03-29T10:58:30.181 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-03-29T10:58:30.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.183+0000 7fc6711ff700 1 -- 172.21.15.111:0/775737583 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc64c069de0 msgr2=0x7fc64c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:30.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.183+0000 7fc6711ff700 1 --2- 172.21.15.111:0/775737583 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc64c069de0 0x7fc64c06c290 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009820 tx=0x7fc65c008040 comp rx=0 tx=0).stop 2023-03-29T10:58:30.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.183+0000 7fc6711ff700 1 -- 172.21.15.111:0/775737583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc66c0cd690 msgr2=0x7fc66c0b07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:30.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.183+0000 7fc6711ff700 1 --2- 172.21.15.111:0/775737583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc66c0cd690 0x7fc66c0b07d0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc6540010e0 tx=0x7fc65400cae0 comp rx=0 tx=0).stop 2023-03-29T10:58:30.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.183+0000 7fc6711ff700 1 -- 172.21.15.111:0/775737583 shutdown_connections 2023-03-29T10:58:30.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.183+0000 7fc6711ff700 1 --2- 172.21.15.111:0/775737583 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fc64c069de0 0x7fc64c06c290 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:30.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.183+0000 7fc6711ff700 1 --2- 172.21.15.111:0/775737583 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fc66c1589d0 0x7fc66c0c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:30.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.183+0000 7fc6711ff700 1 --2- 172.21.15.111:0/775737583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66c06cdd0 0x7fc66c0c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:30.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.183+0000 7fc6711ff700 1 --2- 172.21.15.111:0/775737583 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fc66c0cd690 0x7fc66c0b07d0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:30.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.183+0000 7fc6711ff700 1 -- 172.21.15.111:0/775737583 >> 172.21.15.111:0/775737583 conn(0x7fc66c0cf720 msgr2=0x7fc66c1520f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:30.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.183+0000 7fc6711ff700 1 -- 172.21.15.111:0/775737583 shutdown_connections 2023-03-29T10:58:30.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.183+0000 7fc6711ff700 1 -- 172.21.15.111:0/775737583 wait complete. 2023-03-29T10:58:30.201 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-29T10:58:30.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.386+0000 7fd9167ff700 1 Processor -- start 2023-03-29T10:58:30.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.386+0000 7fd9167ff700 1 -- start start 2023-03-29T10:58:30.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.387+0000 7fd9167ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd9100d2110 0x7fd910167550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:30.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.387+0000 7fd9167ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd910161000 con 0x7fd9100cfbf0 2023-03-29T10:58:30.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.387+0000 7fd9167ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9100d0020 con 0x7fd9100b87a0 2023-03-29T10:58:30.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.387+0000 7fd9167ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9100d02d0 con 0x7fd9100d2110 2023-03-29T10:58:30.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.387+0000 7fd90f7fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fd9100cfbf0 0x7fd9101608f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43524/0 (socket says 172.21.15.111:43524) 2023-03-29T10:58:30.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.387+0000 7fd90f7fe700 1 -- 172.21.15.111:0/1584338285 learned_addr learned my addr 172.21.15.111:0/1584338285 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:30.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.387+0000 7fd914d9c700 1 --2- 172.21.15.111:0/1584338285 >> v2:172.21.15.111:3300/0 conn(0x7fd9100d2110 0x7fd910167550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:30.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.388+0000 7fd914d9c700 1 -- 172.21.15.111:0/1584338285 >> v1:172.21.15.119:6789/0 conn(0x7fd9100b87a0 legacy=0x7fd9101601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:30.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.388+0000 7fd914d9c700 1 -- 172.21.15.111:0/1584338285 >> v1:172.21.15.111:6789/0 conn(0x7fd9100cfbf0 legacy=0x7fd9101608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:30.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.388+0000 7fd914d9c700 1 -- 172.21.15.111:0/1584338285 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9100cdc60 con 0x7fd9100d2110 2023-03-29T10:58:30.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.388+0000 7fd914d9c700 1 --2- 172.21.15.111:0/1584338285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9100d2110 0x7fd910167550 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd908009d90 tx=0x7fd90800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bffed538e6ec032 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:30.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.388+0000 7fd90effd700 1 -- 172.21.15.111:0/1584338285 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90800e070 con 0x7fd9100d2110 2023-03-29T10:58:30.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.388+0000 7fd90effd700 1 -- 172.21.15.111:0/1584338285 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90800bbd0 con 0x7fd9100d2110 2023-03-29T10:58:30.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.388+0000 7fd90effd700 1 -- 172.21.15.111:0/1584338285 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908012430 con 0x7fd9100d2110 2023-03-29T10:58:30.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.389+0000 7fd9167ff700 1 -- 172.21.15.111:0/1584338285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9100d2110 msgr2=0x7fd910167550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:30.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.389+0000 7fd9167ff700 1 --2- 172.21.15.111:0/1584338285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9100d2110 0x7fd910167550 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd908009d90 tx=0x7fd90800b330 comp rx=0 tx=0).stop 2023-03-29T10:58:30.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.389+0000 7fd9167ff700 1 -- 172.21.15.111:0/1584338285 shutdown_connections 2023-03-29T10:58:30.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.389+0000 7fd9167ff700 1 --2- 172.21.15.111:0/1584338285 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9100d2110 0x7fd910167550 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:30.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.389+0000 7fd9167ff700 1 -- 172.21.15.111:0/1584338285 >> 172.21.15.111:0/1584338285 conn(0x7fd910071b10 msgr2=0x7fd910071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:30.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.389+0000 7fd9167ff700 1 -- 172.21.15.111:0/1584338285 shutdown_connections 2023-03-29T10:58:30.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.389+0000 7fd9167ff700 1 -- 172.21.15.111:0/1584338285 wait complete. 2023-03-29T10:58:30.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.391+0000 7fd9167ff700 1 Processor -- start 2023-03-29T10:58:30.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.391+0000 7fd9167ff700 1 -- start start 2023-03-29T10:58:30.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.392+0000 7fd9167ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9100b87a0 0x7fd9100ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:30.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.392+0000 7fd9167ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9100cfbf0 0x7fd9100b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:30.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.392+0000 7fd9167ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9100d2110 0x7fd9101f59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:30.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.392+0000 7fd9167ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd910155640 con 0x7fd9100d2110 2023-03-29T10:58:30.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.392+0000 7fd9167ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd910155890 con 0x7fd9100b87a0 2023-03-29T10:58:30.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.392+0000 7fd9167ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd910155b70 con 0x7fd9100cfbf0 2023-03-29T10:58:30.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.392+0000 7fd914d9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9100d2110 0x7fd9101f59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:30.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.392+0000 7fd914d9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9100d2110 0x7fd9101f59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44718/0 (socket says 172.21.15.111:44718) 2023-03-29T10:58:30.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.392+0000 7fd914d9c700 1 -- 172.21.15.111:0/3275609998 learned_addr learned my addr 172.21.15.111:0/3275609998 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:30.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.392+0000 7fd90ffff700 1 --2- 172.21.15.111:0/3275609998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9100b87a0 0x7fd9100ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:30.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.392+0000 7fd90f7fe700 1 --2- 172.21.15.111:0/3275609998 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9100cfbf0 0x7fd9100b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:30.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.392+0000 7fd914d9c700 1 -- 172.21.15.111:0/3275609998 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9100cfbf0 msgr2=0x7fd9100b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:30.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.392+0000 7fd914d9c700 1 --2- 172.21.15.111:0/3275609998 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9100cfbf0 0x7fd9100b3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:30.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.392+0000 7fd914d9c700 1 -- 172.21.15.111:0/3275609998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9100b87a0 msgr2=0x7fd9100ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:30.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.392+0000 7fd914d9c700 1 --2- 172.21.15.111:0/3275609998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9100b87a0 0x7fd9100ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:30.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.392+0000 7fd914d9c700 1 -- 172.21.15.111:0/3275609998 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd908009a70 con 0x7fd9100d2110 2023-03-29T10:58:30.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.392+0000 7fd90ffff700 1 --2- 172.21.15.111:0/3275609998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9100b87a0 0x7fd9100ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:30.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.393+0000 7fd914d9c700 1 --2- 172.21.15.111:0/3275609998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9100d2110 0x7fd9101f59c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd908014040 tx=0x7fd908004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:30.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.393+0000 7fd90cff9700 1 -- 172.21.15.111:0/3275609998 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9080098f0 con 0x7fd9100d2110 2023-03-29T10:58:30.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.393+0000 7fd90cff9700 1 -- 172.21.15.111:0/3275609998 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd908020070 con 0x7fd9100d2110 2023-03-29T10:58:30.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.393+0000 7fd9167ff700 1 -- 172.21.15.111:0/3275609998 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9100cd980 con 0x7fd9100d2110 2023-03-29T10:58:30.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.393+0000 7fd90cff9700 1 -- 172.21.15.111:0/3275609998 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908004280 con 0x7fd9100d2110 2023-03-29T10:58:30.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.393+0000 7fd9167ff700 1 -- 172.21.15.111:0/3275609998 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9100cd6a0 con 0x7fd9100d2110 2023-03-29T10:58:30.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.394+0000 7fd9167ff700 1 -- 172.21.15.111:0/3275609998 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd910167a90 con 0x7fd9100d2110 2023-03-29T10:58:30.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.395+0000 7fd90cff9700 1 -- 172.21.15.111:0/3275609998 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fd9080043e0 con 0x7fd9100d2110 2023-03-29T10:58:30.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.395+0000 7fd90cff9700 1 --2- 172.21.15.111:0/3275609998 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd8fc069de0 0x7fd8fc06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:30.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.396+0000 7fd90cff9700 1 -- 172.21.15.111:0/3275609998 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd908022070 con 0x7fd9100d2110 2023-03-29T10:58:30.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.396+0000 7fd90ffff700 1 --2- 172.21.15.111:0/3275609998 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd8fc069de0 0x7fd8fc06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:30.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.396+0000 7fd90ffff700 1 --2- 172.21.15.111:0/3275609998 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd8fc069de0 0x7fd8fc06c290 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd9100c9ef0 tx=0x7fd900009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:30.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.400+0000 7fd90cff9700 1 -- 172.21.15.111:0/3275609998 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fd9080558a0 con 0x7fd9100d2110 2023-03-29T10:58:30.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.597+0000 7fd9167ff700 1 -- 172.21.15.111:0/3275609998 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd9100b52f0 con 0x7fd9100d2110 2023-03-29T10:58:30.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.597+0000 7fd90cff9700 1 -- 172.21.15.111:0/3275609998 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd908058ec0 con 0x7fd9100d2110 2023-03-29T10:58:30.598 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 8 2023-03-29T10:58:30.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.600+0000 7fd9167ff700 1 -- 172.21.15.111:0/3275609998 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd8fc069de0 msgr2=0x7fd8fc06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:30.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.600+0000 7fd9167ff700 1 --2- 172.21.15.111:0/3275609998 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd8fc069de0 0x7fd8fc06c290 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd9100c9ef0 tx=0x7fd900009040 comp rx=0 tx=0).stop 2023-03-29T10:58:30.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.600+0000 7fd9167ff700 1 -- 172.21.15.111:0/3275609998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9100d2110 msgr2=0x7fd9101f59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:30.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.600+0000 7fd9167ff700 1 --2- 172.21.15.111:0/3275609998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9100d2110 0x7fd9101f59c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd908014040 tx=0x7fd908004ad0 comp rx=0 tx=0).stop 2023-03-29T10:58:30.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.600+0000 7fd9167ff700 1 -- 172.21.15.111:0/3275609998 shutdown_connections 2023-03-29T10:58:30.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.600+0000 7fd9167ff700 1 --2- 172.21.15.111:0/3275609998 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd8fc069de0 0x7fd8fc06c290 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:30.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.600+0000 7fd9167ff700 1 --2- 172.21.15.111:0/3275609998 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd9100d2110 0x7fd9101f59c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:30.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.600+0000 7fd9167ff700 1 --2- 172.21.15.111:0/3275609998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9100b87a0 0x7fd9100ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:30.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.600+0000 7fd9167ff700 1 --2- 172.21.15.111:0/3275609998 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd9100cfbf0 0x7fd9100b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:30.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.600+0000 7fd9167ff700 1 -- 172.21.15.111:0/3275609998 >> 172.21.15.111:0/3275609998 conn(0x7fd910071b10 msgr2=0x7fd910154510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:30.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.600+0000 7fd9167ff700 1 -- 172.21.15.111:0/3275609998 shutdown_connections 2023-03-29T10:58:30.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.600+0000 7fd9167ff700 1 -- 172.21.15.111:0/3275609998 wait complete. 2023-03-29T10:58:30.618 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-29T10:58:30.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.802+0000 7fdb96dff700 1 Processor -- start 2023-03-29T10:58:30.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.802+0000 7fdb96dff700 1 -- start start 2023-03-29T10:58:30.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.803+0000 7fdb96dff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdb900b87a0 0x7fdb900cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:30.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.803+0000 7fdb96dff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb900cdc60 con 0x7fdb901601e0 2023-03-29T10:58:30.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.803+0000 7fdb96dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb900cd6a0 con 0x7fdb90160d40 2023-03-29T10:58:30.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.803+0000 7fdb96dff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb900d02d0 con 0x7fdb900b87a0 2023-03-29T10:58:30.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.803+0000 7fdb8ffff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fdb901601e0 0x7fdb90160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43538/0 (socket says 172.21.15.111:43538) 2023-03-29T10:58:30.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.803+0000 7fdb94b9b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdb900b87a0 0x7fdb900cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:30.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.803+0000 7fdb8ffff700 1 -- 172.21.15.111:0/1655215193 learned_addr learned my addr 172.21.15.111:0/1655215193 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:30.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.804+0000 7fdb94b9b700 1 -- 172.21.15.111:0/1655215193 >> v1:172.21.15.119:6789/0 conn(0x7fdb90160d40 legacy=0x7fdb900d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:30.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.804+0000 7fdb94b9b700 1 -- 172.21.15.111:0/1655215193 >> v1:172.21.15.111:6789/0 conn(0x7fdb901601e0 legacy=0x7fdb90160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:30.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.804+0000 7fdb94b9b700 1 -- 172.21.15.111:0/1655215193 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb900cd980 con 0x7fdb900b87a0 2023-03-29T10:58:30.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.804+0000 7fdb8f7fe700 1 -- 172.21.15.111:0/1655215193 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912584479 0 0) 0x7fdb900cdc60 con 0x7fdb901601e0 2023-03-29T10:58:30.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.804+0000 7fdb94b9b700 1 --2- 172.21.15.111:0/1655215193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb900b87a0 0x7fdb900cfbf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009840 tx=0x7fdb84009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137ce5d50bb6c436 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:30.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.804+0000 7fdb8f7fe700 1 -- 172.21.15.111:0/1655215193 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84011830 con 0x7fdb900b87a0 2023-03-29T10:58:30.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.804+0000 7fdb8f7fe700 1 -- 172.21.15.111:0/1655215193 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb84011990 con 0x7fdb900b87a0 2023-03-29T10:58:30.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.804+0000 7fdb8f7fe700 1 -- 172.21.15.111:0/1655215193 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84011d20 con 0x7fdb900b87a0 2023-03-29T10:58:30.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.805+0000 7fdb96dff700 1 -- 172.21.15.111:0/1655215193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb900b87a0 msgr2=0x7fdb900cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:30.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.805+0000 7fdb96dff700 1 --2- 172.21.15.111:0/1655215193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb900b87a0 0x7fdb900cfbf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009840 tx=0x7fdb84009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:30.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.805+0000 7fdb96dff700 1 -- 172.21.15.111:0/1655215193 shutdown_connections 2023-03-29T10:58:30.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.805+0000 7fdb96dff700 1 --2- 172.21.15.111:0/1655215193 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb900b87a0 0x7fdb900cfbf0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:30.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.805+0000 7fdb96dff700 1 -- 172.21.15.111:0/1655215193 >> 172.21.15.111:0/1655215193 conn(0x7fdb90071b10 msgr2=0x7fdb90071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:30.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.805+0000 7fdb96dff700 1 -- 172.21.15.111:0/1655215193 shutdown_connections 2023-03-29T10:58:30.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.805+0000 7fdb96dff700 1 -- 172.21.15.111:0/1655215193 wait complete. 2023-03-29T10:58:30.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.807+0000 7fdb96dff700 1 Processor -- start 2023-03-29T10:58:30.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.807+0000 7fdb96dff700 1 -- start start 2023-03-29T10:58:30.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.808+0000 7fdb96dff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb900b87a0 0x7fdb901545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:30.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.808+0000 7fdb96dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdb901601e0 0x7fdb90159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:30.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.808+0000 7fdb96dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb90160d40 0x7fdb90159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:30.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.808+0000 7fdb96dff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb900cb020 con 0x7fdb900b87a0 2023-03-29T10:58:30.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.808+0000 7fdb96dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb900cad40 con 0x7fdb90160d40 2023-03-29T10:58:30.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.808+0000 7fdb96dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb900c6af0 con 0x7fdb901601e0 2023-03-29T10:58:30.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.808+0000 7fdb94b9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb900b87a0 0x7fdb901545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:30.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.808+0000 7fdb94b9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb900b87a0 0x7fdb901545f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44746/0 (socket says 172.21.15.111:44746) 2023-03-29T10:58:30.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.808+0000 7fdb94b9b700 1 -- 172.21.15.111:0/4122399491 learned_addr learned my addr 172.21.15.111:0/4122399491 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:30.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.808+0000 7fdb9539c700 1 --2- 172.21.15.111:0/4122399491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb90160d40 0x7fdb90159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:30.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.808+0000 7fdb8ffff700 1 --2- 172.21.15.111:0/4122399491 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdb901601e0 0x7fdb90159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:30.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.808+0000 7fdb94b9b700 1 -- 172.21.15.111:0/4122399491 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdb901601e0 msgr2=0x7fdb90159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:30.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.808+0000 7fdb94b9b700 1 --2- 172.21.15.111:0/4122399491 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdb901601e0 0x7fdb90159530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:30.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.808+0000 7fdb94b9b700 1 -- 172.21.15.111:0/4122399491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb90160d40 msgr2=0x7fdb90159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:30.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.808+0000 7fdb94b9b700 1 --2- 172.21.15.111:0/4122399491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb90160d40 0x7fdb90159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:30.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.808+0000 7fdb94b9b700 1 -- 172.21.15.111:0/4122399491 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb840094a0 con 0x7fdb900b87a0 2023-03-29T10:58:30.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.809+0000 7fdb94b9b700 1 --2- 172.21.15.111:0/4122399491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb900b87a0 0x7fdb901545f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdb840090b0 tx=0x7fdb84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:30.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.809+0000 7fdb8d7fa700 1 -- 172.21.15.111:0/4122399491 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8401a900 con 0x7fdb900b87a0 2023-03-29T10:58:30.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.809+0000 7fdb8d7fa700 1 -- 172.21.15.111:0/4122399491 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8401aa60 con 0x7fdb900b87a0 2023-03-29T10:58:30.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.809+0000 7fdb96dff700 1 -- 172.21.15.111:0/4122399491 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb900d19a0 con 0x7fdb900b87a0 2023-03-29T10:58:30.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.809+0000 7fdb8d7fa700 1 -- 172.21.15.111:0/4122399491 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8401add0 con 0x7fdb900b87a0 2023-03-29T10:58:30.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.809+0000 7fdb96dff700 1 -- 172.21.15.111:0/4122399491 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb900d1810 con 0x7fdb900b87a0 2023-03-29T10:58:30.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.810+0000 7fdb96dff700 1 -- 172.21.15.111:0/4122399491 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb900d1b30 con 0x7fdb900b87a0 2023-03-29T10:58:30.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.811+0000 7fdb8d7fa700 1 -- 172.21.15.111:0/4122399491 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fdb84022070 con 0x7fdb900b87a0 2023-03-29T10:58:30.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.812+0000 7fdb8d7fa700 1 --2- 172.21.15.111:0/4122399491 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdb74069de0 0x7fdb7406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:30.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.812+0000 7fdb8d7fa700 1 -- 172.21.15.111:0/4122399491 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdb84089550 con 0x7fdb900b87a0 2023-03-29T10:58:30.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.812+0000 7fdb8ffff700 1 --2- 172.21.15.111:0/4122399491 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdb74069de0 0x7fdb7406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:30.813 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.813+0000 7fdb8ffff700 1 --2- 172.21.15.111:0/4122399491 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdb74069de0 0x7fdb7406c290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c007fd0 tx=0x7fdb7c007c20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:30.817 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:30.816+0000 7fdb8d7fa700 1 -- 172.21.15.111:0/4122399491 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fdb84054b70 con 0x7fdb900b87a0 2023-03-29T10:58:31.007 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.007+0000 7fdb96dff700 1 -- 172.21.15.111:0/4122399491 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdb900ccd90 con 0x7fdb900b87a0 2023-03-29T10:58:31.008 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.007+0000 7fdb8d7fa700 1 -- 172.21.15.111:0/4122399491 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdb84058190 con 0x7fdb900b87a0 2023-03-29T10:58:31.008 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:31.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.009+0000 7fdb96dff700 1 -- 172.21.15.111:0/4122399491 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdb74069de0 msgr2=0x7fdb7406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:31.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.009+0000 7fdb96dff700 1 --2- 172.21.15.111:0/4122399491 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdb74069de0 0x7fdb7406c290 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c007fd0 tx=0x7fdb7c007c20 comp rx=0 tx=0).stop 2023-03-29T10:58:31.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.009+0000 7fdb96dff700 1 -- 172.21.15.111:0/4122399491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb900b87a0 msgr2=0x7fdb901545f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:31.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.009+0000 7fdb96dff700 1 --2- 172.21.15.111:0/4122399491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb900b87a0 0x7fdb901545f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdb840090b0 tx=0x7fdb84004750 comp rx=0 tx=0).stop 2023-03-29T10:58:31.010 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.009+0000 7fdb96dff700 1 -- 172.21.15.111:0/4122399491 shutdown_connections 2023-03-29T10:58:31.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.009+0000 7fdb96dff700 1 --2- 172.21.15.111:0/4122399491 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdb74069de0 0x7fdb7406c290 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.009+0000 7fdb96dff700 1 --2- 172.21.15.111:0/4122399491 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdb900b87a0 0x7fdb901545f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.009+0000 7fdb96dff700 1 --2- 172.21.15.111:0/4122399491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb90160d40 0x7fdb90159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.009+0000 7fdb96dff700 1 --2- 172.21.15.111:0/4122399491 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdb901601e0 0x7fdb90159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.009+0000 7fdb96dff700 1 -- 172.21.15.111:0/4122399491 >> 172.21.15.111:0/4122399491 conn(0x7fdb90071b10 msgr2=0x7fdb900d1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:31.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.010+0000 7fdb96dff700 1 -- 172.21.15.111:0/4122399491 shutdown_connections 2023-03-29T10:58:31.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.010+0000 7fdb96dff700 1 -- 172.21.15.111:0/4122399491 wait complete. 2023-03-29T10:58:31.027 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-29T10:58:31.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.212+0000 7faed07ff700 1 Processor -- start 2023-03-29T10:58:31.214 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.212+0000 7faed07ff700 1 -- start start 2023-03-29T10:58:31.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.214+0000 7faed07ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faec80cd760 0x7faec80cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:31.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.214+0000 7faed07ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faec80adf70 0x7faec80cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:31.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.214+0000 7faed07ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec80cc9e0 con 0x7faec815cf10 2023-03-29T10:58:31.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.214+0000 7faed07ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec815de20 con 0x7faec80adf70 2023-03-29T10:58:31.217 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.214+0000 7faed07ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec80ce070 con 0x7faec80cd760 2023-03-29T10:58:31.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.214+0000 7faecdd9a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faec80adf70 0x7faec80cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:31.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.214+0000 7faecdd9a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7faec80adf70 0x7faec80cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44760/0 (socket says 172.21.15.111:44760) 2023-03-29T10:58:31.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.214+0000 7faecdd9a700 1 -- 172.21.15.111:0/2729585419 learned_addr learned my addr 172.21.15.111:0/2729585419 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:31.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.214+0000 7faece59b700 1 --2- 172.21.15.111:0/2729585419 >> v2:172.21.15.119:3300/0 conn(0x7faec80cd760 0x7faec80cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:31.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.214+0000 7faecdd9a700 1 -- 172.21.15.111:0/2729585419 >> v2:172.21.15.119:3300/0 conn(0x7faec80cd760 msgr2=0x7faec80cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:31.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.214+0000 7faecdd9a700 1 --2- 172.21.15.111:0/2729585419 >> v2:172.21.15.119:3300/0 conn(0x7faec80cd760 0x7faec80cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.214+0000 7faecdd9a700 1 -- 172.21.15.111:0/2729585419 >> v1:172.21.15.119:6789/0 conn(0x7faec815cf10 legacy=0x7faec806c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:31.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.214+0000 7faecdd9a700 1 -- 172.21.15.111:0/2729585419 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec815d860 con 0x7faec80adf70 2023-03-29T10:58:31.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.215+0000 7faecdd9a700 1 --2- 172.21.15.111:0/2729585419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faec80adf70 0x7faec80cce70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faec80cc230 tx=0x7faeb8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609fc5fd30815d7d server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:31.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.215+0000 7faecd599700 1 -- 172.21.15.111:0/2729585419 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb8005e30 con 0x7faec80adf70 2023-03-29T10:58:31.218 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.215+0000 7faecd599700 1 -- 172.21.15.111:0/2729585419 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb8004030 con 0x7faec80adf70 2023-03-29T10:58:31.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.215+0000 7faecd599700 1 -- 172.21.15.111:0/2729585419 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb80043e0 con 0x7faec80adf70 2023-03-29T10:58:31.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.215+0000 7faed07ff700 1 -- 172.21.15.111:0/2729585419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faec80adf70 msgr2=0x7faec80cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:31.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.215+0000 7faed07ff700 1 --2- 172.21.15.111:0/2729585419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faec80adf70 0x7faec80cce70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faec80cc230 tx=0x7faeb8009bd0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.215+0000 7faed07ff700 1 -- 172.21.15.111:0/2729585419 shutdown_connections 2023-03-29T10:58:31.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.215+0000 7faed07ff700 1 --2- 172.21.15.111:0/2729585419 >> v2:172.21.15.119:3300/0 conn(0x7faec80cd760 0x7faec80cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.215+0000 7faed07ff700 1 --2- 172.21.15.111:0/2729585419 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faec80adf70 0x7faec80cce70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.215+0000 7faed07ff700 1 -- 172.21.15.111:0/2729585419 >> 172.21.15.111:0/2729585419 conn(0x7faec80cf720 msgr2=0x7faec80cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:31.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.216+0000 7faed07ff700 1 -- 172.21.15.111:0/2729585419 shutdown_connections 2023-03-29T10:58:31.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.216+0000 7faed07ff700 1 -- 172.21.15.111:0/2729585419 wait complete. 2023-03-29T10:58:31.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.217+0000 7faed07ff700 1 Processor -- start 2023-03-29T10:58:31.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.217+0000 7faed07ff700 1 -- start start 2023-03-29T10:58:31.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.218+0000 7faed07ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec80adf70 0x7faec815a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:31.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.218+0000 7faed07ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faec80cd760 0x7faec8147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:31.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.218+0000 7faed07ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faec815cf10 0x7faec80b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:31.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.218+0000 7faed07ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec80b2c90 con 0x7faec80cd760 2023-03-29T10:58:31.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.218+0000 7faed07ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec815a870 con 0x7faec80adf70 2023-03-29T10:58:31.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.218+0000 7faed07ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec815ab50 con 0x7faec815cf10 2023-03-29T10:58:31.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.218+0000 7faecdd9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faec80cd760 0x7faec8147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:31.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.218+0000 7faecdd9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faec80cd760 0x7faec8147fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44770/0 (socket says 172.21.15.111:44770) 2023-03-29T10:58:31.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.218+0000 7faecdd9a700 1 -- 172.21.15.111:0/1440319927 learned_addr learned my addr 172.21.15.111:0/1440319927 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:31.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.219+0000 7faecdd9a700 1 -- 172.21.15.111:0/1440319927 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faec815cf10 msgr2=0x7faec80b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:31.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.219+0000 7faecdd9a700 1 --2- 172.21.15.111:0/1440319927 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faec815cf10 0x7faec80b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.219+0000 7faecdd9a700 1 -- 172.21.15.111:0/1440319927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec80adf70 msgr2=0x7faec815a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:31.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.219+0000 7faecdd9a700 1 --2- 172.21.15.111:0/1440319927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec80adf70 0x7faec815a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.219+0000 7faecdd9a700 1 -- 172.21.15.111:0/1440319927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb8009500 con 0x7faec80cd760 2023-03-29T10:58:31.221 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.219+0000 7faecdd9a700 1 --2- 172.21.15.111:0/1440319927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faec80cd760 0x7faec8147fc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faeb8009c10 tx=0x7faeb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:31.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.219+0000 7faebeffd700 1 -- 172.21.15.111:0/1440319927 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb8003d90 con 0x7faec80cd760 2023-03-29T10:58:31.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.219+0000 7faebeffd700 1 -- 172.21.15.111:0/1440319927 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb801c420 con 0x7faec80cd760 2023-03-29T10:58:31.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.219+0000 7faebeffd700 1 -- 172.21.15.111:0/1440319927 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb801c7b0 con 0x7faec80cd760 2023-03-29T10:58:31.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.219+0000 7faed07ff700 1 -- 172.21.15.111:0/1440319927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec80cc670 con 0x7faec80cd760 2023-03-29T10:58:31.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.219+0000 7faed07ff700 1 -- 172.21.15.111:0/1440319927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec815db40 con 0x7faec80cd760 2023-03-29T10:58:31.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.220+0000 7faed07ff700 1 -- 172.21.15.111:0/1440319927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec8071600 con 0x7faec80cd760 2023-03-29T10:58:31.222 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.222+0000 7faebeffd700 1 -- 172.21.15.111:0/1440319927 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7faeb801c910 con 0x7faec80cd760 2023-03-29T10:58:31.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.222+0000 7faebeffd700 1 --2- 172.21.15.111:0/1440319927 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7faeac069e30 0x7faeac06c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:31.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.222+0000 7faebeffd700 1 -- 172.21.15.111:0/1440319927 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7faeb8089380 con 0x7faec80cd760 2023-03-29T10:58:31.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.223+0000 7faece59b700 1 --2- 172.21.15.111:0/1440319927 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7faeac069e30 0x7faeac06c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:31.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.223+0000 7faece59b700 1 --2- 172.21.15.111:0/1440319927 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7faeac069e30 0x7faeac06c2e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faec4006b20 tx=0x7faec4006ab0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:31.227 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.227+0000 7faebeffd700 1 -- 172.21.15.111:0/1440319927 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7faeb8054a70 con 0x7faec80cd760 2023-03-29T10:58:31.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.420+0000 7faed07ff700 1 -- 172.21.15.111:0/1440319927 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faec8071b50 con 0x7faec80cd760 2023-03-29T10:58:31.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.421+0000 7faebeffd700 1 -- 172.21.15.111:0/1440319927 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7faeb8058090 con 0x7faec80cd760 2023-03-29T10:58:31.422 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-03-29T10:58:31.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.423+0000 7faed07ff700 1 -- 172.21.15.111:0/1440319927 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7faeac069e30 msgr2=0x7faeac06c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:31.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.423+0000 7faed07ff700 1 --2- 172.21.15.111:0/1440319927 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7faeac069e30 0x7faeac06c2e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faec4006b20 tx=0x7faec4006ab0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.423+0000 7faed07ff700 1 -- 172.21.15.111:0/1440319927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faec80cd760 msgr2=0x7faec8147fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:31.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.423+0000 7faed07ff700 1 --2- 172.21.15.111:0/1440319927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faec80cd760 0x7faec8147fc0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faeb8009c10 tx=0x7faeb8004750 comp rx=0 tx=0).stop 2023-03-29T10:58:31.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.423+0000 7faed07ff700 1 -- 172.21.15.111:0/1440319927 shutdown_connections 2023-03-29T10:58:31.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.423+0000 7faed07ff700 1 --2- 172.21.15.111:0/1440319927 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7faeac069e30 0x7faeac06c2e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.423+0000 7faed07ff700 1 --2- 172.21.15.111:0/1440319927 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faec815cf10 0x7faec80b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.423+0000 7faed07ff700 1 --2- 172.21.15.111:0/1440319927 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec80adf70 0x7faec815a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.423+0000 7faed07ff700 1 --2- 172.21.15.111:0/1440319927 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faec80cd760 0x7faec8147fc0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.423+0000 7faed07ff700 1 -- 172.21.15.111:0/1440319927 >> 172.21.15.111:0/1440319927 conn(0x7faec80cf720 msgr2=0x7faec8155700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:31.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.423+0000 7faed07ff700 1 -- 172.21.15.111:0/1440319927 shutdown_connections 2023-03-29T10:58:31.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.423+0000 7faed07ff700 1 -- 172.21.15.111:0/1440319927 wait complete. 2023-03-29T10:58:31.441 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-29T10:58:31.629 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.629+0000 7f76055ff700 1 Processor -- start 2023-03-29T10:58:31.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.629+0000 7f76055ff700 1 -- start start 2023-03-29T10:58:31.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.630+0000 7f76055ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76001589d0 0x7f76000cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:31.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.630+0000 7f76055ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f760006cdd0 0x7f760006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:31.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.630+0000 7f76055ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76000ca890 con 0x7f76000cd690 2023-03-29T10:58:31.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.630+0000 7f76055ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76000ca5b0 con 0x7f760006cdd0 2023-03-29T10:58:31.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.630+0000 7f76055ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76000cc670 con 0x7f76001589d0 2023-03-29T10:58:31.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.630+0000 7f75ff7fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f760006cdd0 0x7f760006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:31.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.630+0000 7f75fe7fc700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f76000cd690 0x7f760006c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43550/0 (socket says 172.21.15.111:43550) 2023-03-29T10:58:31.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.630+0000 7f75fe7fc700 1 -- 172.21.15.111:0/211907810 learned_addr learned my addr 172.21.15.111:0/211907810 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:31.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.631+0000 7f75feffd700 1 --2- 172.21.15.111:0/211907810 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76001589d0 0x7f76000cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:31.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.631+0000 7f75ff7fe700 1 -- 172.21.15.111:0/211907810 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76001589d0 msgr2=0x7f76000cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:31.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.631+0000 7f75ff7fe700 1 --2- 172.21.15.111:0/211907810 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76001589d0 0x7f76000cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.631+0000 7f75ff7fe700 1 -- 172.21.15.111:0/211907810 >> v1:172.21.15.111:6789/0 conn(0x7f76000cd690 legacy=0x7f760006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:31.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.631+0000 7f75ff7fe700 1 -- 172.21.15.111:0/211907810 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76000cc950 con 0x7f760006cdd0 2023-03-29T10:58:31.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.631+0000 7f75fdffb700 1 -- 172.21.15.111:0/211907810 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2995435462 0 0) 0x7f76000ca890 con 0x7f76000cd690 2023-03-29T10:58:31.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.631+0000 7f75feffd700 1 --2- 172.21.15.111:0/211907810 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76001589d0 0x7f76000cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:31.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.631+0000 7f75ff7fe700 1 --2- 172.21.15.111:0/211907810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f760006cdd0 0x7f760006a3e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f75f8009d90 tx=0x7f75f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92d526ff84c93a2 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:31.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.631+0000 7f75fdffb700 1 -- 172.21.15.111:0/211907810 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f800e070 con 0x7f760006cdd0 2023-03-29T10:58:31.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.631+0000 7f75fdffb700 1 -- 172.21.15.111:0/211907810 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f800bbd0 con 0x7f760006cdd0 2023-03-29T10:58:31.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.631+0000 7f75fdffb700 1 -- 172.21.15.111:0/211907810 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f8012430 con 0x7f760006cdd0 2023-03-29T10:58:31.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.632+0000 7f76055ff700 1 -- 172.21.15.111:0/211907810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f760006cdd0 msgr2=0x7f760006a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:31.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.632+0000 7f76055ff700 1 --2- 172.21.15.111:0/211907810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f760006cdd0 0x7f760006a3e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f75f8009d90 tx=0x7f75f800b330 comp rx=0 tx=0).stop 2023-03-29T10:58:31.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.632+0000 7f76055ff700 1 -- 172.21.15.111:0/211907810 shutdown_connections 2023-03-29T10:58:31.633 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.632+0000 7f76055ff700 1 --2- 172.21.15.111:0/211907810 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f760006cdd0 0x7f760006a3e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.632+0000 7f76055ff700 1 --2- 172.21.15.111:0/211907810 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76001589d0 0x7f76000cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.632+0000 7f76055ff700 1 -- 172.21.15.111:0/211907810 >> 172.21.15.111:0/211907810 conn(0x7f76000cf720 msgr2=0x7f76000cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:31.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.632+0000 7f76055ff700 1 -- 172.21.15.111:0/211907810 shutdown_connections 2023-03-29T10:58:31.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.632+0000 7f76055ff700 1 -- 172.21.15.111:0/211907810 wait complete. 2023-03-29T10:58:31.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.634+0000 7f76055ff700 1 Processor -- start 2023-03-29T10:58:31.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.634+0000 7f76055ff700 1 -- start start 2023-03-29T10:58:31.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.635+0000 7f76055ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f760006cdd0 0x7f760014bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:31.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.635+0000 7f76055ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76000cd690 0x7f760014a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:31.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.635+0000 7f76055ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76001589d0 0x7f76001e1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:31.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.635+0000 7f76055ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76001e1680 con 0x7f760006cdd0 2023-03-29T10:58:31.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.635+0000 7f76055ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76000c6b20 con 0x7f76001589d0 2023-03-29T10:58:31.636 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.635+0000 7f76055ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760014c390 con 0x7f76000cd690 2023-03-29T10:58:31.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.635+0000 7f75feffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f760006cdd0 0x7f760014bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:31.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.635+0000 7f75feffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f760006cdd0 0x7f760014bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44782/0 (socket says 172.21.15.111:44782) 2023-03-29T10:58:31.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.635+0000 7f75feffd700 1 -- 172.21.15.111:0/1014083383 learned_addr learned my addr 172.21.15.111:0/1014083383 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:31.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.635+0000 7f75fe7fc700 1 --2- 172.21.15.111:0/1014083383 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76000cd690 0x7f760014a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:31.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.635+0000 7f75ff7fe700 1 --2- 172.21.15.111:0/1014083383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76001589d0 0x7f76001e1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:31.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.635+0000 7f75feffd700 1 -- 172.21.15.111:0/1014083383 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76000cd690 msgr2=0x7f760014a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:31.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.635+0000 7f75feffd700 1 --2- 172.21.15.111:0/1014083383 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76000cd690 0x7f760014a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.635+0000 7f75feffd700 1 -- 172.21.15.111:0/1014083383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76001589d0 msgr2=0x7f76001e1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:31.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.635+0000 7f75feffd700 1 --2- 172.21.15.111:0/1014083383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76001589d0 0x7f76001e1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.635+0000 7f75feffd700 1 -- 172.21.15.111:0/1014083383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f8009a70 con 0x7f760006cdd0 2023-03-29T10:58:31.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.636+0000 7f75feffd700 1 --2- 172.21.15.111:0/1014083383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f760006cdd0 0x7f760014bdb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f75f000c750 tx=0x7f75f000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:31.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.636+0000 7f75e77fe700 1 -- 172.21.15.111:0/1014083383 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0011800 con 0x7f760006cdd0 2023-03-29T10:58:31.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.636+0000 7f75e77fe700 1 -- 172.21.15.111:0/1014083383 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f0011960 con 0x7f760006cdd0 2023-03-29T10:58:31.637 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.636+0000 7f76055ff700 1 -- 172.21.15.111:0/1014083383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7600158da0 con 0x7f760006cdd0 2023-03-29T10:58:31.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.636+0000 7f75e77fe700 1 -- 172.21.15.111:0/1014083383 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0011cd0 con 0x7f760006cdd0 2023-03-29T10:58:31.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.636+0000 7f76055ff700 1 -- 172.21.15.111:0/1014083383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76000ccc30 con 0x7f760006cdd0 2023-03-29T10:58:31.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.637+0000 7f76055ff700 1 -- 172.21.15.111:0/1014083383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7600158f30 con 0x7f760006cdd0 2023-03-29T10:58:31.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.638+0000 7f75e77fe700 1 -- 172.21.15.111:0/1014083383 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f75f0021350 con 0x7f760006cdd0 2023-03-29T10:58:31.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.639+0000 7f75e77fe700 1 --2- 172.21.15.111:0/1014083383 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f75ec069de0 0x7f75ec06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:31.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.639+0000 7f75e77fe700 1 -- 172.21.15.111:0/1014083383 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f75f00882e0 con 0x7f760006cdd0 2023-03-29T10:58:31.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.639+0000 7f75fe7fc700 1 --2- 172.21.15.111:0/1014083383 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f75ec069de0 0x7f75ec06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:31.640 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.639+0000 7f75fe7fc700 1 --2- 172.21.15.111:0/1014083383 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f75ec069de0 0x7f75ec06c290 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f75f4007fd0 tx=0x7f75f4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:31.643 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.643+0000 7f75e77fe700 1 -- 172.21.15.111:0/1014083383 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f75f0057280 con 0x7f760006cdd0 2023-03-29T10:58:31.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.836+0000 7f76055ff700 1 -- 172.21.15.111:0/1014083383 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f76000c2010 con 0x7f760006cdd0 2023-03-29T10:58:31.837 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.837+0000 7f75e77fe700 1 -- 172.21.15.111:0/1014083383 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f75f0056e10 con 0x7f760006cdd0 2023-03-29T10:58:31.837 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:31.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.839+0000 7f76055ff700 1 -- 172.21.15.111:0/1014083383 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f75ec069de0 msgr2=0x7f75ec06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:31.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.839+0000 7f76055ff700 1 --2- 172.21.15.111:0/1014083383 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f75ec069de0 0x7f75ec06c290 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f75f4007fd0 tx=0x7f75f4009040 comp rx=0 tx=0).stop 2023-03-29T10:58:31.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.839+0000 7f76055ff700 1 -- 172.21.15.111:0/1014083383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f760006cdd0 msgr2=0x7f760014bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:31.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.839+0000 7f76055ff700 1 --2- 172.21.15.111:0/1014083383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f760006cdd0 0x7f760014bdb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f75f000c750 tx=0x7f75f000cb10 comp rx=0 tx=0).stop 2023-03-29T10:58:31.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.839+0000 7f76055ff700 1 -- 172.21.15.111:0/1014083383 shutdown_connections 2023-03-29T10:58:31.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.839+0000 7f76055ff700 1 --2- 172.21.15.111:0/1014083383 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f75ec069de0 0x7f75ec06c290 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.839+0000 7f76055ff700 1 --2- 172.21.15.111:0/1014083383 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f760006cdd0 0x7f760014bdb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.839+0000 7f76055ff700 1 --2- 172.21.15.111:0/1014083383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76001589d0 0x7f76001e1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.839+0000 7f76055ff700 1 --2- 172.21.15.111:0/1014083383 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f76000cd690 0x7f760014a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:31.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.839+0000 7f76055ff700 1 -- 172.21.15.111:0/1014083383 >> 172.21.15.111:0/1014083383 conn(0x7f76000cf720 msgr2=0x7f76000ad210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:31.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.839+0000 7f76055ff700 1 -- 172.21.15.111:0/1014083383 shutdown_connections 2023-03-29T10:58:31.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:31.839+0000 7f76055ff700 1 -- 172.21.15.111:0/1014083383 wait complete. 2023-03-29T10:58:31.858 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-03-29T10:58:32.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.041+0000 7f516ffff700 1 Processor -- start 2023-03-29T10:58:32.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.041+0000 7f516ffff700 1 -- start start 2023-03-29T10:58:32.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.042+0000 7f516ffff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f51680cf010 0x7f5168160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:32.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.042+0000 7f516ffff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f51680cff50 0x7f51680d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:32.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.042+0000 7f516ffff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51680cd520 con 0x7f516815c590 2023-03-29T10:58:32.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.042+0000 7f516ffff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51680d2640 con 0x7f51680cf010 2023-03-29T10:58:32.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.042+0000 7f516ffff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51681610d0 con 0x7f51680cff50 2023-03-29T10:58:32.043 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.042+0000 7f516d59a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f51680cf010 0x7f5168160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:32.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.042+0000 7f516dd9b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f516815c590 0x7f51680cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43566/0 (socket says 172.21.15.111:43566) 2023-03-29T10:58:32.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.042+0000 7f516dd9b700 1 -- 172.21.15.111:0/3584641756 learned_addr learned my addr 172.21.15.111:0/3584641756 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:32.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.042+0000 7f516e59c700 1 --2- 172.21.15.111:0/3584641756 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f51680cff50 0x7f51680d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:32.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.043+0000 7f516cd99700 1 -- 172.21.15.111:0/3584641756 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562196530 0 0) 0x7f51680cd520 con 0x7f516815c590 2023-03-29T10:58:32.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.043+0000 7f516d59a700 1 -- 172.21.15.111:0/3584641756 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f51680cff50 msgr2=0x7f51680d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:32.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.043+0000 7f516d59a700 1 --2- 172.21.15.111:0/3584641756 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f51680cff50 0x7f51680d2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.043+0000 7f516d59a700 1 -- 172.21.15.111:0/3584641756 >> v1:172.21.15.111:6789/0 conn(0x7f516815c590 legacy=0x7f51680cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:32.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.043+0000 7f516d59a700 1 -- 172.21.15.111:0/3584641756 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51680b23c0 con 0x7f51680cf010 2023-03-29T10:58:32.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.043+0000 7f516d59a700 1 --2- 172.21.15.111:0/3584641756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51680cf010 0x7f5168160b90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5158009840 tx=0x7f5158009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8b3103ba51f88f server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:32.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.043+0000 7f516cd99700 1 -- 172.21.15.111:0/3584641756 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158011830 con 0x7f51680cf010 2023-03-29T10:58:32.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.043+0000 7f516cd99700 1 -- 172.21.15.111:0/3584641756 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5158011990 con 0x7f51680cf010 2023-03-29T10:58:32.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.043+0000 7f516cd99700 1 -- 172.21.15.111:0/3584641756 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5158011d20 con 0x7f51680cf010 2023-03-29T10:58:32.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.044+0000 7f516ffff700 1 -- 172.21.15.111:0/3584641756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51680cf010 msgr2=0x7f5168160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:32.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.044+0000 7f516ffff700 1 --2- 172.21.15.111:0/3584641756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51680cf010 0x7f5168160b90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5158009840 tx=0x7f5158009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:32.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.044+0000 7f516ffff700 1 -- 172.21.15.111:0/3584641756 shutdown_connections 2023-03-29T10:58:32.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.044+0000 7f516ffff700 1 --2- 172.21.15.111:0/3584641756 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f51680cff50 0x7f51680d2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.044+0000 7f516ffff700 1 --2- 172.21.15.111:0/3584641756 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f51680cf010 0x7f5168160b90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.044+0000 7f516ffff700 1 -- 172.21.15.111:0/3584641756 >> 172.21.15.111:0/3584641756 conn(0x7f5168071b10 msgr2=0x7f5168071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:32.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.044+0000 7f516ffff700 1 -- 172.21.15.111:0/3584641756 shutdown_connections 2023-03-29T10:58:32.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.044+0000 7f516ffff700 1 -- 172.21.15.111:0/3584641756 wait complete. 2023-03-29T10:58:32.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.045+0000 7f516ffff700 1 Processor -- start 2023-03-29T10:58:32.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.046+0000 7f516ffff700 1 -- start start 2023-03-29T10:58:32.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.046+0000 7f516ffff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f51680cf010 0x7f51680b6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:32.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.046+0000 7f516ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51680cff50 0x7f516815b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:32.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.047+0000 7f516ffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f516815c590 0x7f516815de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:32.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.047+0000 7f516ffff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51680b7790 con 0x7f516815c590 2023-03-29T10:58:32.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.047+0000 7f516ffff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51681f20d0 con 0x7f51680cff50 2023-03-29T10:58:32.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.047+0000 7f516ffff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51681f2370 con 0x7f51680cf010 2023-03-29T10:58:32.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.047+0000 7f516e59c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f516815c590 0x7f516815de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:32.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.047+0000 7f516e59c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f516815c590 0x7f516815de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44788/0 (socket says 172.21.15.111:44788) 2023-03-29T10:58:32.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.047+0000 7f516e59c700 1 -- 172.21.15.111:0/1421784181 learned_addr learned my addr 172.21.15.111:0/1421784181 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:32.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.047+0000 7f516dd9b700 1 --2- 172.21.15.111:0/1421784181 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f51680cf010 0x7f51680b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:32.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.047+0000 7f516d59a700 1 --2- 172.21.15.111:0/1421784181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51680cff50 0x7f516815b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:32.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.047+0000 7f516e59c700 1 -- 172.21.15.111:0/1421784181 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f51680cf010 msgr2=0x7f51680b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:32.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.047+0000 7f516e59c700 1 --2- 172.21.15.111:0/1421784181 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f51680cf010 0x7f51680b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.047+0000 7f516e59c700 1 -- 172.21.15.111:0/1421784181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51680cff50 msgr2=0x7f516815b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:32.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.047+0000 7f516e59c700 1 --2- 172.21.15.111:0/1421784181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51680cff50 0x7f516815b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.047+0000 7f516e59c700 1 -- 172.21.15.111:0/1421784181 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51580094a0 con 0x7f516815c590 2023-03-29T10:58:32.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.047+0000 7f516e59c700 1 --2- 172.21.15.111:0/1421784181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f516815c590 0x7f516815de80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5160007e40 tx=0x7f516000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:32.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.048+0000 7f515e7fc700 1 -- 172.21.15.111:0/1421784181 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516000ed30 con 0x7f516815c590 2023-03-29T10:58:32.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.048+0000 7f515e7fc700 1 -- 172.21.15.111:0/1421784181 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5160014420 con 0x7f516815c590 2023-03-29T10:58:32.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.048+0000 7f515e7fc700 1 -- 172.21.15.111:0/1421784181 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51600147d0 con 0x7f516815c590 2023-03-29T10:58:32.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.048+0000 7f516ffff700 1 -- 172.21.15.111:0/1421784181 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51680cd770 con 0x7f516815c590 2023-03-29T10:58:32.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.048+0000 7f516ffff700 1 -- 172.21.15.111:0/1421784181 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51680cf560 con 0x7f516815c590 2023-03-29T10:58:32.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.049+0000 7f516ffff700 1 -- 172.21.15.111:0/1421784181 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516815c960 con 0x7f516815c590 2023-03-29T10:58:32.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.050+0000 7f515e7fc700 1 -- 172.21.15.111:0/1421784181 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f5160014930 con 0x7f516815c590 2023-03-29T10:58:32.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.051+0000 7f515e7fc700 1 --2- 172.21.15.111:0/1421784181 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f514c069de0 0x7f514c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:32.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.051+0000 7f515e7fc700 1 -- 172.21.15.111:0/1421784181 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5160088aa0 con 0x7f516815c590 2023-03-29T10:58:32.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.051+0000 7f516dd9b700 1 --2- 172.21.15.111:0/1421784181 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f514c069de0 0x7f514c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:32.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.052+0000 7f516dd9b700 1 --2- 172.21.15.111:0/1421784181 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f514c069de0 0x7f514c06c290 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5164000f60 tx=0x7f5164007e70 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:32.056 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.055+0000 7f515e7fc700 1 -- 172.21.15.111:0/1421784181 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f5160053fa0 con 0x7f516815c590 2023-03-29T10:58:32.266 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.266+0000 7f516ffff700 1 -- 172.21.15.111:0/1421784181 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f51680b6c50 con 0x7f516815c590 2023-03-29T10:58:32.267 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.266+0000 7f515e7fc700 1 -- 172.21.15.111:0/1421784181 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f51600575c0 con 0x7f516815c590 2023-03-29T10:58:32.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.269+0000 7f516ffff700 1 -- 172.21.15.111:0/1421784181 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f514c069de0 msgr2=0x7f514c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:32.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.269+0000 7f516ffff700 1 --2- 172.21.15.111:0/1421784181 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f514c069de0 0x7f514c06c290 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5164000f60 tx=0x7f5164007e70 comp rx=0 tx=0).stop 2023-03-29T10:58:32.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.269+0000 7f516ffff700 1 -- 172.21.15.111:0/1421784181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f516815c590 msgr2=0x7f516815de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:32.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.269+0000 7f516ffff700 1 --2- 172.21.15.111:0/1421784181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f516815c590 0x7f516815de80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5160007e40 tx=0x7f516000e3c0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.269+0000 7f516ffff700 1 -- 172.21.15.111:0/1421784181 shutdown_connections 2023-03-29T10:58:32.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.269+0000 7f516ffff700 1 --2- 172.21.15.111:0/1421784181 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f514c069de0 0x7f514c06c290 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.269+0000 7f516ffff700 1 --2- 172.21.15.111:0/1421784181 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f516815c590 0x7f516815de80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.269+0000 7f516ffff700 1 --2- 172.21.15.111:0/1421784181 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f51680cf010 0x7f51680b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.269+0000 7f516ffff700 1 --2- 172.21.15.111:0/1421784181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51680cff50 0x7f516815b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.269+0000 7f516ffff700 1 -- 172.21.15.111:0/1421784181 >> 172.21.15.111:0/1421784181 conn(0x7f5168071b10 msgr2=0x7f51680d0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:32.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.269+0000 7f516ffff700 1 -- 172.21.15.111:0/1421784181 shutdown_connections 2023-03-29T10:58:32.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.269+0000 7f516ffff700 1 -- 172.21.15.111:0/1421784181 wait complete. 2023-03-29T10:58:32.270 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-03-29T10:58:32.286 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:58:32.286 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-29T10:58:32.327 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:58:32.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.548+0000 7f0e28bff700 1 Processor -- start 2023-03-29T10:58:32.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.548+0000 7f0e28bff700 1 -- start start 2023-03-29T10:58:32.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.549+0000 7f0e28bff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f0e240d3a30 0x7f0e240b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:32.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.549+0000 7f0e28bff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e240cdc80 con 0x7f0e240cf950 2023-03-29T10:58:32.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.550+0000 7f0e28bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e240cd6c0 con 0x7f0e2415c950 2023-03-29T10:58:32.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.550+0000 7f0e28bff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e240d3e00 con 0x7f0e240d3a30 2023-03-29T10:58:32.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.550+0000 7f0e2259c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f0e240cf950 0x7f0e240cfd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43568/0 (socket says 172.21.15.111:43568) 2023-03-29T10:58:32.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.550+0000 7f0e2259c700 1 -- 172.21.15.111:0/3583816594 learned_addr learned my addr 172.21.15.111:0/3583816594 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:32.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.550+0000 7f0e22d9d700 1 --2- 172.21.15.111:0/3583816594 >> v2:172.21.15.111:3300/0 conn(0x7f0e240d3a30 0x7f0e240b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:32.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.550+0000 7f0e2159a700 1 -- 172.21.15.111:0/3583816594 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828202087 0 0) 0x7f0e240cdc80 con 0x7f0e240cf950 2023-03-29T10:58:32.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.550+0000 7f0e2159a700 1 -- 172.21.15.111:0/3583816594 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e0c0035b0 con 0x7f0e240cf950 2023-03-29T10:58:32.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.550+0000 7f0e22d9d700 1 -- 172.21.15.111:0/3583816594 >> v1:172.21.15.119:6789/0 conn(0x7f0e2415c950 legacy=0x7f0e240cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:32.551 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.550+0000 7f0e22d9d700 1 -- 172.21.15.111:0/3583816594 >> v1:172.21.15.111:6789/0 conn(0x7f0e240cf950 legacy=0x7f0e240cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:32.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.550+0000 7f0e22d9d700 1 -- 172.21.15.111:0/3583816594 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e240cd9a0 con 0x7f0e240d3a30 2023-03-29T10:58:32.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.551+0000 7f0e22d9d700 1 --2- 172.21.15.111:0/3583816594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e240d3a30 0x7f0e240b5d90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009d90 tx=0x7f0e1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45bf76e3232fd4c2 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:32.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.551+0000 7f0e2159a700 1 -- 172.21.15.111:0/3583816594 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c00e070 con 0x7f0e240d3a30 2023-03-29T10:58:32.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.551+0000 7f0e2159a700 1 -- 172.21.15.111:0/3583816594 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c00bb20 con 0x7f0e240d3a30 2023-03-29T10:58:32.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.551+0000 7f0e2159a700 1 -- 172.21.15.111:0/3583816594 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c012430 con 0x7f0e240d3a30 2023-03-29T10:58:32.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.551+0000 7f0e28bff700 1 -- 172.21.15.111:0/3583816594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e240d3a30 msgr2=0x7f0e240b5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:32.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.551+0000 7f0e28bff700 1 --2- 172.21.15.111:0/3583816594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e240d3a30 0x7f0e240b5d90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009d90 tx=0x7f0e1c00b330 comp rx=0 tx=0).stop 2023-03-29T10:58:32.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.551+0000 7f0e28bff700 1 -- 172.21.15.111:0/3583816594 shutdown_connections 2023-03-29T10:58:32.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.551+0000 7f0e28bff700 1 --2- 172.21.15.111:0/3583816594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e240d3a30 0x7f0e240b5d90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.551+0000 7f0e28bff700 1 -- 172.21.15.111:0/3583816594 >> 172.21.15.111:0/3583816594 conn(0x7f0e24071b10 msgr2=0x7f0e24071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:32.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.552+0000 7f0e28bff700 1 -- 172.21.15.111:0/3583816594 shutdown_connections 2023-03-29T10:58:32.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.552+0000 7f0e28bff700 1 -- 172.21.15.111:0/3583816594 wait complete. 2023-03-29T10:58:32.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.553+0000 7f0e28bff700 1 Processor -- start 2023-03-29T10:58:32.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.553+0000 7f0e28bff700 1 -- start start 2023-03-29T10:58:32.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.554+0000 7f0e28bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e240cf950 0x7f0e241f1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:32.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.554+0000 7f0e28bff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e240d3a30 0x7f0e2415f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:32.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.554+0000 7f0e28bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e2415c950 0x7f0e241f2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:32.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.554+0000 7f0e28bff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e241f2e30 con 0x7f0e240d3a30 2023-03-29T10:58:32.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.554+0000 7f0e28bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e241f3b70 con 0x7f0e240cf950 2023-03-29T10:58:32.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.554+0000 7f0e28bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e241f2bb0 con 0x7f0e2415c950 2023-03-29T10:58:32.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.554+0000 7f0e21d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e240d3a30 0x7f0e2415f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:32.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.554+0000 7f0e21d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e240d3a30 0x7f0e2415f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44796/0 (socket says 172.21.15.111:44796) 2023-03-29T10:58:32.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.554+0000 7f0e21d9b700 1 -- 172.21.15.111:0/1901850265 learned_addr learned my addr 172.21.15.111:0/1901850265 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:32.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.555+0000 7f0e2259c700 1 --2- 172.21.15.111:0/1901850265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e240cf950 0x7f0e241f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:32.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.555+0000 7f0e22d9d700 1 --2- 172.21.15.111:0/1901850265 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e2415c950 0x7f0e241f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:32.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.555+0000 7f0e21d9b700 1 -- 172.21.15.111:0/1901850265 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e2415c950 msgr2=0x7f0e241f2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:32.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.555+0000 7f0e21d9b700 1 --2- 172.21.15.111:0/1901850265 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e2415c950 0x7f0e241f2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.555+0000 7f0e21d9b700 1 -- 172.21.15.111:0/1901850265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e240cf950 msgr2=0x7f0e241f1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:32.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.555+0000 7f0e21d9b700 1 --2- 172.21.15.111:0/1901850265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e240cf950 0x7f0e241f1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.555+0000 7f0e21d9b700 1 -- 172.21.15.111:0/1901850265 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1c009a70 con 0x7f0e240d3a30 2023-03-29T10:58:32.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.555+0000 7f0e2259c700 1 --2- 172.21.15.111:0/1901850265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e240cf950 0x7f0e241f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:32.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.555+0000 7f0e22d9d700 1 --2- 172.21.15.111:0/1901850265 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e2415c950 0x7f0e241f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:32.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.555+0000 7f0e21d9b700 1 --2- 172.21.15.111:0/1901850265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e240d3a30 0x7f0e2415f1b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800c510 tx=0x7f0e1800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:32.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.555+0000 7f0e12ffd700 1 -- 172.21.15.111:0/1901850265 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e180052d0 con 0x7f0e240d3a30 2023-03-29T10:58:32.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.555+0000 7f0e12ffd700 1 -- 172.21.15.111:0/1901850265 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e18005430 con 0x7f0e240d3a30 2023-03-29T10:58:32.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.555+0000 7f0e28bff700 1 -- 172.21.15.111:0/1901850265 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e240d19a0 con 0x7f0e240d3a30 2023-03-29T10:58:32.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.555+0000 7f0e12ffd700 1 -- 172.21.15.111:0/1901850265 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e180038c0 con 0x7f0e240d3a30 2023-03-29T10:58:32.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.556+0000 7f0e28bff700 1 -- 172.21.15.111:0/1901850265 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e240d16c0 con 0x7f0e240d3a30 2023-03-29T10:58:32.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.556+0000 7f0e28bff700 1 -- 172.21.15.111:0/1901850265 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e240b62d0 con 0x7f0e240d3a30 2023-03-29T10:58:32.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.557+0000 7f0e12ffd700 1 -- 172.21.15.111:0/1901850265 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f0e18003a20 con 0x7f0e240d3a30 2023-03-29T10:58:32.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.558+0000 7f0e12ffd700 1 --2- 172.21.15.111:0/1901850265 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0e04069e10 0x7f0e0406c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:32.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.558+0000 7f0e12ffd700 1 -- 172.21.15.111:0/1901850265 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0e18087820 con 0x7f0e240d3a30 2023-03-29T10:58:32.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.559+0000 7f0e2259c700 1 --2- 172.21.15.111:0/1901850265 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0e04069e10 0x7f0e0406c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:32.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.559+0000 7f0e2259c700 1 --2- 172.21.15.111:0/1901850265 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0e04069e10 0x7f0e0406c2c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0e140011e0 tx=0x7f0e14009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:32.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.563+0000 7f0e12ffd700 1 -- 172.21.15.111:0/1901850265 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f0e18052d90 con 0x7f0e240d3a30 2023-03-29T10:58:32.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.755+0000 7f0e28bff700 1 -- 172.21.15.111:0/1901850265 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e241f3ce0 con 0x7f0e240d3a30 2023-03-29T10:58:32.757 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.757+0000 7f0e12ffd700 1 -- 172.21.15.111:0/1901850265 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19892 (secure 0 0 0) 0x7f0e180563b0 con 0x7f0e240d3a30 2023-03-29T10:58:32.758 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:32.758 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":30,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:58:09.476334+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.26","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:58:32.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.759+0000 7f0e28bff700 1 -- 172.21.15.111:0/1901850265 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0e04069e10 msgr2=0x7f0e0406c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:32.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.759+0000 7f0e28bff700 1 --2- 172.21.15.111:0/1901850265 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0e04069e10 0x7f0e0406c2c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0e140011e0 tx=0x7f0e14009040 comp rx=0 tx=0).stop 2023-03-29T10:58:32.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.759+0000 7f0e28bff700 1 -- 172.21.15.111:0/1901850265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e240d3a30 msgr2=0x7f0e2415f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:32.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.759+0000 7f0e28bff700 1 --2- 172.21.15.111:0/1901850265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e240d3a30 0x7f0e2415f1b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0e1800c510 tx=0x7f0e1800c8d0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.759+0000 7f0e28bff700 1 -- 172.21.15.111:0/1901850265 shutdown_connections 2023-03-29T10:58:32.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.759+0000 7f0e28bff700 1 --2- 172.21.15.111:0/1901850265 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0e04069e10 0x7f0e0406c2c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.759+0000 7f0e28bff700 1 --2- 172.21.15.111:0/1901850265 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0e2415c950 0x7f0e241f2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.759+0000 7f0e28bff700 1 --2- 172.21.15.111:0/1901850265 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e240cf950 0x7f0e241f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.759+0000 7f0e28bff700 1 --2- 172.21.15.111:0/1901850265 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0e240d3a30 0x7f0e2415f1b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.759+0000 7f0e28bff700 1 -- 172.21.15.111:0/1901850265 >> 172.21.15.111:0/1901850265 conn(0x7f0e24071b10 msgr2=0x7f0e240b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:32.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.759+0000 7f0e28bff700 1 -- 172.21.15.111:0/1901850265 shutdown_connections 2023-03-29T10:58:32.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.759+0000 7f0e28bff700 1 -- 172.21.15.111:0/1901850265 wait complete. 2023-03-29T10:58:32.777 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-29T10:57:16.183979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-29T10:57:21.239809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-29T10:57:26.192124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-29T10:57:26.796777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-29T10:57:29.874795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-29T10:58:32.778 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-29T10:58:32.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.961+0000 7feea83ff700 1 Processor -- start 2023-03-29T10:58:32.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.961+0000 7feea83ff700 1 -- start start 2023-03-29T10:58:32.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.962+0000 7feea83ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feea00b87a0 0x7feea00cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:32.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.962+0000 7feea83ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feea0160d40 0x7feea00d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:32.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.962+0000 7feea83ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea00cdc60 con 0x7feea01601e0 2023-03-29T10:58:32.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.962+0000 7feea83ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea00d02d0 con 0x7feea00b87a0 2023-03-29T10:58:32.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.962+0000 7feea83ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea00d0cb0 con 0x7feea0160d40 2023-03-29T10:58:32.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.963+0000 7feea699c700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feea0160d40 0x7feea00d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:32.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.963+0000 7feea619b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feea00b87a0 0x7feea00cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:32.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.963+0000 7feea599a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7feea01601e0 0x7feea0160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.111:41630/0 (socket says 172.21.15.111:41630) 2023-03-29T10:58:32.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.963+0000 7feea599a700 1 -- 172.21.15.111:0/1234280378 learned_addr learned my addr 172.21.15.111:0/1234280378 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:32.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.963+0000 7feea699c700 1 -- 172.21.15.111:0/1234280378 >> v2:172.21.15.119:3300/0 conn(0x7feea00b87a0 msgr2=0x7feea00cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:32.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.963+0000 7feea699c700 1 --2- 172.21.15.111:0/1234280378 >> v2:172.21.15.119:3300/0 conn(0x7feea00b87a0 0x7feea00cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.963+0000 7feea699c700 1 -- 172.21.15.111:0/1234280378 >> v1:172.21.15.119:6789/0 conn(0x7feea01601e0 legacy=0x7feea0160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:32.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.963+0000 7feea699c700 1 -- 172.21.15.111:0/1234280378 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea00cd6a0 con 0x7feea0160d40 2023-03-29T10:58:32.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.963+0000 7feea619b700 1 --2- 172.21.15.111:0/1234280378 >> v2:172.21.15.119:3300/0 conn(0x7feea00b87a0 0x7feea00cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:58:32.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.964+0000 7feea699c700 1 --2- 172.21.15.111:0/1234280378 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feea0160d40 0x7feea00d1100 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fee98009d90 tx=0x7fee9800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b85cb4f93b6ac6ec server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:32.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.964+0000 7feea5199700 1 -- 172.21.15.111:0/1234280378 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9800e070 con 0x7feea0160d40 2023-03-29T10:58:32.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.964+0000 7feea5199700 1 -- 172.21.15.111:0/1234280378 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9800bb20 con 0x7feea0160d40 2023-03-29T10:58:32.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.964+0000 7feea5199700 1 -- 172.21.15.111:0/1234280378 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee98012430 con 0x7feea0160d40 2023-03-29T10:58:32.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.964+0000 7feea83ff700 1 -- 172.21.15.111:0/1234280378 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feea0160d40 msgr2=0x7feea00d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:32.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.965+0000 7feea83ff700 1 --2- 172.21.15.111:0/1234280378 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feea0160d40 0x7feea00d1100 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fee98009d90 tx=0x7fee9800b330 comp rx=0 tx=0).stop 2023-03-29T10:58:32.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.965+0000 7feea83ff700 1 -- 172.21.15.111:0/1234280378 shutdown_connections 2023-03-29T10:58:32.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.965+0000 7feea83ff700 1 --2- 172.21.15.111:0/1234280378 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feea0160d40 0x7feea00d1100 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.965+0000 7feea83ff700 1 --2- 172.21.15.111:0/1234280378 >> v2:172.21.15.119:3300/0 conn(0x7feea00b87a0 0x7feea00cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.965+0000 7feea83ff700 1 -- 172.21.15.111:0/1234280378 >> 172.21.15.111:0/1234280378 conn(0x7feea0071b10 msgr2=0x7feea0071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:32.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.965+0000 7feea83ff700 1 -- 172.21.15.111:0/1234280378 shutdown_connections 2023-03-29T10:58:32.966 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.965+0000 7feea83ff700 1 -- 172.21.15.111:0/1234280378 wait complete. 2023-03-29T10:58:32.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.966+0000 7feea83ff700 1 Processor -- start 2023-03-29T10:58:32.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.966+0000 7feea83ff700 1 -- start start 2023-03-29T10:58:32.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.967+0000 7feea83ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea00b87a0 0x7feea0155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:32.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.967+0000 7feea83ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feea01601e0 0x7feea0159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:32.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.967+0000 7feea83ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feea0160d40 0x7feea0150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:32.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.967+0000 7feea83ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea0158040 con 0x7feea00b87a0 2023-03-29T10:58:32.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.967+0000 7feea83ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea0157430 con 0x7feea0160d40 2023-03-29T10:58:32.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.967+0000 7feea83ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea0157710 con 0x7feea01601e0 2023-03-29T10:58:32.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.968+0000 7feea619b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea00b87a0 0x7feea0155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:32.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.968+0000 7feea619b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea00b87a0 0x7feea0155c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44798/0 (socket says 172.21.15.111:44798) 2023-03-29T10:58:32.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.968+0000 7feea619b700 1 -- 172.21.15.111:0/2403512869 learned_addr learned my addr 172.21.15.111:0/2403512869 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:32.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.968+0000 7feea599a700 1 --2- 172.21.15.111:0/2403512869 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feea01601e0 0x7feea0159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:32.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.968+0000 7feea699c700 1 --2- 172.21.15.111:0/2403512869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feea0160d40 0x7feea0150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:32.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.968+0000 7feea619b700 1 -- 172.21.15.111:0/2403512869 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feea01601e0 msgr2=0x7feea0159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:32.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.968+0000 7feea619b700 1 --2- 172.21.15.111:0/2403512869 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feea01601e0 0x7feea0159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.968+0000 7feea619b700 1 -- 172.21.15.111:0/2403512869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feea0160d40 msgr2=0x7feea0150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:32.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.968+0000 7feea619b700 1 --2- 172.21.15.111:0/2403512869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feea0160d40 0x7feea0150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:32.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.968+0000 7feea619b700 1 -- 172.21.15.111:0/2403512869 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c0094a0 con 0x7feea00b87a0 2023-03-29T10:58:32.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.968+0000 7feea599a700 1 --2- 172.21.15.111:0/2403512869 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feea01601e0 0x7feea0159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:32.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.968+0000 7feea619b700 1 --2- 172.21.15.111:0/2403512869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea00b87a0 0x7feea0155c70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00ebe0 tx=0x7fee9c009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:32.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.968+0000 7fee96ffd700 1 -- 172.21.15.111:0/2403512869 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00c910 con 0x7feea00b87a0 2023-03-29T10:58:32.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.969+0000 7fee96ffd700 1 -- 172.21.15.111:0/2403512869 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9c00ca70 con 0x7feea00b87a0 2023-03-29T10:58:32.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.969+0000 7feea83ff700 1 -- 172.21.15.111:0/2403512869 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee98009a70 con 0x7feea00b87a0 2023-03-29T10:58:32.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.969+0000 7fee96ffd700 1 -- 172.21.15.111:0/2403512869 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00cde0 con 0x7feea00b87a0 2023-03-29T10:58:32.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.969+0000 7feea83ff700 1 -- 172.21.15.111:0/2403512869 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea00d1640 con 0x7feea00b87a0 2023-03-29T10:58:32.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.970+0000 7feea83ff700 1 -- 172.21.15.111:0/2403512869 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea00cd980 con 0x7feea00b87a0 2023-03-29T10:58:32.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.971+0000 7fee96ffd700 1 -- 172.21.15.111:0/2403512869 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fee9c013470 con 0x7feea00b87a0 2023-03-29T10:58:32.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.972+0000 7fee96ffd700 1 --2- 172.21.15.111:0/2403512869 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fee84069de0 0x7fee8406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:32.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.972+0000 7fee96ffd700 1 -- 172.21.15.111:0/2403512869 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fee9c088420 con 0x7feea00b87a0 2023-03-29T10:58:32.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.972+0000 7feea599a700 1 --2- 172.21.15.111:0/2403512869 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fee84069de0 0x7fee8406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:32.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.972+0000 7feea599a700 1 --2- 172.21.15.111:0/2403512869 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fee84069de0 0x7fee8406c290 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fee90007fd0 tx=0x7fee90009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:32.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:32.976+0000 7fee96ffd700 1 -- 172.21.15.111:0/2403512869 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fee9c053a30 con 0x7feea00b87a0 2023-03-29T10:58:33.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.168+0000 7feea83ff700 1 -- 172.21.15.111:0/2403512869 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feea015cce0 con 0x7feea00b87a0 2023-03-29T10:58:33.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.169+0000 7fee96ffd700 1 -- 172.21.15.111:0/2403512869 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee9c057050 con 0x7feea00b87a0 2023-03-29T10:58:33.170 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-03-29T10:58:33.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.171+0000 7feea83ff700 1 -- 172.21.15.111:0/2403512869 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fee84069de0 msgr2=0x7fee8406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:33.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.171+0000 7feea83ff700 1 --2- 172.21.15.111:0/2403512869 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fee84069de0 0x7fee8406c290 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fee90007fd0 tx=0x7fee90009040 comp rx=0 tx=0).stop 2023-03-29T10:58:33.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.171+0000 7feea83ff700 1 -- 172.21.15.111:0/2403512869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea00b87a0 msgr2=0x7feea0155c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:33.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.171+0000 7feea83ff700 1 --2- 172.21.15.111:0/2403512869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea00b87a0 0x7feea0155c70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00ebe0 tx=0x7fee9c009ec0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.171+0000 7feea83ff700 1 -- 172.21.15.111:0/2403512869 shutdown_connections 2023-03-29T10:58:33.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.171+0000 7feea83ff700 1 --2- 172.21.15.111:0/2403512869 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fee84069de0 0x7fee8406c290 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.171+0000 7feea83ff700 1 --2- 172.21.15.111:0/2403512869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7feea00b87a0 0x7feea0155c70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.171+0000 7feea83ff700 1 --2- 172.21.15.111:0/2403512869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feea0160d40 0x7feea0150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.171+0000 7feea83ff700 1 --2- 172.21.15.111:0/2403512869 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7feea01601e0 0x7feea0159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.171+0000 7feea83ff700 1 -- 172.21.15.111:0/2403512869 >> 172.21.15.111:0/2403512869 conn(0x7feea0071b10 msgr2=0x7feea00cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:33.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.171+0000 7feea83ff700 1 -- 172.21.15.111:0/2403512869 shutdown_connections 2023-03-29T10:58:33.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.171+0000 7feea83ff700 1 -- 172.21.15.111:0/2403512869 wait complete. 2023-03-29T10:58:33.190 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-29T10:58:33.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.374+0000 7fd31adff700 1 Processor -- start 2023-03-29T10:58:33.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.374+0000 7fd31adff700 1 -- start start 2023-03-29T10:58:33.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.375+0000 7fd31adff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd3140cd520 0x7fd3140cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:33.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.375+0000 7fd31adff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd31415d660 con 0x7fd31415a140 2023-03-29T10:58:33.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.376+0000 7fd31adff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd31415cd70 con 0x7fd3140cfe20 2023-03-29T10:58:33.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.376+0000 7fd31adff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3140d02a0 con 0x7fd3140cd520 2023-03-29T10:58:33.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.376+0000 7fd318b9b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fd31415a140 0x7fd314160ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43578/0 (socket says 172.21.15.111:43578) 2023-03-29T10:58:33.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.376+0000 7fd318b9b700 1 -- 172.21.15.111:0/3970199003 learned_addr learned my addr 172.21.15.111:0/3970199003 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:33.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.376+0000 7fd313fff700 1 --2- 172.21.15.111:0/3970199003 >> v2:172.21.15.119:3300/0 conn(0x7fd3140cd520 0x7fd3140cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:33.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.376+0000 7fd3137fe700 1 -- 172.21.15.111:0/3970199003 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894939462 0 0) 0x7fd31415d660 con 0x7fd31415a140 2023-03-29T10:58:33.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.376+0000 7fd3137fe700 1 -- 172.21.15.111:0/3970199003 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3040035b0 con 0x7fd31415a140 2023-03-29T10:58:33.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.376+0000 7fd3137fe700 1 -- 172.21.15.111:0/3970199003 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2615121448 0 0) 0x7fd31415cd70 con 0x7fd3140cfe20 2023-03-29T10:58:33.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.376+0000 7fd3137fe700 1 -- 172.21.15.111:0/3970199003 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd31415d660 con 0x7fd3140cfe20 2023-03-29T10:58:33.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.376+0000 7fd313fff700 1 -- 172.21.15.111:0/3970199003 >> v1:172.21.15.119:6789/0 conn(0x7fd3140cfe20 legacy=0x7fd3140b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:33.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.376+0000 7fd313fff700 1 -- 172.21.15.111:0/3970199003 >> v1:172.21.15.111:6789/0 conn(0x7fd31415a140 legacy=0x7fd314160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:33.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.376+0000 7fd313fff700 1 -- 172.21.15.111:0/3970199003 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd31415d010 con 0x7fd3140cd520 2023-03-29T10:58:33.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.377+0000 7fd313fff700 1 --2- 172.21.15.111:0/3970199003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3140cd520 0x7fd3140cf8e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd300009840 tx=0x7fd300009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bfacc0d4cebfde2 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:33.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.377+0000 7fd3137fe700 1 -- 172.21.15.111:0/3970199003 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300011830 con 0x7fd3140cd520 2023-03-29T10:58:33.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.377+0000 7fd3137fe700 1 -- 172.21.15.111:0/3970199003 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd300011990 con 0x7fd3140cd520 2023-03-29T10:58:33.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.377+0000 7fd3137fe700 1 -- 172.21.15.111:0/3970199003 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300011d20 con 0x7fd3140cd520 2023-03-29T10:58:33.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.377+0000 7fd31adff700 1 -- 172.21.15.111:0/3970199003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3140cd520 msgr2=0x7fd3140cf8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:33.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.377+0000 7fd31adff700 1 --2- 172.21.15.111:0/3970199003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3140cd520 0x7fd3140cf8e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd300009840 tx=0x7fd300009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:33.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.377+0000 7fd31adff700 1 -- 172.21.15.111:0/3970199003 shutdown_connections 2023-03-29T10:58:33.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.378+0000 7fd31adff700 1 --2- 172.21.15.111:0/3970199003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3140cd520 0x7fd3140cf8e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.378+0000 7fd31adff700 1 -- 172.21.15.111:0/3970199003 >> 172.21.15.111:0/3970199003 conn(0x7fd314071b10 msgr2=0x7fd314071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:33.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.378+0000 7fd31adff700 1 -- 172.21.15.111:0/3970199003 shutdown_connections 2023-03-29T10:58:33.379 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.378+0000 7fd31adff700 1 -- 172.21.15.111:0/3970199003 wait complete. 2023-03-29T10:58:33.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.380+0000 7fd31adff700 1 Processor -- start 2023-03-29T10:58:33.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.380+0000 7fd31adff700 1 -- start start 2023-03-29T10:58:33.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.381+0000 7fd31adff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3140cd520 0x7fd3140c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:33.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.381+0000 7fd31adff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3140cfe20 0x7fd3140b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:33.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.381+0000 7fd31adff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd31415a140 0x7fd3140ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:33.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.381+0000 7fd31adff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3141e7f70 con 0x7fd3140cfe20 2023-03-29T10:58:33.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.381+0000 7fd31adff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3141e64e0 con 0x7fd3140cd520 2023-03-29T10:58:33.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.381+0000 7fd31adff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3141e67c0 con 0x7fd31415a140 2023-03-29T10:58:33.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.381+0000 7fd313fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3140cfe20 0x7fd3140b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:33.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.381+0000 7fd313fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3140cfe20 0x7fd3140b7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44804/0 (socket says 172.21.15.111:44804) 2023-03-29T10:58:33.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.381+0000 7fd313fff700 1 -- 172.21.15.111:0/1231364389 learned_addr learned my addr 172.21.15.111:0/1231364389 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:33.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.382+0000 7fd318b9b700 1 --2- 172.21.15.111:0/1231364389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3140cd520 0x7fd3140c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:33.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.382+0000 7fd31939c700 1 --2- 172.21.15.111:0/1231364389 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd31415a140 0x7fd3140ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:33.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.382+0000 7fd313fff700 1 -- 172.21.15.111:0/1231364389 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd31415a140 msgr2=0x7fd3140ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:33.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.382+0000 7fd313fff700 1 --2- 172.21.15.111:0/1231364389 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd31415a140 0x7fd3140ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.382+0000 7fd313fff700 1 -- 172.21.15.111:0/1231364389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3140cd520 msgr2=0x7fd3140c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:33.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.382+0000 7fd313fff700 1 --2- 172.21.15.111:0/1231364389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3140cd520 0x7fd3140c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.382+0000 7fd313fff700 1 -- 172.21.15.111:0/1231364389 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3000094a0 con 0x7fd3140cfe20 2023-03-29T10:58:33.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.382+0000 7fd31939c700 1 --2- 172.21.15.111:0/1231364389 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd31415a140 0x7fd3140ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:33.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.382+0000 7fd318b9b700 1 --2- 172.21.15.111:0/1231364389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3140cd520 0x7fd3140c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:33.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.382+0000 7fd313fff700 1 --2- 172.21.15.111:0/1231364389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3140cfe20 0x7fd3140b7790 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd300009fd0 tx=0x7fd300004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:33.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.382+0000 7fd3117fa700 1 -- 172.21.15.111:0/1231364389 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000045d0 con 0x7fd3140cfe20 2023-03-29T10:58:33.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.382+0000 7fd3117fa700 1 -- 172.21.15.111:0/1231364389 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30001f070 con 0x7fd3140cfe20 2023-03-29T10:58:33.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.382+0000 7fd31adff700 1 -- 172.21.15.111:0/1231364389 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd31415daf0 con 0x7fd3140cfe20 2023-03-29T10:58:33.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.382+0000 7fd3117fa700 1 -- 172.21.15.111:0/1231364389 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000038c0 con 0x7fd3140cfe20 2023-03-29T10:58:33.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.383+0000 7fd31adff700 1 -- 172.21.15.111:0/1231364389 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd31415d2f0 con 0x7fd3140cfe20 2023-03-29T10:58:33.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.383+0000 7fd31adff700 1 -- 172.21.15.111:0/1231364389 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3140cdb00 con 0x7fd3140cfe20 2023-03-29T10:58:33.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.384+0000 7fd3117fa700 1 -- 172.21.15.111:0/1231364389 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fd300003a20 con 0x7fd3140cfe20 2023-03-29T10:58:33.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.385+0000 7fd3117fa700 1 --2- 172.21.15.111:0/1231364389 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd2f8069de0 0x7fd2f806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:33.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.385+0000 7fd3117fa700 1 -- 172.21.15.111:0/1231364389 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd300021070 con 0x7fd3140cfe20 2023-03-29T10:58:33.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.385+0000 7fd318b9b700 1 --2- 172.21.15.111:0/1231364389 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd2f8069de0 0x7fd2f806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:33.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.386+0000 7fd318b9b700 1 --2- 172.21.15.111:0/1231364389 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd2f8069de0 0x7fd2f806c290 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd308000f60 tx=0x7fd308009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:33.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.389+0000 7fd3117fa700 1 -- 172.21.15.111:0/1231364389 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fd300055250 con 0x7fd3140cfe20 2023-03-29T10:58:33.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.583+0000 7fd31adff700 1 -- 172.21.15.111:0/1231364389 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd3141e80b0 con 0x7fd3140cfe20 2023-03-29T10:58:33.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.583+0000 7fd3117fa700 1 -- 172.21.15.111:0/1231364389 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd3000150a0 con 0x7fd3140cfe20 2023-03-29T10:58:33.583 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 8 2023-03-29T10:58:33.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.586+0000 7fd31adff700 1 -- 172.21.15.111:0/1231364389 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd2f8069de0 msgr2=0x7fd2f806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:33.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.586+0000 7fd31adff700 1 --2- 172.21.15.111:0/1231364389 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd2f8069de0 0x7fd2f806c290 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd308000f60 tx=0x7fd308009040 comp rx=0 tx=0).stop 2023-03-29T10:58:33.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.586+0000 7fd31adff700 1 -- 172.21.15.111:0/1231364389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3140cfe20 msgr2=0x7fd3140b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:33.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.586+0000 7fd31adff700 1 --2- 172.21.15.111:0/1231364389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3140cfe20 0x7fd3140b7790 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd300009fd0 tx=0x7fd300004750 comp rx=0 tx=0).stop 2023-03-29T10:58:33.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.586+0000 7fd31adff700 1 -- 172.21.15.111:0/1231364389 shutdown_connections 2023-03-29T10:58:33.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.586+0000 7fd31adff700 1 --2- 172.21.15.111:0/1231364389 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd2f8069de0 0x7fd2f806c290 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.586+0000 7fd31adff700 1 --2- 172.21.15.111:0/1231364389 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd31415a140 0x7fd3140ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.586+0000 7fd31adff700 1 --2- 172.21.15.111:0/1231364389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3140cd520 0x7fd3140c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.586+0000 7fd31adff700 1 --2- 172.21.15.111:0/1231364389 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd3140cfe20 0x7fd3140b7790 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.586+0000 7fd31adff700 1 -- 172.21.15.111:0/1231364389 >> 172.21.15.111:0/1231364389 conn(0x7fd314071b10 msgr2=0x7fd3140cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:33.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.586+0000 7fd31adff700 1 -- 172.21.15.111:0/1231364389 shutdown_connections 2023-03-29T10:58:33.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.586+0000 7fd31adff700 1 -- 172.21.15.111:0/1231364389 wait complete. 2023-03-29T10:58:33.603 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-29T10:58:33.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.788+0000 7fd8ef3ff700 1 Processor -- start 2023-03-29T10:58:33.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.788+0000 7fd8ef3ff700 1 -- start start 2023-03-29T10:58:33.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.789+0000 7fd8ef3ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd8e8158da0 0x7fd8e806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:33.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.789+0000 7fd8ef3ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd8e806cc00 0x7fd8e806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:33.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.789+0000 7fd8ef3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8e80ca890 con 0x7fd8e81589d0 2023-03-29T10:58:33.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.789+0000 7fd8ef3ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e80cd860 con 0x7fd8e8158da0 2023-03-29T10:58:33.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.789+0000 7fd8ef3ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8e80ca5b0 con 0x7fd8e806cc00 2023-03-29T10:58:33.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.790+0000 7fd8ec99a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd8e8158da0 0x7fd8e806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:33.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.790+0000 7fd8ec99a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd8e8158da0 0x7fd8e806c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44816/0 (socket says 172.21.15.111:44816) 2023-03-29T10:58:33.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.790+0000 7fd8ec99a700 1 -- 172.21.15.111:0/2464453295 learned_addr learned my addr 172.21.15.111:0/2464453295 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:33.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.790+0000 7fd8ed99c700 1 --2- 172.21.15.111:0/2464453295 >> v2:172.21.15.119:3300/0 conn(0x7fd8e806cc00 0x7fd8e806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:33.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.790+0000 7fd8ec99a700 1 -- 172.21.15.111:0/2464453295 >> v2:172.21.15.119:3300/0 conn(0x7fd8e806cc00 msgr2=0x7fd8e806a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:33.790 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.790+0000 7fd8ec99a700 1 --2- 172.21.15.111:0/2464453295 >> v2:172.21.15.119:3300/0 conn(0x7fd8e806cc00 0x7fd8e806a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.790+0000 7fd8ec99a700 1 -- 172.21.15.111:0/2464453295 >> v1:172.21.15.119:6789/0 conn(0x7fd8e81589d0 legacy=0x7fd8e80cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.790+0000 7fd8ec99a700 1 -- 172.21.15.111:0/2464453295 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e80cc670 con 0x7fd8e8158da0 2023-03-29T10:58:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.790+0000 7fd8ec99a700 1 --2- 172.21.15.111:0/2464453295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8e8158da0 0x7fd8e806c6c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd8e806b370 tx=0x7fd8d800ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a5c0d06b96ceb6 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.791+0000 7fd8dffff700 1 -- 172.21.15.111:0/2464453295 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d8005e30 con 0x7fd8e8158da0 2023-03-29T10:58:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.791+0000 7fd8dffff700 1 -- 172.21.15.111:0/2464453295 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d8004030 con 0x7fd8e8158da0 2023-03-29T10:58:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.791+0000 7fd8dffff700 1 -- 172.21.15.111:0/2464453295 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d80043e0 con 0x7fd8e8158da0 2023-03-29T10:58:33.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.791+0000 7fd8ef3ff700 1 -- 172.21.15.111:0/2464453295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8e8158da0 msgr2=0x7fd8e806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.791+0000 7fd8ef3ff700 1 --2- 172.21.15.111:0/2464453295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8e8158da0 0x7fd8e806c6c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd8e806b370 tx=0x7fd8d800ad70 comp rx=0 tx=0).stop 2023-03-29T10:58:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.791+0000 7fd8ef3ff700 1 -- 172.21.15.111:0/2464453295 shutdown_connections 2023-03-29T10:58:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.791+0000 7fd8ef3ff700 1 --2- 172.21.15.111:0/2464453295 >> v2:172.21.15.119:3300/0 conn(0x7fd8e806cc00 0x7fd8e806a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.791+0000 7fd8ef3ff700 1 --2- 172.21.15.111:0/2464453295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8e8158da0 0x7fd8e806c6c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.791+0000 7fd8ef3ff700 1 -- 172.21.15.111:0/2464453295 >> 172.21.15.111:0/2464453295 conn(0x7fd8e80cf720 msgr2=0x7fd8e80cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.791+0000 7fd8ef3ff700 1 -- 172.21.15.111:0/2464453295 shutdown_connections 2023-03-29T10:58:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.791+0000 7fd8ef3ff700 1 -- 172.21.15.111:0/2464453295 wait complete. 2023-03-29T10:58:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.793+0000 7fd8ef3ff700 1 Processor -- start 2023-03-29T10:58:33.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.793+0000 7fd8ef3ff700 1 -- start start 2023-03-29T10:58:33.794 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.794+0000 7fd8ef3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e806cc00 0x7fd8e8159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:33.794 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.794+0000 7fd8ef3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8e81589d0 0x7fd8e81f2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:33.794 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.794+0000 7fd8ef3ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8e8158da0 0x7fd8e80c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:33.794 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.794+0000 7fd8ef3ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e80c2530 con 0x7fd8e8158da0 2023-03-29T10:58:33.794 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.794+0000 7fd8ef3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e815a290 con 0x7fd8e806cc00 2023-03-29T10:58:33.794 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.794+0000 7fd8ef3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e815a570 con 0x7fd8e81589d0 2023-03-29T10:58:33.794 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.794+0000 7fd8ed99c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8e8158da0 0x7fd8e80c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:33.794 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.794+0000 7fd8ed99c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8e8158da0 0x7fd8e80c4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44826/0 (socket says 172.21.15.111:44826) 2023-03-29T10:58:33.794 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.794+0000 7fd8ed99c700 1 -- 172.21.15.111:0/1056294870 learned_addr learned my addr 172.21.15.111:0/1056294870 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:33.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.795+0000 7fd8ed19b700 1 --2- 172.21.15.111:0/1056294870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e806cc00 0x7fd8e8159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:33.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.795+0000 7fd8ed99c700 1 -- 172.21.15.111:0/1056294870 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8e81589d0 msgr2=0x7fd8e81f2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:33.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.795+0000 7fd8ed99c700 1 --2- 172.21.15.111:0/1056294870 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8e81589d0 0x7fd8e81f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.795+0000 7fd8ed99c700 1 -- 172.21.15.111:0/1056294870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e806cc00 msgr2=0x7fd8e8159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:33.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.795+0000 7fd8ed99c700 1 --2- 172.21.15.111:0/1056294870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e806cc00 0x7fd8e8159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.795 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.795+0000 7fd8ed99c700 1 -- 172.21.15.111:0/1056294870 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d800a6a0 con 0x7fd8e8158da0 2023-03-29T10:58:33.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.795+0000 7fd8ed19b700 1 --2- 172.21.15.111:0/1056294870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e806cc00 0x7fd8e8159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:33.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.795+0000 7fd8ed99c700 1 --2- 172.21.15.111:0/1056294870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8e8158da0 0x7fd8e80c4510 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0007e40 tx=0x7fd8e000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:33.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.795+0000 7fd8ddffb700 1 -- 172.21.15.111:0/1056294870 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e000eea0 con 0x7fd8e8158da0 2023-03-29T10:58:33.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.795+0000 7fd8ddffb700 1 -- 172.21.15.111:0/1056294870 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e0014420 con 0x7fd8e8158da0 2023-03-29T10:58:33.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.795+0000 7fd8ef3ff700 1 -- 172.21.15.111:0/1056294870 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8e80ccc30 con 0x7fd8e8158da0 2023-03-29T10:58:33.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.795+0000 7fd8ddffb700 1 -- 172.21.15.111:0/1056294870 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e00147b0 con 0x7fd8e8158da0 2023-03-29T10:58:33.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.795+0000 7fd8ef3ff700 1 -- 172.21.15.111:0/1056294870 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8e80cc950 con 0x7fd8e8158da0 2023-03-29T10:58:33.796 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.796+0000 7fd8ef3ff700 1 -- 172.21.15.111:0/1056294870 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8e806a920 con 0x7fd8e8158da0 2023-03-29T10:58:33.797 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.797+0000 7fd8ddffb700 1 -- 172.21.15.111:0/1056294870 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fd8e0014910 con 0x7fd8e8158da0 2023-03-29T10:58:33.798 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.798+0000 7fd8ddffb700 1 --2- 172.21.15.111:0/1056294870 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd8cc069de0 0x7fd8cc06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:33.798 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.798+0000 7fd8ddffb700 1 -- 172.21.15.111:0/1056294870 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd8e0088b00 con 0x7fd8e8158da0 2023-03-29T10:58:33.798 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.798+0000 7fd8ed19b700 1 --2- 172.21.15.111:0/1056294870 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd8cc069de0 0x7fd8cc06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:33.798 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.799+0000 7fd8ed19b700 1 --2- 172.21.15.111:0/1056294870 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd8cc069de0 0x7fd8cc06c290 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4000f60 tx=0x7fd8e4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:33.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.802+0000 7fd8ddffb700 1 -- 172.21.15.111:0/1056294870 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fd8e008ae60 con 0x7fd8e8158da0 2023-03-29T10:58:33.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.995+0000 7fd8ef3ff700 1 -- 172.21.15.111:0/1056294870 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd8e80c2670 con 0x7fd8e8158da0 2023-03-29T10:58:33.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.996+0000 7fd8ddffb700 1 -- 172.21.15.111:0/1056294870 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd8e80c2670 con 0x7fd8e8158da0 2023-03-29T10:58:33.995 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:33.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.998+0000 7fd8ef3ff700 1 -- 172.21.15.111:0/1056294870 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd8cc069de0 msgr2=0x7fd8cc06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:33.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.998+0000 7fd8ef3ff700 1 --2- 172.21.15.111:0/1056294870 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd8cc069de0 0x7fd8cc06c290 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd8e4000f60 tx=0x7fd8e4009040 comp rx=0 tx=0).stop 2023-03-29T10:58:33.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.998+0000 7fd8ef3ff700 1 -- 172.21.15.111:0/1056294870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8e8158da0 msgr2=0x7fd8e80c4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:33.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.998+0000 7fd8ef3ff700 1 --2- 172.21.15.111:0/1056294870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8e8158da0 0x7fd8e80c4510 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0007e40 tx=0x7fd8e000e3c0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.998+0000 7fd8ef3ff700 1 -- 172.21.15.111:0/1056294870 shutdown_connections 2023-03-29T10:58:33.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.998+0000 7fd8ef3ff700 1 --2- 172.21.15.111:0/1056294870 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd8cc069de0 0x7fd8cc06c290 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.998+0000 7fd8ef3ff700 1 --2- 172.21.15.111:0/1056294870 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd8e8158da0 0x7fd8e80c4510 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.998+0000 7fd8ef3ff700 1 --2- 172.21.15.111:0/1056294870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8e806cc00 0x7fd8e8159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.998+0000 7fd8ef3ff700 1 --2- 172.21.15.111:0/1056294870 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd8e81589d0 0x7fd8e81f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:33.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.998+0000 7fd8ef3ff700 1 -- 172.21.15.111:0/1056294870 >> 172.21.15.111:0/1056294870 conn(0x7fd8e80cf720 msgr2=0x7fd8e8070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:33.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.998+0000 7fd8ef3ff700 1 -- 172.21.15.111:0/1056294870 shutdown_connections 2023-03-29T10:58:33.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:33.998+0000 7fd8ef3ff700 1 -- 172.21.15.111:0/1056294870 wait complete. 2023-03-29T10:58:34.015 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-29T10:58:34.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.203+0000 7f3f3afff700 1 Processor -- start 2023-03-29T10:58:34.202 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.203+0000 7f3f3afff700 1 -- start start 2023-03-29T10:58:34.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.204+0000 7f3f3afff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3f34158da0 0x7f3f3406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:34.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.204+0000 7f3f3afff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f3406cc00 0x7f3f3406a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:34.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.204+0000 7f3f3afff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f340ca890 con 0x7f3f341589d0 2023-03-29T10:58:34.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.204+0000 7f3f3afff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f340cd860 con 0x7f3f34158da0 2023-03-29T10:58:34.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.204+0000 7f3f3afff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f340ca5b0 con 0x7f3f3406cc00 2023-03-29T10:58:34.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.204+0000 7f3f33fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3f34158da0 0x7f3f3406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:34.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.204+0000 7f3f33fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3f34158da0 0x7f3f3406c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44828/0 (socket says 172.21.15.111:44828) 2023-03-29T10:58:34.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.204+0000 7f3f33fff700 1 -- 172.21.15.111:0/1021204706 learned_addr learned my addr 172.21.15.111:0/1021204706 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:34.204 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.204+0000 7f3f3959c700 1 --2- 172.21.15.111:0/1021204706 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f3406cc00 0x7f3f3406a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:34.205 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.205+0000 7f3f33fff700 1 -- 172.21.15.111:0/1021204706 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f3406cc00 msgr2=0x7f3f3406a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:34.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.205+0000 7f3f33fff700 1 --2- 172.21.15.111:0/1021204706 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f3406cc00 0x7f3f3406a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:34.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.205+0000 7f3f33fff700 1 -- 172.21.15.111:0/1021204706 >> v1:172.21.15.119:6789/0 conn(0x7f3f341589d0 legacy=0x7f3f340cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:34.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.205+0000 7f3f33fff700 1 -- 172.21.15.111:0/1021204706 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f340cc670 con 0x7f3f34158da0 2023-03-29T10:58:34.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.205+0000 7f3f33fff700 1 --2- 172.21.15.111:0/1021204706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f34158da0 0x7f3f3406c6c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3f20006fd0 tx=0x7f3f2000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ad9537c58b3a8c server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:34.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.205+0000 7f3f337fe700 1 -- 172.21.15.111:0/1021204706 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f20005e30 con 0x7f3f34158da0 2023-03-29T10:58:34.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.205+0000 7f3f337fe700 1 -- 172.21.15.111:0/1021204706 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f20004030 con 0x7f3f34158da0 2023-03-29T10:58:34.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.205+0000 7f3f337fe700 1 -- 172.21.15.111:0/1021204706 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f200043e0 con 0x7f3f34158da0 2023-03-29T10:58:34.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.206+0000 7f3f3afff700 1 -- 172.21.15.111:0/1021204706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f34158da0 msgr2=0x7f3f3406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:34.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.206+0000 7f3f3afff700 1 --2- 172.21.15.111:0/1021204706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f34158da0 0x7f3f3406c6c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3f20006fd0 tx=0x7f3f2000be70 comp rx=0 tx=0).stop 2023-03-29T10:58:34.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.206+0000 7f3f3afff700 1 -- 172.21.15.111:0/1021204706 shutdown_connections 2023-03-29T10:58:34.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.206+0000 7f3f3afff700 1 --2- 172.21.15.111:0/1021204706 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f3406cc00 0x7f3f3406a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:34.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.206+0000 7f3f3afff700 1 --2- 172.21.15.111:0/1021204706 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f34158da0 0x7f3f3406c6c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:34.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.206+0000 7f3f3afff700 1 -- 172.21.15.111:0/1021204706 >> 172.21.15.111:0/1021204706 conn(0x7f3f340cf720 msgr2=0x7f3f340cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:34.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.206+0000 7f3f3afff700 1 -- 172.21.15.111:0/1021204706 shutdown_connections 2023-03-29T10:58:34.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.206+0000 7f3f3afff700 1 -- 172.21.15.111:0/1021204706 wait complete. 2023-03-29T10:58:34.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.208+0000 7f3f3afff700 1 Processor -- start 2023-03-29T10:58:34.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.208+0000 7f3f3afff700 1 -- start start 2023-03-29T10:58:34.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.208+0000 7f3f3afff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f3406cc00 0x7f3f3415bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:34.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.209+0000 7f3f3afff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f341589d0 0x7f3f340c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:34.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.209+0000 7f3f3afff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f34158da0 0x7f3f341574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:34.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.209+0000 7f3f3afff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3415c0e0 con 0x7f3f34158da0 2023-03-29T10:58:34.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.209+0000 7f3f3afff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f340c81e0 con 0x7f3f341589d0 2023-03-29T10:58:34.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.209+0000 7f3f3afff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f340c7330 con 0x7f3f3406cc00 2023-03-29T10:58:34.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.209+0000 7f3f3959c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f34158da0 0x7f3f341574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:34.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.209+0000 7f3f3959c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f34158da0 0x7f3f341574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44844/0 (socket says 172.21.15.111:44844) 2023-03-29T10:58:34.209 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.209+0000 7f3f3959c700 1 -- 172.21.15.111:0/1368802254 learned_addr learned my addr 172.21.15.111:0/1368802254 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:34.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.209+0000 7f3f33fff700 1 --2- 172.21.15.111:0/1368802254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f341589d0 0x7f3f340c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:34.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.209+0000 7f3f38d9b700 1 --2- 172.21.15.111:0/1368802254 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f3406cc00 0x7f3f3415bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:34.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.209+0000 7f3f3959c700 1 -- 172.21.15.111:0/1368802254 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f3406cc00 msgr2=0x7f3f3415bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:34.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.209+0000 7f3f3959c700 1 --2- 172.21.15.111:0/1368802254 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f3406cc00 0x7f3f3415bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:34.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.209+0000 7f3f3959c700 1 -- 172.21.15.111:0/1368802254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f341589d0 msgr2=0x7f3f340c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:34.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.209+0000 7f3f3959c700 1 --2- 172.21.15.111:0/1368802254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f341589d0 0x7f3f340c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:34.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.209+0000 7f3f3959c700 1 -- 172.21.15.111:0/1368802254 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2000b7a0 con 0x7f3f34158da0 2023-03-29T10:58:34.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.209+0000 7f3f33fff700 1 --2- 172.21.15.111:0/1368802254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f341589d0 0x7f3f340c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:34.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.209+0000 7f3f38d9b700 1 --2- 172.21.15.111:0/1368802254 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f3406cc00 0x7f3f3415bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:34.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.209+0000 7f3f3959c700 1 --2- 172.21.15.111:0/1368802254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f34158da0 0x7f3f341574e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c007e70 tx=0x7f3f2c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:34.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.210+0000 7f3f317fa700 1 -- 172.21.15.111:0/1368802254 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c00ee50 con 0x7f3f34158da0 2023-03-29T10:58:34.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.210+0000 7f3f317fa700 1 -- 172.21.15.111:0/1368802254 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f2c014420 con 0x7f3f34158da0 2023-03-29T10:58:34.210 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.210+0000 7f3f317fa700 1 -- 172.21.15.111:0/1368802254 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c0147d0 con 0x7f3f34158da0 2023-03-29T10:58:34.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.210+0000 7f3f3afff700 1 -- 172.21.15.111:0/1368802254 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f340ccc30 con 0x7f3f34158da0 2023-03-29T10:58:34.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.210+0000 7f3f3afff700 1 -- 172.21.15.111:0/1368802254 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f340cc950 con 0x7f3f34158da0 2023-03-29T10:58:34.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.211+0000 7f3f3afff700 1 -- 172.21.15.111:0/1368802254 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3406a920 con 0x7f3f34158da0 2023-03-29T10:58:34.211 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.212+0000 7f3f317fa700 1 -- 172.21.15.111:0/1368802254 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f3f2c014930 con 0x7f3f34158da0 2023-03-29T10:58:34.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.213+0000 7f3f317fa700 1 --2- 172.21.15.111:0/1368802254 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3f18069de0 0x7f3f1806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:34.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.213+0000 7f3f317fa700 1 -- 172.21.15.111:0/1368802254 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3f2c088ab0 con 0x7f3f34158da0 2023-03-29T10:58:34.212 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.213+0000 7f3f38d9b700 1 --2- 172.21.15.111:0/1368802254 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3f18069de0 0x7f3f1806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:34.213 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.214+0000 7f3f38d9b700 1 --2- 172.21.15.111:0/1368802254 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3f18069de0 0x7f3f1806c290 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3f28000f60 tx=0x7f3f28009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:34.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.217+0000 7f3f317fa700 1 -- 172.21.15.111:0/1368802254 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f3f2c057a60 con 0x7f3f34158da0 2023-03-29T10:58:34.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.409+0000 7f3f3afff700 1 -- 172.21.15.111:0/1368802254 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3f340b4a10 con 0x7f3f34158da0 2023-03-29T10:58:34.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.409+0000 7f3f317fa700 1 -- 172.21.15.111:0/1368802254 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3f2c057650 con 0x7f3f34158da0 2023-03-29T10:58:34.410 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-03-29T10:58:34.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.412+0000 7f3f3afff700 1 -- 172.21.15.111:0/1368802254 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3f18069de0 msgr2=0x7f3f1806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:34.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.412+0000 7f3f3afff700 1 --2- 172.21.15.111:0/1368802254 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3f18069de0 0x7f3f1806c290 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3f28000f60 tx=0x7f3f28009040 comp rx=0 tx=0).stop 2023-03-29T10:58:34.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.412+0000 7f3f3afff700 1 -- 172.21.15.111:0/1368802254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f34158da0 msgr2=0x7f3f341574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:34.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.412+0000 7f3f3afff700 1 --2- 172.21.15.111:0/1368802254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f34158da0 0x7f3f341574e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c007e70 tx=0x7f3f2c00e3c0 comp rx=0 tx=0).stop 2023-03-29T10:58:34.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.412+0000 7f3f3afff700 1 -- 172.21.15.111:0/1368802254 shutdown_connections 2023-03-29T10:58:34.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.412+0000 7f3f3afff700 1 --2- 172.21.15.111:0/1368802254 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3f18069de0 0x7f3f1806c290 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:34.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.412+0000 7f3f3afff700 1 --2- 172.21.15.111:0/1368802254 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3f34158da0 0x7f3f341574e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:34.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.412+0000 7f3f3afff700 1 --2- 172.21.15.111:0/1368802254 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3f3406cc00 0x7f3f3415bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:34.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.412+0000 7f3f3afff700 1 --2- 172.21.15.111:0/1368802254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f341589d0 0x7f3f340c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:34.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.412+0000 7f3f3afff700 1 -- 172.21.15.111:0/1368802254 >> 172.21.15.111:0/1368802254 conn(0x7f3f340cf720 msgr2=0x7f3f3406d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:34.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.412+0000 7f3f3afff700 1 -- 172.21.15.111:0/1368802254 shutdown_connections 2023-03-29T10:58:34.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.412+0000 7f3f3afff700 1 -- 172.21.15.111:0/1368802254 wait complete. 2023-03-29T10:58:34.428 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-29T10:58:34.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.616+0000 7f8ff55ff700 1 Processor -- start 2023-03-29T10:58:34.616 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.616+0000 7f8ff55ff700 1 -- start start 2023-03-29T10:58:34.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.617+0000 7f8ff55ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ff00cfbf0 0x7f8ff01608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:34.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.617+0000 7f8ff55ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8ff00d1100 0x7f8ff00d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:34.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.617+0000 7f8ff55ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff00d0020 con 0x7f8ff00b87a0 2023-03-29T10:58:34.617 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.617+0000 7f8ff55ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff00d02d0 con 0x7f8ff00d1100 2023-03-29T10:58:34.618 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.617+0000 7f8ff55ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff00d0cb0 con 0x7f8ff00cfbf0 2023-03-29T10:58:34.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.618+0000 7f8fe67fc700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ff00cfbf0 0x7f8ff01608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:34.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.618+0000 7f8fef7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8ff00d1100 0x7f8ff00d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:34.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.618+0000 7f8feeffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8ff00b87a0 0x7f8ff01601e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.111:41682/0 (socket says 172.21.15.111:41682) 2023-03-29T10:58:34.619 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.618+0000 7f8feeffd700 1 -- 172.21.15.111:0/545442414 learned_addr learned my addr 172.21.15.111:0/545442414 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:34.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.618+0000 7f8fef7fe700 1 -- 172.21.15.111:0/545442414 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ff00cfbf0 msgr2=0x7f8ff01608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:34.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.618+0000 7f8fef7fe700 1 --2- 172.21.15.111:0/545442414 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ff00cfbf0 0x7f8ff01608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:34.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.618+0000 7f8fef7fe700 1 -- 172.21.15.111:0/545442414 >> v1:172.21.15.119:6789/0 conn(0x7f8ff00b87a0 legacy=0x7f8ff01601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:34.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.618+0000 7f8fef7fe700 1 -- 172.21.15.111:0/545442414 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff00cdc60 con 0x7f8ff00d1100 2023-03-29T10:58:34.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.619+0000 7f8fef7fe700 1 --2- 172.21.15.111:0/545442414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff00d1100 0x7f8ff00d1580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009d90 tx=0x7f8fe800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebcc53d0b7f7afc6 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:34.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.619+0000 7f8fee7fc700 1 -- 172.21.15.111:0/545442414 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe800e070 con 0x7f8ff00d1100 2023-03-29T10:58:34.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.619+0000 7f8fee7fc700 1 -- 172.21.15.111:0/545442414 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe800bb20 con 0x7f8ff00d1100 2023-03-29T10:58:34.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.619+0000 7f8fee7fc700 1 -- 172.21.15.111:0/545442414 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8012430 con 0x7f8ff00d1100 2023-03-29T10:58:34.620 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.620+0000 7f8ff55ff700 1 -- 172.21.15.111:0/545442414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff00d1100 msgr2=0x7f8ff00d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:34.621 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.620+0000 7f8ff55ff700 1 --2- 172.21.15.111:0/545442414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff00d1100 0x7f8ff00d1580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009d90 tx=0x7f8fe800b330 comp rx=0 tx=0).stop 2023-03-29T10:58:34.621 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.620+0000 7f8ff55ff700 1 -- 172.21.15.111:0/545442414 shutdown_connections 2023-03-29T10:58:34.621 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.620+0000 7f8ff55ff700 1 --2- 172.21.15.111:0/545442414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff00d1100 0x7f8ff00d1580 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:34.621 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.620+0000 7f8ff55ff700 1 --2- 172.21.15.111:0/545442414 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ff00cfbf0 0x7f8ff01608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:34.621 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.620+0000 7f8ff55ff700 1 -- 172.21.15.111:0/545442414 >> 172.21.15.111:0/545442414 conn(0x7f8ff0071b10 msgr2=0x7f8ff0071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:34.621 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.620+0000 7f8ff55ff700 1 -- 172.21.15.111:0/545442414 shutdown_connections 2023-03-29T10:58:34.621 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.620+0000 7f8ff55ff700 1 -- 172.21.15.111:0/545442414 wait complete. 2023-03-29T10:58:34.621 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.621+0000 7f8ff55ff700 1 Processor -- start 2023-03-29T10:58:34.621 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.622+0000 7f8ff55ff700 1 -- start start 2023-03-29T10:58:34.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.622+0000 7f8ff55ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ff00b87a0 0x7f8ff014d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:34.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.623+0000 7f8ff55ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ff00cfbf0 0x7f8ff00d28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:34.622 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.623+0000 7f8ff55ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff00d1100 0x7f8ff00d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:34.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.623+0000 7f8ff55ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff00c8450 con 0x7f8ff00b87a0 2023-03-29T10:58:34.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.623+0000 7f8ff55ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff014d8c0 con 0x7f8ff00d1100 2023-03-29T10:58:34.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.623+0000 7f8ff55ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff00c77d0 con 0x7f8ff00cfbf0 2023-03-29T10:58:34.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.623+0000 7f8feeffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ff00b87a0 0x7f8ff014d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:34.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.623+0000 7f8feeffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ff00b87a0 0x7f8ff014d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44850/0 (socket says 172.21.15.111:44850) 2023-03-29T10:58:34.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.623+0000 7f8feeffd700 1 -- 172.21.15.111:0/3131237451 learned_addr learned my addr 172.21.15.111:0/3131237451 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:34.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.623+0000 7f8fe67fc700 1 --2- 172.21.15.111:0/3131237451 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ff00cfbf0 0x7f8ff00d28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:34.623 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.623+0000 7f8fef7fe700 1 --2- 172.21.15.111:0/3131237451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff00d1100 0x7f8ff00d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:34.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.623+0000 7f8feeffd700 1 -- 172.21.15.111:0/3131237451 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ff00cfbf0 msgr2=0x7f8ff00d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:34.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.623+0000 7f8feeffd700 1 --2- 172.21.15.111:0/3131237451 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ff00cfbf0 0x7f8ff00d28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:34.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.623+0000 7f8feeffd700 1 -- 172.21.15.111:0/3131237451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff00d1100 msgr2=0x7f8ff00d2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:34.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.623+0000 7f8feeffd700 1 --2- 172.21.15.111:0/3131237451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff00d1100 0x7f8ff00d2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:34.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.623+0000 7f8feeffd700 1 -- 172.21.15.111:0/3131237451 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe8009a70 con 0x7f8ff00b87a0 2023-03-29T10:58:34.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.623+0000 7f8feeffd700 1 --2- 172.21.15.111:0/3131237451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ff00b87a0 0x7f8ff014d380 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000c480 tx=0x7f8fe000c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:34.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.624+0000 7f8fe7fff700 1 -- 172.21.15.111:0/3131237451 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0005d70 con 0x7f8ff00b87a0 2023-03-29T10:58:34.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.624+0000 7f8fe7fff700 1 -- 172.21.15.111:0/3131237451 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe000ce90 con 0x7f8ff00b87a0 2023-03-29T10:58:34.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.624+0000 7f8fe7fff700 1 -- 172.21.15.111:0/3131237451 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00072d0 con 0x7f8ff00b87a0 2023-03-29T10:58:34.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.624+0000 7f8ff55ff700 1 -- 172.21.15.111:0/3131237451 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff00cd980 con 0x7f8ff00b87a0 2023-03-29T10:58:34.624 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.624+0000 7f8ff55ff700 1 -- 172.21.15.111:0/3131237451 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff00cd6a0 con 0x7f8ff00b87a0 2023-03-29T10:58:34.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.625+0000 7f8ff55ff700 1 -- 172.21.15.111:0/3131237451 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff0160e30 con 0x7f8ff00b87a0 2023-03-29T10:58:34.625 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.626+0000 7f8fe7fff700 1 -- 172.21.15.111:0/3131237451 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f8fe0007430 con 0x7f8ff00b87a0 2023-03-29T10:58:34.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.627+0000 7f8fe7fff700 1 --2- 172.21.15.111:0/3131237451 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8fd0069de0 0x7f8fd006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:34.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.627+0000 7f8fe7fff700 1 -- 172.21.15.111:0/3131237451 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8fe00875d0 con 0x7f8ff00b87a0 2023-03-29T10:58:34.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.628+0000 7f8fe67fc700 1 --2- 172.21.15.111:0/3131237451 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8fd0069de0 0x7f8fd006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:34.627 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.628+0000 7f8fe67fc700 1 --2- 172.21.15.111:0/3131237451 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8fd0069de0 0x7f8fd006c290 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00b2650 tx=0x7f8fd8009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:34.631 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.631+0000 7f8fe7fff700 1 -- 172.21.15.111:0/3131237451 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f8fe004c160 con 0x7f8ff00b87a0 2023-03-29T10:58:34.821 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.822+0000 7f8ff55ff700 1 -- 172.21.15.111:0/3131237451 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8ff0074430 con 0x7f8ff00b87a0 2023-03-29T10:58:34.822 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.822+0000 7f8fe7fff700 1 -- 172.21.15.111:0/3131237451 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8fe0052ad0 con 0x7f8ff00b87a0 2023-03-29T10:58:34.822 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:34.824 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.825+0000 7f8ff55ff700 1 -- 172.21.15.111:0/3131237451 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8fd0069de0 msgr2=0x7f8fd006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:34.824 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.825+0000 7f8ff55ff700 1 --2- 172.21.15.111:0/3131237451 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8fd0069de0 0x7f8fd006c290 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8ff00b2650 tx=0x7f8fd8009210 comp rx=0 tx=0).stop 2023-03-29T10:58:34.824 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.825+0000 7f8ff55ff700 1 -- 172.21.15.111:0/3131237451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ff00b87a0 msgr2=0x7f8ff014d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:34.824 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.825+0000 7f8ff55ff700 1 --2- 172.21.15.111:0/3131237451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ff00b87a0 0x7f8ff014d380 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000c480 tx=0x7f8fe000c790 comp rx=0 tx=0).stop 2023-03-29T10:58:34.825 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.825+0000 7f8ff55ff700 1 -- 172.21.15.111:0/3131237451 shutdown_connections 2023-03-29T10:58:34.825 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.825+0000 7f8ff55ff700 1 --2- 172.21.15.111:0/3131237451 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8fd0069de0 0x7f8fd006c290 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:34.825 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.825+0000 7f8ff55ff700 1 --2- 172.21.15.111:0/3131237451 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8ff00b87a0 0x7f8ff014d380 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:34.825 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.825+0000 7f8ff55ff700 1 --2- 172.21.15.111:0/3131237451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ff00d1100 0x7f8ff00d2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:34.825 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.825+0000 7f8ff55ff700 1 --2- 172.21.15.111:0/3131237451 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8ff00cfbf0 0x7f8ff00d28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:34.825 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.825+0000 7f8ff55ff700 1 -- 172.21.15.111:0/3131237451 >> 172.21.15.111:0/3131237451 conn(0x7f8ff0071b10 msgr2=0x7f8ff00cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:34.825 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.825+0000 7f8ff55ff700 1 -- 172.21.15.111:0/3131237451 shutdown_connections 2023-03-29T10:58:34.825 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:34.825+0000 7f8ff55ff700 1 -- 172.21.15.111:0/3131237451 wait complete. 2023-03-29T10:58:34.843 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-03-29T10:58:35.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.028+0000 7effab7ff700 1 Processor -- start 2023-03-29T10:58:35.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.028+0000 7effab7ff700 1 -- start start 2023-03-29T10:58:35.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.029+0000 7effab7ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effa415a280 0x7effa415cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:35.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.029+0000 7effab7ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7effa40cd640 0x7effa40cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:35.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.029+0000 7effab7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa415d6e0 con 0x7effa40d00b0 2023-03-29T10:58:35.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.029+0000 7effab7ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa415d9c0 con 0x7effa40cd640 2023-03-29T10:58:35.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.029+0000 7effab7ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa415dca0 con 0x7effa415a280 2023-03-29T10:58:35.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.030+0000 7effa959b700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effa415a280 0x7effa415cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:35.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.030+0000 7effa9d9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7effa40d00b0 0x7effa40d0b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.111:41686/0 (socket says 172.21.15.111:41686) 2023-03-29T10:58:35.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.030+0000 7effa959b700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effa415a280 0x7effa415cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:56312/0 (socket says 172.21.15.111:56312) 2023-03-29T10:58:35.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.030+0000 7effa8d9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7effa40cd640 0x7effa40cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:35.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.030+0000 7effa9d9c700 1 -- 172.21.15.111:0/1259081291 learned_addr learned my addr 172.21.15.111:0/1259081291 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:35.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.030+0000 7eff9bfff700 1 -- 172.21.15.111:0/1259081291 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 178528480 0 0) 0x7effa415d6e0 con 0x7effa40d00b0 2023-03-29T10:58:35.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.030+0000 7eff9bfff700 1 -- 172.21.15.111:0/1259081291 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff900035b0 con 0x7effa40d00b0 2023-03-29T10:58:35.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.030+0000 7effa8d9a700 1 -- 172.21.15.111:0/1259081291 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effa415a280 msgr2=0x7effa415cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:35.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.030+0000 7effa8d9a700 1 --2- 172.21.15.111:0/1259081291 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effa415a280 0x7effa415cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.030+0000 7effa8d9a700 1 -- 172.21.15.111:0/1259081291 >> v1:172.21.15.119:6789/0 conn(0x7effa40d00b0 legacy=0x7effa40d0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:35.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.030+0000 7effa8d9a700 1 -- 172.21.15.111:0/1259081291 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa40cf370 con 0x7effa40cd640 2023-03-29T10:58:35.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.030+0000 7effa959b700 1 --2- 172.21.15.111:0/1259081291 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effa415a280 0x7effa415cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:58:35.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.030+0000 7effa8d9a700 1 --2- 172.21.15.111:0/1259081291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effa40cd640 0x7effa40cfb70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7eff94009840 tx=0x7eff94009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a61374cbf666de83 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:35.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.031+0000 7eff9bfff700 1 -- 172.21.15.111:0/1259081291 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94011830 con 0x7effa40cd640 2023-03-29T10:58:35.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.031+0000 7eff9bfff700 1 -- 172.21.15.111:0/1259081291 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff94011990 con 0x7effa40cd640 2023-03-29T10:58:35.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.031+0000 7eff9bfff700 1 -- 172.21.15.111:0/1259081291 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94011d00 con 0x7effa40cd640 2023-03-29T10:58:35.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.031+0000 7effab7ff700 1 -- 172.21.15.111:0/1259081291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effa40cd640 msgr2=0x7effa40cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:35.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.031+0000 7effab7ff700 1 --2- 172.21.15.111:0/1259081291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effa40cd640 0x7effa40cfb70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7eff94009840 tx=0x7eff94009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:35.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.031+0000 7effab7ff700 1 -- 172.21.15.111:0/1259081291 shutdown_connections 2023-03-29T10:58:35.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.031+0000 7effab7ff700 1 --2- 172.21.15.111:0/1259081291 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effa415a280 0x7effa415cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.031+0000 7effab7ff700 1 --2- 172.21.15.111:0/1259081291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effa40cd640 0x7effa40cfb70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.031+0000 7effab7ff700 1 -- 172.21.15.111:0/1259081291 >> 172.21.15.111:0/1259081291 conn(0x7effa4066260 msgr2=0x7effa4071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:35.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.031+0000 7effab7ff700 1 -- 172.21.15.111:0/1259081291 shutdown_connections 2023-03-29T10:58:35.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.031+0000 7effab7ff700 1 -- 172.21.15.111:0/1259081291 wait complete. 2023-03-29T10:58:35.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.033+0000 7effab7ff700 1 Processor -- start 2023-03-29T10:58:35.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.033+0000 7effab7ff700 1 -- start start 2023-03-29T10:58:35.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.034+0000 7effab7ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effa40cd640 0x7effa414d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:35.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.034+0000 7effab7ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effa40d00b0 0x7effa40cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:35.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.034+0000 7effab7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effa415a280 0x7effa40b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:35.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.034+0000 7effab7ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa40cb620 con 0x7effa40cd640 2023-03-29T10:58:35.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.034+0000 7effab7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa40ccf20 con 0x7effa415a280 2023-03-29T10:58:35.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.034+0000 7effab7ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa40b6520 con 0x7effa40d00b0 2023-03-29T10:58:35.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.034+0000 7effa959b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effa40cd640 0x7effa414d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:35.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.034+0000 7effa959b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effa40cd640 0x7effa414d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44858/0 (socket says 172.21.15.111:44858) 2023-03-29T10:58:35.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.034+0000 7effa959b700 1 -- 172.21.15.111:0/3817725091 learned_addr learned my addr 172.21.15.111:0/3817725091 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:35.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.034+0000 7effa8d9a700 1 --2- 172.21.15.111:0/3817725091 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effa40d00b0 0x7effa40cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:35.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.034+0000 7effa9d9c700 1 --2- 172.21.15.111:0/3817725091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effa415a280 0x7effa40b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:35.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.035+0000 7effa959b700 1 -- 172.21.15.111:0/3817725091 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effa40d00b0 msgr2=0x7effa40cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:35.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.035+0000 7effa959b700 1 --2- 172.21.15.111:0/3817725091 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effa40d00b0 0x7effa40cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.035+0000 7effa959b700 1 -- 172.21.15.111:0/3817725091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effa415a280 msgr2=0x7effa40b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:35.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.035+0000 7effa959b700 1 --2- 172.21.15.111:0/3817725091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effa415a280 0x7effa40b75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.035+0000 7effa959b700 1 -- 172.21.15.111:0/3817725091 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa00094a0 con 0x7effa40cd640 2023-03-29T10:58:35.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.035+0000 7effa959b700 1 --2- 172.21.15.111:0/3817725091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effa40cd640 0x7effa414d000 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7effa000ebe0 tx=0x7effa000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:35.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.035+0000 7eff99ffb700 1 -- 172.21.15.111:0/3817725091 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0009760 con 0x7effa40cd640 2023-03-29T10:58:35.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.035+0000 7eff99ffb700 1 -- 172.21.15.111:0/3817725091 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa000c850 con 0x7effa40cd640 2023-03-29T10:58:35.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.035+0000 7effab7ff700 1 -- 172.21.15.111:0/3817725091 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff940094a0 con 0x7effa40cd640 2023-03-29T10:58:35.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.035+0000 7eff99ffb700 1 -- 172.21.15.111:0/3817725091 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa000cbe0 con 0x7effa40cd640 2023-03-29T10:58:35.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.035+0000 7effab7ff700 1 -- 172.21.15.111:0/3817725091 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa415a650 con 0x7effa40cd640 2023-03-29T10:58:35.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.036+0000 7effab7ff700 1 -- 172.21.15.111:0/3817725091 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa40cf650 con 0x7effa40cd640 2023-03-29T10:58:35.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.037+0000 7eff99ffb700 1 -- 172.21.15.111:0/3817725091 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7effa000cd40 con 0x7effa40cd640 2023-03-29T10:58:35.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.038+0000 7eff99ffb700 1 --2- 172.21.15.111:0/3817725091 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7eff88069de0 0x7eff8806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:35.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.038+0000 7eff99ffb700 1 -- 172.21.15.111:0/3817725091 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7effa0088290 con 0x7effa40cd640 2023-03-29T10:58:35.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.038+0000 7effa8d9a700 1 --2- 172.21.15.111:0/3817725091 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7eff88069de0 0x7eff8806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:35.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.039+0000 7effa8d9a700 1 --2- 172.21.15.111:0/3817725091 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7eff88069de0 0x7eff8806c290 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7eff94009fd0 tx=0x7eff94008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:35.042 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.042+0000 7eff99ffb700 1 -- 172.21.15.111:0/3817725091 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7effa00572d0 con 0x7effa40cd640 2023-03-29T10:58:35.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.251+0000 7effab7ff700 1 -- 172.21.15.111:0/3817725091 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7effa415b960 con 0x7effa40cd640 2023-03-29T10:58:35.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.251+0000 7eff99ffb700 1 -- 172.21.15.111:0/3817725091 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7effa0056ec0 con 0x7effa40cd640 2023-03-29T10:58:35.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.254+0000 7effab7ff700 1 -- 172.21.15.111:0/3817725091 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7eff88069de0 msgr2=0x7eff8806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:35.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.254+0000 7effab7ff700 1 --2- 172.21.15.111:0/3817725091 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7eff88069de0 0x7eff8806c290 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7eff94009fd0 tx=0x7eff94008030 comp rx=0 tx=0).stop 2023-03-29T10:58:35.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.254+0000 7effab7ff700 1 -- 172.21.15.111:0/3817725091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effa40cd640 msgr2=0x7effa414d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:35.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.254+0000 7effab7ff700 1 --2- 172.21.15.111:0/3817725091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effa40cd640 0x7effa414d000 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7effa000ebe0 tx=0x7effa000eef0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.254+0000 7effab7ff700 1 -- 172.21.15.111:0/3817725091 shutdown_connections 2023-03-29T10:58:35.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.254+0000 7effab7ff700 1 --2- 172.21.15.111:0/3817725091 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7eff88069de0 0x7eff8806c290 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.254+0000 7effab7ff700 1 --2- 172.21.15.111:0/3817725091 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7effa40cd640 0x7effa414d000 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.254+0000 7effab7ff700 1 --2- 172.21.15.111:0/3817725091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7effa415a280 0x7effa40b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.254+0000 7effab7ff700 1 --2- 172.21.15.111:0/3817725091 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7effa40d00b0 0x7effa40cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.254+0000 7effab7ff700 1 -- 172.21.15.111:0/3817725091 >> 172.21.15.111:0/3817725091 conn(0x7effa4066260 msgr2=0x7effa415bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:35.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.254+0000 7effab7ff700 1 -- 172.21.15.111:0/3817725091 shutdown_connections 2023-03-29T10:58:35.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.254+0000 7effab7ff700 1 -- 172.21.15.111:0/3817725091 wait complete. 2023-03-29T10:58:35.254 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-03-29T10:58:35.270 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi111.front.sepia.ceph.com... 2023-03-29T10:58:35.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:35.271 DEBUG:teuthology.orchestra.run.smithi111:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-03-29T10:58:35.333 INFO:teuthology.orchestra.run:waiting for 300 2023-03-29T10:58:35.333 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-03-29T10:58:35.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:35.333 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-03-29T10:58:35.374 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:58:35.374 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-29T10:58:35.420 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:58:35.421 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-29T10:58:35.505 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:58:35.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.690+0000 7fcade5ff700 1 Processor -- start 2023-03-29T10:58:35.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.691+0000 7fcade5ff700 1 -- start start 2023-03-29T10:58:35.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.692+0000 7fcade5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcad815c950 0x7fcad80cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:35.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.692+0000 7fcade5ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad80cdc80 con 0x7fcad80cf950 2023-03-29T10:58:35.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.692+0000 7fcade5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad80cd6c0 con 0x7fcad80d3a30 2023-03-29T10:58:35.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.692+0000 7fcade5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad80d3e00 con 0x7fcad815c950 2023-03-29T10:58:35.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.692+0000 7fcad7fff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fcad80cf950 0x7fcad80cfd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43590/0 (socket says 172.21.15.111:43590) 2023-03-29T10:58:35.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.692+0000 7fcad7fff700 1 -- 172.21.15.111:0/208877177 learned_addr learned my addr 172.21.15.111:0/208877177 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:35.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.692+0000 7fcad6ffd700 1 -- 172.21.15.111:0/208877177 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575726359 0 0) 0x7fcad80cdc80 con 0x7fcad80cf950 2023-03-29T10:58:35.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.692+0000 7fcad6ffd700 1 -- 172.21.15.111:0/208877177 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcac00035b0 con 0x7fcad80cf950 2023-03-29T10:58:35.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.692+0000 7fcad77fe700 1 --2- 172.21.15.111:0/208877177 >> v2:172.21.15.119:3300/0 conn(0x7fcad815c950 0x7fcad80cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:35.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcad6ffd700 1 -- 172.21.15.111:0/208877177 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1850174543 0 0) 0x7fcac00035b0 con 0x7fcad80cf950 2023-03-29T10:58:35.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcad6ffd700 1 -- 172.21.15.111:0/208877177 >> v2:172.21.15.119:3300/0 conn(0x7fcad815c950 msgr2=0x7fcad80cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:35.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcad6ffd700 1 --2- 172.21.15.111:0/208877177 >> v2:172.21.15.119:3300/0 conn(0x7fcad815c950 0x7fcad80cf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcad6ffd700 1 -- 172.21.15.111:0/208877177 >> v1:172.21.15.119:6789/0 conn(0x7fcad80d3a30 legacy=0x7fcad80b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:35.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcad6ffd700 1 -- 172.21.15.111:0/208877177 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad80cd9a0 con 0x7fcad80cf950 2023-03-29T10:58:35.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcad6ffd700 1 -- 172.21.15.111:0/208877177 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7fcac8001670 con 0x7fcad80cf950 2023-03-29T10:58:35.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcad6ffd700 1 -- 172.21.15.111:0/208877177 >> v1:172.21.15.111:6789/0 conn(0x7fcad80cf950 legacy=0x7fcad80cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:35.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcad6ffd700 1 --2- 172.21.15.111:0/208877177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcac0003a70 0x7fcac0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:35.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcad6ffd700 1 --2- 172.21.15.111:0/208877177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcac0004850 0x7fcac0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:35.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcad6ffd700 1 --2- 172.21.15.111:0/208877177 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcac0005270 0x7fcac000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:35.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcad6ffd700 1 -- 172.21.15.111:0/208877177 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac000c180 con 0x7fcac0003a70 2023-03-29T10:58:35.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcad6ffd700 1 -- 172.21.15.111:0/208877177 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac000c2f0 con 0x7fcac0004850 2023-03-29T10:58:35.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcad6ffd700 1 -- 172.21.15.111:0/208877177 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac000c4b0 con 0x7fcac0005270 2023-03-29T10:58:35.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcad7fff700 1 --2- 172.21.15.111:0/208877177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcac0003a70 0x7fcac0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:35.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcad77fe700 1 --2- 172.21.15.111:0/208877177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcac0004850 0x7fcac0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:35.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcadcb9c700 1 --2- 172.21.15.111:0/208877177 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcac0005270 0x7fcac000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:35.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcad77fe700 1 -- 172.21.15.111:0/208877177 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcac0005270 msgr2=0x7fcac000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:35.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcad77fe700 1 --2- 172.21.15.111:0/208877177 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcac0005270 0x7fcac000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcad77fe700 1 -- 172.21.15.111:0/208877177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcac0003a70 msgr2=0x7fcac0003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:35.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.693+0000 7fcad77fe700 1 --2- 172.21.15.111:0/208877177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcac0003a70 0x7fcac0003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.694+0000 7fcad77fe700 1 -- 172.21.15.111:0/208877177 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac00035b0 con 0x7fcac0004850 2023-03-29T10:58:35.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.694+0000 7fcad7fff700 1 --2- 172.21.15.111:0/208877177 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcac0003a70 0x7fcac0003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:58:35.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.694+0000 7fcadcb9c700 1 -- 172.21.15.111:0/208877177 reap_dead start 2023-03-29T10:58:35.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.694+0000 7fcad77fe700 1 --2- 172.21.15.111:0/208877177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcac0004850 0x7fcac0004cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009b60 tx=0x7fcac0005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=803dced332b61114 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:35.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.694+0000 7fcad6ffd700 1 -- 172.21.15.111:0/208877177 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc00e6c0 con 0x7fcac0004850 2023-03-29T10:58:35.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.694+0000 7fcad6ffd700 1 -- 172.21.15.111:0/208877177 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcacc00e820 con 0x7fcac0004850 2023-03-29T10:58:35.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.694+0000 7fcad6ffd700 1 -- 172.21.15.111:0/208877177 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc00ebb0 con 0x7fcac0004850 2023-03-29T10:58:35.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.694+0000 7fcade5ff700 1 -- 172.21.15.111:0/208877177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcac0004850 msgr2=0x7fcac0004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:35.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.694+0000 7fcade5ff700 1 --2- 172.21.15.111:0/208877177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcac0004850 0x7fcac0004cc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcacc009b60 tx=0x7fcac0005880 comp rx=0 tx=0).stop 2023-03-29T10:58:35.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.695+0000 7fcade5ff700 1 -- 172.21.15.111:0/208877177 shutdown_connections 2023-03-29T10:58:35.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.695+0000 7fcade5ff700 1 --2- 172.21.15.111:0/208877177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcac0004850 0x7fcac0004cc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.695+0000 7fcade5ff700 1 -- 172.21.15.111:0/208877177 >> 172.21.15.111:0/208877177 conn(0x7fcad8071b10 msgr2=0x7fcad8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:35.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.695+0000 7fcade5ff700 1 -- 172.21.15.111:0/208877177 shutdown_connections 2023-03-29T10:58:35.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.695+0000 7fcade5ff700 1 -- 172.21.15.111:0/208877177 wait complete. 2023-03-29T10:58:35.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.696+0000 7fcade5ff700 1 Processor -- start 2023-03-29T10:58:35.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.696+0000 7fcade5ff700 1 -- start start 2023-03-29T10:58:35.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.697+0000 7fcade5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcac0004850 0x7fcad8146380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:35.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.697+0000 7fcade5ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcad814f180 0x7fcad815e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:35.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.697+0000 7fcade5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcad80ca010 0x7fcad80ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:35.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.697+0000 7fcade5ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad80ca9c0 con 0x7fcad814f180 2023-03-29T10:58:35.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.697+0000 7fcade5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad80b6ad0 con 0x7fcad80ca010 2023-03-29T10:58:35.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.697+0000 7fcade5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad815f000 con 0x7fcac0004850 2023-03-29T10:58:35.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.698+0000 7fcad77fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcad814f180 0x7fcad815e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:35.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.698+0000 7fcad77fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcad814f180 0x7fcad815e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44880/0 (socket says 172.21.15.111:44880) 2023-03-29T10:58:35.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.698+0000 7fcad77fe700 1 -- 172.21.15.111:0/2250132245 learned_addr learned my addr 172.21.15.111:0/2250132245 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:35.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.698+0000 7fcadcb9c700 1 --2- 172.21.15.111:0/2250132245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcad80ca010 0x7fcad80ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:35.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.698+0000 7fcad7fff700 1 --2- 172.21.15.111:0/2250132245 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcac0004850 0x7fcad8146380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:35.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.698+0000 7fcad77fe700 1 -- 172.21.15.111:0/2250132245 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcac0004850 msgr2=0x7fcad8146380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:35.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.698+0000 7fcad77fe700 1 --2- 172.21.15.111:0/2250132245 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcac0004850 0x7fcad8146380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.698+0000 7fcad77fe700 1 -- 172.21.15.111:0/2250132245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcad80ca010 msgr2=0x7fcad80ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:35.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.698+0000 7fcad77fe700 1 --2- 172.21.15.111:0/2250132245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcad80ca010 0x7fcad80ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.698+0000 7fcad77fe700 1 -- 172.21.15.111:0/2250132245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad80d19a0 con 0x7fcad814f180 2023-03-29T10:58:35.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.698+0000 7fcad7fff700 1 --2- 172.21.15.111:0/2250132245 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcac0004850 0x7fcad8146380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:35.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.698+0000 7fcadcb9c700 1 --2- 172.21.15.111:0/2250132245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcad80ca010 0x7fcad80ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:35.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.698+0000 7fcad77fe700 1 --2- 172.21.15.111:0/2250132245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcad814f180 0x7fcad815e7d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0090b0 tx=0x7fcacc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:35.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.698+0000 7fcad4ff9700 1 -- 172.21.15.111:0/2250132245 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc00e6c0 con 0x7fcad814f180 2023-03-29T10:58:35.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.698+0000 7fcad4ff9700 1 -- 172.21.15.111:0/2250132245 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcacc003ab0 con 0x7fcad814f180 2023-03-29T10:58:35.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.699+0000 7fcad4ff9700 1 -- 172.21.15.111:0/2250132245 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc003e60 con 0x7fcad814f180 2023-03-29T10:58:35.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.699+0000 7fcade5ff700 1 -- 172.21.15.111:0/2250132245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad80d16c0 con 0x7fcad814f180 2023-03-29T10:58:35.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.699+0000 7fcade5ff700 1 -- 172.21.15.111:0/2250132245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad80cfd20 con 0x7fcad814f180 2023-03-29T10:58:35.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.700+0000 7fcade5ff700 1 -- 172.21.15.111:0/2250132245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad80b6c80 con 0x7fcad814f180 2023-03-29T10:58:35.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.701+0000 7fcad4ff9700 1 -- 172.21.15.111:0/2250132245 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fcacc013470 con 0x7fcad814f180 2023-03-29T10:58:35.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.702+0000 7fcad4ff9700 1 --2- 172.21.15.111:0/2250132245 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fcac4069e10 0x7fcac406c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:35.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.702+0000 7fcad4ff9700 1 -- 172.21.15.111:0/2250132245 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcacc089bd0 con 0x7fcad814f180 2023-03-29T10:58:35.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.702+0000 7fcad7fff700 1 --2- 172.21.15.111:0/2250132245 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fcac4069e10 0x7fcac406c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:35.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.703+0000 7fcad7fff700 1 --2- 172.21.15.111:0/2250132245 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fcac4069e10 0x7fcac406c2c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcac8001200 tx=0x7fcac8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:35.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.706+0000 7fcad4ff9700 1 -- 172.21.15.111:0/2250132245 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fcad80b6c80 con 0x7fcad814f180 2023-03-29T10:58:35.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.907+0000 7fcade5ff700 1 -- 172.21.15.111:0/2250132245 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcad80b6c40 con 0x7fcad814f180 2023-03-29T10:58:35.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.909+0000 7fcad4ff9700 1 -- 172.21.15.111:0/2250132245 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19892 (secure 0 0 0) 0x7fcacc058b70 con 0x7fcad814f180 2023-03-29T10:58:35.909 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:35.909 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":30,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:58:09.476334+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.26","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:58:35.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.911+0000 7fcade5ff700 1 -- 172.21.15.111:0/2250132245 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fcac4069e10 msgr2=0x7fcac406c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:35.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.911+0000 7fcade5ff700 1 --2- 172.21.15.111:0/2250132245 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fcac4069e10 0x7fcac406c2c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcac8001200 tx=0x7fcac8009040 comp rx=0 tx=0).stop 2023-03-29T10:58:35.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.911+0000 7fcade5ff700 1 -- 172.21.15.111:0/2250132245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcad814f180 msgr2=0x7fcad815e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:35.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.911+0000 7fcade5ff700 1 --2- 172.21.15.111:0/2250132245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcad814f180 0x7fcad815e7d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0090b0 tx=0x7fcacc0047c0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.911+0000 7fcade5ff700 1 -- 172.21.15.111:0/2250132245 shutdown_connections 2023-03-29T10:58:35.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.911+0000 7fcade5ff700 1 --2- 172.21.15.111:0/2250132245 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fcac4069e10 0x7fcac406c2c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.911+0000 7fcade5ff700 1 --2- 172.21.15.111:0/2250132245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcad80ca010 0x7fcad80ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.911+0000 7fcade5ff700 1 --2- 172.21.15.111:0/2250132245 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcac0004850 0x7fcad8146380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.911+0000 7fcade5ff700 1 --2- 172.21.15.111:0/2250132245 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcad814f180 0x7fcad815e7d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:35.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.911+0000 7fcade5ff700 1 -- 172.21.15.111:0/2250132245 >> 172.21.15.111:0/2250132245 conn(0x7fcad8071b10 msgr2=0x7fcad815bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:35.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.911+0000 7fcade5ff700 1 -- 172.21.15.111:0/2250132245 shutdown_connections 2023-03-29T10:58:35.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:35.911+0000 7fcade5ff700 1 -- 172.21.15.111:0/2250132245 wait complete. 2023-03-29T10:58:35.929 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-29T10:57:16.183979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-29T10:57:21.239809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-29T10:57:26.192124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-29T10:57:26.796777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-29T10:57:29.874795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-29T10:58:35.929 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-29T10:58:36.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.115+0000 7f8ea0bff700 1 Processor -- start 2023-03-29T10:58:36.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.115+0000 7f8ea0bff700 1 -- start start 2023-03-29T10:58:36.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.116+0000 7f8ea0bff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8e9c0b87a0 0x7f8e9c0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.116+0000 7f8ea0bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e9c1601e0 0x7f8e9c160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.116+0000 7f8ea0bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8e9c160b70 0x7f8e9c0d2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.116+0000 7f8ea0bff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e9c0d02d0 con 0x7f8e9c0b87a0 2023-03-29T10:58:36.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.116+0000 7f8ea0bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e9c0d0cb0 con 0x7f8e9c160b70 2023-03-29T10:58:36.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.116+0000 7f8ea0bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c0b2730 con 0x7f8e9c1601e0 2023-03-29T10:58:36.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.116+0000 7f8e9a59c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8e9c0b87a0 0x7f8e9c0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.116+0000 7f8e9a59c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f8e9c0b87a0 0x7f8e9c0cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44888/0 (socket says 172.21.15.111:44888) 2023-03-29T10:58:36.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.116+0000 7f8e9a59c700 1 -- 172.21.15.111:0/2478935251 learned_addr learned my addr 172.21.15.111:0/2478935251 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:36.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.117+0000 7f8e9ad9d700 1 --2- 172.21.15.111:0/2478935251 >> v2:172.21.15.119:3300/0 conn(0x7f8e9c160b70 0x7f8e9c0d2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.117+0000 7f8e99d9b700 1 --2- 172.21.15.111:0/2478935251 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e9c1601e0 0x7f8e9c160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.117+0000 7f8e9a59c700 1 -- 172.21.15.111:0/2478935251 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e9c1601e0 msgr2=0x7f8e9c160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:36.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.117+0000 7f8e9a59c700 1 --2- 172.21.15.111:0/2478935251 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e9c1601e0 0x7f8e9c160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.117+0000 7f8e9a59c700 1 -- 172.21.15.111:0/2478935251 >> v2:172.21.15.119:3300/0 conn(0x7f8e9c160b70 msgr2=0x7f8e9c0d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:36.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.117+0000 7f8e9a59c700 1 --2- 172.21.15.111:0/2478935251 >> v2:172.21.15.119:3300/0 conn(0x7f8e9c160b70 0x7f8e9c0d2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.117+0000 7f8e9a59c700 1 -- 172.21.15.111:0/2478935251 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9c160ff0 con 0x7f8e9c0b87a0 2023-03-29T10:58:36.118 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.117+0000 7f8e9a59c700 1 --2- 172.21.15.111:0/2478935251 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e9c0b87a0 0x7f8e9c0cfbf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c08be30 tx=0x7f8e8c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba54a06d2a913479 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:36.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.117+0000 7f8e9959a700 1 -- 172.21.15.111:0/2478935251 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c00b870 con 0x7f8e9c0b87a0 2023-03-29T10:58:36.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.117+0000 7f8e9959a700 1 -- 172.21.15.111:0/2478935251 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8c00b9d0 con 0x7f8e9c0b87a0 2023-03-29T10:58:36.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.117+0000 7f8e9959a700 1 -- 172.21.15.111:0/2478935251 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c00bd60 con 0x7f8e9c0b87a0 2023-03-29T10:58:36.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.118+0000 7f8ea0bff700 1 -- 172.21.15.111:0/2478935251 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e9c0b87a0 msgr2=0x7f8e9c0cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:36.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.118+0000 7f8ea0bff700 1 --2- 172.21.15.111:0/2478935251 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e9c0b87a0 0x7f8e9c0cfbf0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c08be30 tx=0x7f8e8c009bd0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.118+0000 7f8ea0bff700 1 -- 172.21.15.111:0/2478935251 shutdown_connections 2023-03-29T10:58:36.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.118+0000 7f8ea0bff700 1 --2- 172.21.15.111:0/2478935251 >> v2:172.21.15.119:3300/0 conn(0x7f8e9c160b70 0x7f8e9c0d2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.119 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.118+0000 7f8ea0bff700 1 --2- 172.21.15.111:0/2478935251 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e9c0b87a0 0x7f8e9c0cfbf0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.118+0000 7f8ea0bff700 1 --2- 172.21.15.111:0/2478935251 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e9c1601e0 0x7f8e9c160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.118+0000 7f8ea0bff700 1 -- 172.21.15.111:0/2478935251 >> 172.21.15.111:0/2478935251 conn(0x7f8e9c071b10 msgr2=0x7f8e9c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:36.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.118+0000 7f8ea0bff700 1 -- 172.21.15.111:0/2478935251 shutdown_connections 2023-03-29T10:58:36.120 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.118+0000 7f8ea0bff700 1 -- 172.21.15.111:0/2478935251 wait complete. 2023-03-29T10:58:36.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.120+0000 7f8ea0bff700 1 Processor -- start 2023-03-29T10:58:36.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.120+0000 7f8ea0bff700 1 -- start start 2023-03-29T10:58:36.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.120+0000 7f8ea0bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e9c0b87a0 0x7f8e9c15f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.121+0000 7f8ea0bff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e9c1601e0 0x7f8e9c0d3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.121+0000 7f8ea0bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e9c160b70 0x7f8e9c150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.121+0000 7f8ea0bff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c1e5f00 con 0x7f8e9c1601e0 2023-03-29T10:58:36.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.121+0000 7f8ea0bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c1e67a0 con 0x7f8e9c0b87a0 2023-03-29T10:58:36.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.121+0000 7f8ea0bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c15fa00 con 0x7f8e9c160b70 2023-03-29T10:58:36.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.121+0000 7f8e99d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e9c1601e0 0x7f8e9c0d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.122 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.121+0000 7f8e99d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e9c1601e0 0x7f8e9c0d3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44892/0 (socket says 172.21.15.111:44892) 2023-03-29T10:58:36.123 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.121+0000 7f8e99d9b700 1 -- 172.21.15.111:0/1119418643 learned_addr learned my addr 172.21.15.111:0/1119418643 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:36.123 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.121+0000 7f8e9a59c700 1 --2- 172.21.15.111:0/1119418643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e9c0b87a0 0x7f8e9c15f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.123 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.121+0000 7f8e9ad9d700 1 --2- 172.21.15.111:0/1119418643 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e9c160b70 0x7f8e9c150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.123 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.121+0000 7f8e99d9b700 1 -- 172.21.15.111:0/1119418643 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e9c160b70 msgr2=0x7f8e9c150720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:36.123 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.121+0000 7f8e99d9b700 1 --2- 172.21.15.111:0/1119418643 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e9c160b70 0x7f8e9c150720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.123 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.121+0000 7f8e99d9b700 1 -- 172.21.15.111:0/1119418643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e9c0b87a0 msgr2=0x7f8e9c15f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:36.123 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.121+0000 7f8e99d9b700 1 --2- 172.21.15.111:0/1119418643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e9c0b87a0 0x7f8e9c15f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.123 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.121+0000 7f8e99d9b700 1 -- 172.21.15.111:0/1119418643 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8c009550 con 0x7f8e9c1601e0 2023-03-29T10:58:36.123 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.121+0000 7f8e9a59c700 1 --2- 172.21.15.111:0/1119418643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e9c0b87a0 0x7f8e9c15f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:36.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.121+0000 7f8e9ad9d700 1 --2- 172.21.15.111:0/1119418643 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e9c160b70 0x7f8e9c150720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:36.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.121+0000 7f8e99d9b700 1 --2- 172.21.15.111:0/1119418643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e9c1601e0 0x7f8e9c0d3120 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000e810 tx=0x7f8e9000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:36.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.122+0000 7f8e8affd700 1 -- 172.21.15.111:0/1119418643 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90013610 con 0x7f8e9c1601e0 2023-03-29T10:58:36.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.122+0000 7f8e8affd700 1 -- 172.21.15.111:0/1119418643 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e90013770 con 0x7f8e9c1601e0 2023-03-29T10:58:36.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.122+0000 7f8e8affd700 1 -- 172.21.15.111:0/1119418643 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e90013b00 con 0x7f8e9c1601e0 2023-03-29T10:58:36.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.122+0000 7f8ea0bff700 1 -- 172.21.15.111:0/1119418643 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e9c0cd6a0 con 0x7f8e9c1601e0 2023-03-29T10:58:36.124 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.122+0000 7f8ea0bff700 1 -- 172.21.15.111:0/1119418643 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e9c0cdc60 con 0x7f8e9c1601e0 2023-03-29T10:58:36.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.123+0000 7f8ea0bff700 1 -- 172.21.15.111:0/1119418643 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e9c0cd980 con 0x7f8e9c1601e0 2023-03-29T10:58:36.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.124+0000 7f8e8affd700 1 -- 172.21.15.111:0/1119418643 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f8e90013c60 con 0x7f8e9c1601e0 2023-03-29T10:58:36.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.125+0000 7f8e8affd700 1 --2- 172.21.15.111:0/1119418643 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8e7c069de0 0x7f8e7c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.125+0000 7f8e8affd700 1 -- 172.21.15.111:0/1119418643 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8e90088050 con 0x7f8e9c1601e0 2023-03-29T10:58:36.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.125+0000 7f8e9a59c700 1 --2- 172.21.15.111:0/1119418643 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8e7c069de0 0x7f8e7c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.125+0000 7f8e9a59c700 1 --2- 172.21.15.111:0/1119418643 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8e7c069de0 0x7f8e7c06c290 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0b84b0 tx=0x7f8e8c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:36.128 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.128+0000 7f8e8affd700 1 -- 172.21.15.111:0/1119418643 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f8e90053660 con 0x7f8e9c1601e0 2023-03-29T10:58:36.321 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.322+0000 7f8ea0bff700 1 -- 172.21.15.111:0/1119418643 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8e9c1e6910 con 0x7f8e9c1601e0 2023-03-29T10:58:36.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.322+0000 7f8e8affd700 1 -- 172.21.15.111:0/1119418643 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8e9000f040 con 0x7f8e9c1601e0 2023-03-29T10:58:36.322 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-03-29T10:58:36.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.324+0000 7f8ea0bff700 1 -- 172.21.15.111:0/1119418643 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8e7c069de0 msgr2=0x7f8e7c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:36.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.324+0000 7f8ea0bff700 1 --2- 172.21.15.111:0/1119418643 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8e7c069de0 0x7f8e7c06c290 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c0b84b0 tx=0x7f8e8c008030 comp rx=0 tx=0).stop 2023-03-29T10:58:36.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.324+0000 7f8ea0bff700 1 -- 172.21.15.111:0/1119418643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e9c1601e0 msgr2=0x7f8e9c0d3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:36.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.324+0000 7f8ea0bff700 1 --2- 172.21.15.111:0/1119418643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e9c1601e0 0x7f8e9c0d3120 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000e810 tx=0x7f8e9000ebd0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.324+0000 7f8ea0bff700 1 -- 172.21.15.111:0/1119418643 shutdown_connections 2023-03-29T10:58:36.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.324+0000 7f8ea0bff700 1 --2- 172.21.15.111:0/1119418643 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8e7c069de0 0x7f8e7c06c290 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.324+0000 7f8ea0bff700 1 --2- 172.21.15.111:0/1119418643 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f8e9c160b70 0x7f8e9c150720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.324+0000 7f8ea0bff700 1 --2- 172.21.15.111:0/1119418643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e9c0b87a0 0x7f8e9c15f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.324+0000 7f8ea0bff700 1 --2- 172.21.15.111:0/1119418643 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f8e9c1601e0 0x7f8e9c0d3120 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.324+0000 7f8ea0bff700 1 -- 172.21.15.111:0/1119418643 >> 172.21.15.111:0/1119418643 conn(0x7f8e9c071b10 msgr2=0x7f8e9c0b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:36.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.325+0000 7f8ea0bff700 1 -- 172.21.15.111:0/1119418643 shutdown_connections 2023-03-29T10:58:36.325 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.325+0000 7f8ea0bff700 1 -- 172.21.15.111:0/1119418643 wait complete. 2023-03-29T10:58:36.342 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-29T10:58:36.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.528+0000 7f39f8bff700 1 Processor -- start 2023-03-29T10:58:36.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.528+0000 7f39f8bff700 1 -- start start 2023-03-29T10:58:36.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.529+0000 7f39f8bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f39f415c590 0x7f39f4160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.529+0000 7f39f8bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f39f40cff50 0x7f39f40d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.529+0000 7f39f8bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f40cd520 con 0x7f39f40cf010 2023-03-29T10:58:36.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.529+0000 7f39f8bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f40d2640 con 0x7f39f40cff50 2023-03-29T10:58:36.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.529+0000 7f39f8bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f41610d0 con 0x7f39f415c590 2023-03-29T10:58:36.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.530+0000 7f39f259c700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f39f415c590 0x7f39f4160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.530+0000 7f39f2d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f39f40cff50 0x7f39f40d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.530+0000 7f39f259c700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f39f415c590 0x7f39f4160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:56370/0 (socket says 172.21.15.111:56370) 2023-03-29T10:58:36.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.530+0000 7f39f259c700 1 -- 172.21.15.111:0/2034309021 learned_addr learned my addr 172.21.15.111:0/2034309021 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:36.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.530+0000 7f39f2d9d700 1 -- 172.21.15.111:0/2034309021 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f39f415c590 msgr2=0x7f39f4160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:36.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.530+0000 7f39f2d9d700 1 --2- 172.21.15.111:0/2034309021 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f39f415c590 0x7f39f4160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.530+0000 7f39f2d9d700 1 -- 172.21.15.111:0/2034309021 >> v1:172.21.15.119:6789/0 conn(0x7f39f40cf010 legacy=0x7f39f40cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:36.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.530+0000 7f39f2d9d700 1 -- 172.21.15.111:0/2034309021 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f40b23c0 con 0x7f39f40cff50 2023-03-29T10:58:36.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.530+0000 7f39f259c700 1 --2- 172.21.15.111:0/2034309021 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f39f415c590 0x7f39f4160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:58:36.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.531+0000 7f39f2d9d700 1 --2- 172.21.15.111:0/2034309021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39f40cff50 0x7f39f40d2ad0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f39ec009d90 tx=0x7f39ec00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d8e549d40722288 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:36.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.531+0000 7f39f1d9b700 1 -- 172.21.15.111:0/2034309021 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec00e070 con 0x7f39f40cff50 2023-03-29T10:58:36.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.531+0000 7f39f1d9b700 1 -- 172.21.15.111:0/2034309021 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ec00bb20 con 0x7f39f40cff50 2023-03-29T10:58:36.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.531+0000 7f39f1d9b700 1 -- 172.21.15.111:0/2034309021 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ec012430 con 0x7f39f40cff50 2023-03-29T10:58:36.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.531+0000 7f39f8bff700 1 -- 172.21.15.111:0/2034309021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39f40cff50 msgr2=0x7f39f40d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:36.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.531+0000 7f39f8bff700 1 --2- 172.21.15.111:0/2034309021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39f40cff50 0x7f39f40d2ad0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f39ec009d90 tx=0x7f39ec00b330 comp rx=0 tx=0).stop 2023-03-29T10:58:36.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.532+0000 7f39f8bff700 1 -- 172.21.15.111:0/2034309021 shutdown_connections 2023-03-29T10:58:36.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.532+0000 7f39f8bff700 1 --2- 172.21.15.111:0/2034309021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39f40cff50 0x7f39f40d2ad0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.532+0000 7f39f8bff700 1 --2- 172.21.15.111:0/2034309021 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f39f415c590 0x7f39f4160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.532+0000 7f39f8bff700 1 -- 172.21.15.111:0/2034309021 >> 172.21.15.111:0/2034309021 conn(0x7f39f4071b10 msgr2=0x7f39f4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:36.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.532+0000 7f39f8bff700 1 -- 172.21.15.111:0/2034309021 shutdown_connections 2023-03-29T10:58:36.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.532+0000 7f39f8bff700 1 -- 172.21.15.111:0/2034309021 wait complete. 2023-03-29T10:58:36.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.533+0000 7f39f8bff700 1 Processor -- start 2023-03-29T10:58:36.533 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.533+0000 7f39f8bff700 1 -- start start 2023-03-29T10:58:36.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.534+0000 7f39f8bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39f40cf010 0x7f39f40cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.534+0000 7f39f8bff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39f40cff50 0x7f39f4158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.534+0000 7f39f8bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f39f415c590 0x7f39f4158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.534+0000 7f39f8bff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f40ccc70 con 0x7f39f40cff50 2023-03-29T10:58:36.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.534+0000 7f39f8bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f414bd80 con 0x7f39f40cf010 2023-03-29T10:58:36.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.534+0000 7f39f8bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f415ec50 con 0x7f39f415c590 2023-03-29T10:58:36.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.535+0000 7f39ebfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39f40cff50 0x7f39f4158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.535+0000 7f39ebfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39f40cff50 0x7f39f4158940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44904/0 (socket says 172.21.15.111:44904) 2023-03-29T10:58:36.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.535+0000 7f39ebfff700 1 -- 172.21.15.111:0/3748984563 learned_addr learned my addr 172.21.15.111:0/3748984563 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:36.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.535+0000 7f39f259c700 1 --2- 172.21.15.111:0/3748984563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39f40cf010 0x7f39f40cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.535+0000 7f39f2d9d700 1 --2- 172.21.15.111:0/3748984563 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f39f415c590 0x7f39f4158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.535+0000 7f39ebfff700 1 -- 172.21.15.111:0/3748984563 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f39f415c590 msgr2=0x7f39f4158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:36.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.535+0000 7f39ebfff700 1 --2- 172.21.15.111:0/3748984563 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f39f415c590 0x7f39f4158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.535+0000 7f39ebfff700 1 -- 172.21.15.111:0/3748984563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39f40cf010 msgr2=0x7f39f40cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:36.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.535+0000 7f39ebfff700 1 --2- 172.21.15.111:0/3748984563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39f40cf010 0x7f39f40cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.535+0000 7f39ebfff700 1 -- 172.21.15.111:0/3748984563 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e4009550 con 0x7f39f40cff50 2023-03-29T10:58:36.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.535+0000 7f39ebfff700 1 --2- 172.21.15.111:0/3748984563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39f40cff50 0x7f39f4158940 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00c4e0 tx=0x7f39dc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:36.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.536+0000 7f39eaffd700 1 -- 172.21.15.111:0/3748984563 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc0052d0 con 0x7f39f40cff50 2023-03-29T10:58:36.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.536+0000 7f39eaffd700 1 -- 172.21.15.111:0/3748984563 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39dc005430 con 0x7f39f40cff50 2023-03-29T10:58:36.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.536+0000 7f39eaffd700 1 -- 172.21.15.111:0/3748984563 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39dc0038c0 con 0x7f39f40cff50 2023-03-29T10:58:36.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.536+0000 7f39f8bff700 1 -- 172.21.15.111:0/3748984563 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ec009a70 con 0x7f39f40cff50 2023-03-29T10:58:36.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.536+0000 7f39f8bff700 1 -- 172.21.15.111:0/3748984563 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f40cd770 con 0x7f39f40cff50 2023-03-29T10:58:36.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.537+0000 7f39f8bff700 1 -- 172.21.15.111:0/3748984563 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f40cf560 con 0x7f39f40cff50 2023-03-29T10:58:36.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.538+0000 7f39eaffd700 1 -- 172.21.15.111:0/3748984563 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f39dc003a20 con 0x7f39f40cff50 2023-03-29T10:58:36.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.538+0000 7f39eaffd700 1 --2- 172.21.15.111:0/3748984563 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f39d4069de0 0x7f39d406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.538+0000 7f39eaffd700 1 -- 172.21.15.111:0/3748984563 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f39dc087710 con 0x7f39f40cff50 2023-03-29T10:58:36.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.539+0000 7f39f259c700 1 --2- 172.21.15.111:0/3748984563 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f39d4069de0 0x7f39d406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.539+0000 7f39f259c700 1 --2- 172.21.15.111:0/3748984563 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f39d4069de0 0x7f39d406c290 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f39e4006d30 tx=0x7f39e40092c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:36.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.543+0000 7f39eaffd700 1 -- 172.21.15.111:0/3748984563 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f39dc052bf0 con 0x7f39f40cff50 2023-03-29T10:58:36.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.737+0000 7f39f8bff700 1 -- 172.21.15.111:0/3748984563 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f39f415c960 con 0x7f39f40cff50 2023-03-29T10:58:36.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.737+0000 7f39eaffd700 1 -- 172.21.15.111:0/3748984563 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f39dc056210 con 0x7f39f40cff50 2023-03-29T10:58:36.737 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 8 2023-03-29T10:58:36.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.740+0000 7f39f8bff700 1 -- 172.21.15.111:0/3748984563 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f39d4069de0 msgr2=0x7f39d406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:36.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.740+0000 7f39f8bff700 1 --2- 172.21.15.111:0/3748984563 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f39d4069de0 0x7f39d406c290 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f39e4006d30 tx=0x7f39e40092c0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.740+0000 7f39f8bff700 1 -- 172.21.15.111:0/3748984563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39f40cff50 msgr2=0x7f39f4158940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:36.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.740+0000 7f39f8bff700 1 --2- 172.21.15.111:0/3748984563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39f40cff50 0x7f39f4158940 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f39dc00c4e0 tx=0x7f39dc00c7f0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.740+0000 7f39f8bff700 1 -- 172.21.15.111:0/3748984563 shutdown_connections 2023-03-29T10:58:36.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.740+0000 7f39f8bff700 1 --2- 172.21.15.111:0/3748984563 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f39d4069de0 0x7f39d406c290 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.740+0000 7f39f8bff700 1 --2- 172.21.15.111:0/3748984563 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f39f415c590 0x7f39f4158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.740+0000 7f39f8bff700 1 --2- 172.21.15.111:0/3748984563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39f40cf010 0x7f39f40cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.740+0000 7f39f8bff700 1 --2- 172.21.15.111:0/3748984563 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f39f40cff50 0x7f39f4158940 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.740+0000 7f39f8bff700 1 -- 172.21.15.111:0/3748984563 >> 172.21.15.111:0/3748984563 conn(0x7f39f4071b10 msgr2=0x7f39f40b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:36.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.740+0000 7f39f8bff700 1 -- 172.21.15.111:0/3748984563 shutdown_connections 2023-03-29T10:58:36.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.740+0000 7f39f8bff700 1 -- 172.21.15.111:0/3748984563 wait complete. 2023-03-29T10:58:36.757 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-29T10:58:36.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.943+0000 7f93bf9ff700 1 Processor -- start 2023-03-29T10:58:36.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.943+0000 7f93bf9ff700 1 -- start start 2023-03-29T10:58:36.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.944+0000 7f93bf9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f93b80d2110 0x7f93b8167550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.944+0000 7f93bf9ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b8161000 con 0x7f93b80cfbf0 2023-03-29T10:58:36.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.944+0000 7f93bf9ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b80d0020 con 0x7f93b80b87a0 2023-03-29T10:58:36.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.944+0000 7f93bf9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b80d02d0 con 0x7f93b80d2110 2023-03-29T10:58:36.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.944+0000 7f93bcf9a700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f93b80cfbf0 0x7f93b81608f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43600/0 (socket says 172.21.15.111:43600) 2023-03-29T10:58:36.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.944+0000 7f93bcf9a700 1 -- 172.21.15.111:0/3803454802 learned_addr learned my addr 172.21.15.111:0/3803454802 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:36.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.945+0000 7f93affff700 1 -- 172.21.15.111:0/3803454802 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3825576759 0 0) 0x7f93b8161000 con 0x7f93b80cfbf0 2023-03-29T10:58:36.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.945+0000 7f93affff700 1 -- 172.21.15.111:0/3803454802 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93a40035b0 con 0x7f93b80cfbf0 2023-03-29T10:58:36.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.945+0000 7f93bdf9c700 1 --2- 172.21.15.111:0/3803454802 >> v2:172.21.15.119:3300/0 conn(0x7f93b80d2110 0x7f93b8167550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.945+0000 7f93affff700 1 -- 172.21.15.111:0/3803454802 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2937142330 0 0) 0x7f93a40035b0 con 0x7f93b80cfbf0 2023-03-29T10:58:36.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.945+0000 7f93affff700 1 -- 172.21.15.111:0/3803454802 >> v2:172.21.15.119:3300/0 conn(0x7f93b80d2110 msgr2=0x7f93b8167550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:36.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.945+0000 7f93affff700 1 --2- 172.21.15.111:0/3803454802 >> v2:172.21.15.119:3300/0 conn(0x7f93b80d2110 0x7f93b8167550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.945+0000 7f93affff700 1 -- 172.21.15.111:0/3803454802 >> v1:172.21.15.119:6789/0 conn(0x7f93b80b87a0 legacy=0x7f93b81601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:36.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.945+0000 7f93affff700 1 -- 172.21.15.111:0/3803454802 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b80cdc60 con 0x7f93b80cfbf0 2023-03-29T10:58:36.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.945+0000 7f93affff700 1 -- 172.21.15.111:0/3803454802 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f93a8001670 con 0x7f93b80cfbf0 2023-03-29T10:58:36.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.945+0000 7f93affff700 1 -- 172.21.15.111:0/3803454802 >> v1:172.21.15.111:6789/0 conn(0x7f93b80cfbf0 legacy=0x7f93b81608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:36.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.945+0000 7f93affff700 1 --2- 172.21.15.111:0/3803454802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93a4003a50 0x7f93a4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.945+0000 7f93affff700 1 --2- 172.21.15.111:0/3803454802 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93a4004850 0x7f93a4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.945+0000 7f93affff700 1 --2- 172.21.15.111:0/3803454802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93a4005200 0x7f93a400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.945+0000 7f93affff700 1 -- 172.21.15.111:0/3803454802 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b80d02d0 con 0x7f93a4003a50 2023-03-29T10:58:36.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.945+0000 7f93affff700 1 -- 172.21.15.111:0/3803454802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a400c0e0 con 0x7f93a4005200 2023-03-29T10:58:36.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.945+0000 7f93affff700 1 -- 172.21.15.111:0/3803454802 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a400c2a0 con 0x7f93a4004850 2023-03-29T10:58:36.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.945+0000 7f93bd79b700 1 --2- 172.21.15.111:0/3803454802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93a4003a50 0x7f93a4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.945+0000 7f93bcf9a700 1 --2- 172.21.15.111:0/3803454802 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93a4004850 0x7f93a4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.946+0000 7f93bdf9c700 1 --2- 172.21.15.111:0/3803454802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93a4005200 0x7f93a400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.946+0000 7f93bd79b700 1 -- 172.21.15.111:0/3803454802 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93a4004850 msgr2=0x7f93a4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:36.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.946+0000 7f93bd79b700 1 --2- 172.21.15.111:0/3803454802 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93a4004850 0x7f93a4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.946+0000 7f93bd79b700 1 -- 172.21.15.111:0/3803454802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93a4005200 msgr2=0x7f93a400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:36.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.946+0000 7f93bd79b700 1 --2- 172.21.15.111:0/3803454802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93a4005200 0x7f93a400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.946+0000 7f93bd79b700 1 -- 172.21.15.111:0/3803454802 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a40035b0 con 0x7f93a4003a50 2023-03-29T10:58:36.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.946+0000 7f93bdf9c700 1 -- 172.21.15.111:0/3803454802 reap_dead start 2023-03-29T10:58:36.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.946+0000 7f93bd79b700 1 --2- 172.21.15.111:0/3803454802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93a4003a50 0x7f93a4003e60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f93b4007fd0 tx=0x7f93a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aeb5c6af8553e42 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:36.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.946+0000 7f93affff700 1 -- 172.21.15.111:0/3803454802 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b40052d0 con 0x7f93a4003a50 2023-03-29T10:58:36.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.946+0000 7f93affff700 1 -- 172.21.15.111:0/3803454802 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b4005430 con 0x7f93a4003a50 2023-03-29T10:58:36.947 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.946+0000 7f93affff700 1 -- 172.21.15.111:0/3803454802 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b40038c0 con 0x7f93a4003a50 2023-03-29T10:58:36.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.947+0000 7f93bf9ff700 1 -- 172.21.15.111:0/3803454802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93a4003a50 msgr2=0x7f93a4003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:36.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.947+0000 7f93bf9ff700 1 --2- 172.21.15.111:0/3803454802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93a4003a50 0x7f93a4003e60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f93b4007fd0 tx=0x7f93a4005880 comp rx=0 tx=0).stop 2023-03-29T10:58:36.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.947+0000 7f93bf9ff700 1 -- 172.21.15.111:0/3803454802 shutdown_connections 2023-03-29T10:58:36.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.947+0000 7f93bf9ff700 1 --2- 172.21.15.111:0/3803454802 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93a4003a50 0x7f93a4003e60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.947+0000 7f93bf9ff700 1 -- 172.21.15.111:0/3803454802 >> 172.21.15.111:0/3803454802 conn(0x7f93b8071b10 msgr2=0x7f93b8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:36.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.947+0000 7f93bf9ff700 1 -- 172.21.15.111:0/3803454802 shutdown_connections 2023-03-29T10:58:36.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.947+0000 7f93bf9ff700 1 -- 172.21.15.111:0/3803454802 wait complete. 2023-03-29T10:58:36.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.948+0000 7f93bf9ff700 1 Processor -- start 2023-03-29T10:58:36.948 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.949+0000 7f93bf9ff700 1 -- start start 2023-03-29T10:58:36.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.949+0000 7f93bf9ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93a4003a50 0x7f93b8142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.950+0000 7f93bf9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b8143d10 0x7f93b81434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.950+0000 7f93bf9ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93b81f5990 0x7f93b81f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.950+0000 7f93bf9ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b815e3f0 con 0x7f93b81f5990 2023-03-29T10:58:36.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.950+0000 7f93bf9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b81422a0 con 0x7f93b8143d10 2023-03-29T10:58:36.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.950+0000 7f93bf9ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b8142580 con 0x7f93a4003a50 2023-03-29T10:58:36.950 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.950+0000 7f93bdf9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93b81f5990 0x7f93b81f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.950 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.950+0000 7f93bcf9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b8143d10 0x7f93b81434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.950 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.950+0000 7f93bcf9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b8143d10 0x7f93b81434f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:52174/0 (socket says 172.21.15.111:52174) 2023-03-29T10:58:36.950 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.950+0000 7f93bd79b700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93a4003a50 0x7f93b8142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.951 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.950+0000 7f93bcf9a700 1 -- 172.21.15.111:0/3978514842 learned_addr learned my addr 172.21.15.111:0/3978514842 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:36.951 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.950+0000 7f93bdf9c700 1 -- 172.21.15.111:0/3978514842 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93a4003a50 msgr2=0x7f93b8142fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:36.951 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.950+0000 7f93bdf9c700 1 --2- 172.21.15.111:0/3978514842 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93a4003a50 0x7f93b8142fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.951 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.950+0000 7f93bdf9c700 1 -- 172.21.15.111:0/3978514842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b8143d10 msgr2=0x7f93b81434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:36.951 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.950+0000 7f93bdf9c700 1 --2- 172.21.15.111:0/3978514842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b8143d10 0x7f93b81434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:36.951 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.950+0000 7f93bdf9c700 1 -- 172.21.15.111:0/3978514842 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b80cd980 con 0x7f93b81f5990 2023-03-29T10:58:36.951 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.951+0000 7f93bdf9c700 1 --2- 172.21.15.111:0/3978514842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93b81f5990 0x7f93b81f5e10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f93b808c150 tx=0x7f93b00111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:36.951 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.951+0000 7f93adffb700 1 -- 172.21.15.111:0/3978514842 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b000e070 con 0x7f93b81f5990 2023-03-29T10:58:36.951 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.951+0000 7f93adffb700 1 -- 172.21.15.111:0/3978514842 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b0003cf0 con 0x7f93b81f5990 2023-03-29T10:58:36.951 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.951+0000 7f93adffb700 1 -- 172.21.15.111:0/3978514842 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b00117a0 con 0x7f93b81f5990 2023-03-29T10:58:36.951 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.951+0000 7f93bf9ff700 1 -- 172.21.15.111:0/3978514842 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b80cd6a0 con 0x7f93b81f5990 2023-03-29T10:58:36.951 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.951+0000 7f93bf9ff700 1 -- 172.21.15.111:0/3978514842 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b81601e0 con 0x7f93b81f5990 2023-03-29T10:58:36.951 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.952+0000 7f93bf9ff700 1 -- 172.21.15.111:0/3978514842 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b80b52f0 con 0x7f93b81f5990 2023-03-29T10:58:36.952 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.953+0000 7f93adffb700 1 -- 172.21.15.111:0/3978514842 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f93b0011900 con 0x7f93b81f5990 2023-03-29T10:58:36.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.954+0000 7f93adffb700 1 --2- 172.21.15.111:0/3978514842 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f939c069de0 0x7f939c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:36.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.954+0000 7f93adffb700 1 -- 172.21.15.111:0/3978514842 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f93b00884e0 con 0x7f93b81f5990 2023-03-29T10:58:36.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.954+0000 7f93bd79b700 1 --2- 172.21.15.111:0/3978514842 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f939c069de0 0x7f939c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:36.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.954+0000 7f93bd79b700 1 --2- 172.21.15.111:0/3978514842 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f939c069de0 0x7f939c06c290 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f93b4005040 tx=0x7f93b4009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:36.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:36.958+0000 7f93adffb700 1 -- 172.21.15.111:0/3978514842 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f93b0053af0 con 0x7f93b81f5990 2023-03-29T10:58:37.148 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.149+0000 7f93bf9ff700 1 -- 172.21.15.111:0/3978514842 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f93b815bae0 con 0x7f93b81f5990 2023-03-29T10:58:37.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.149+0000 7f93adffb700 1 -- 172.21.15.111:0/3978514842 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f93b0057110 con 0x7f93b81f5990 2023-03-29T10:58:37.149 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:37.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.152+0000 7f93bf9ff700 1 -- 172.21.15.111:0/3978514842 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f939c069de0 msgr2=0x7f939c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:37.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.152+0000 7f93bf9ff700 1 --2- 172.21.15.111:0/3978514842 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f939c069de0 0x7f939c06c290 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f93b4005040 tx=0x7f93b4009030 comp rx=0 tx=0).stop 2023-03-29T10:58:37.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.152+0000 7f93bf9ff700 1 -- 172.21.15.111:0/3978514842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93b81f5990 msgr2=0x7f93b81f5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:37.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.152+0000 7f93bf9ff700 1 --2- 172.21.15.111:0/3978514842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93b81f5990 0x7f93b81f5e10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f93b808c150 tx=0x7f93b00111d0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.152+0000 7f93bf9ff700 1 -- 172.21.15.111:0/3978514842 shutdown_connections 2023-03-29T10:58:37.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.152+0000 7f93bf9ff700 1 --2- 172.21.15.111:0/3978514842 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f939c069de0 0x7f939c06c290 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.152+0000 7f93bf9ff700 1 --2- 172.21.15.111:0/3978514842 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93b81f5990 0x7f93b81f5e10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.152+0000 7f93bf9ff700 1 --2- 172.21.15.111:0/3978514842 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93a4003a50 0x7f93b8142fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.152+0000 7f93bf9ff700 1 --2- 172.21.15.111:0/3978514842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b8143d10 0x7f93b81434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.152+0000 7f93bf9ff700 1 -- 172.21.15.111:0/3978514842 >> 172.21.15.111:0/3978514842 conn(0x7f93b8071b10 msgr2=0x7f93b80b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:37.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.152+0000 7f93bf9ff700 1 -- 172.21.15.111:0/3978514842 shutdown_connections 2023-03-29T10:58:37.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.152+0000 7f93bf9ff700 1 -- 172.21.15.111:0/3978514842 wait complete. 2023-03-29T10:58:37.168 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-29T10:58:37.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.354+0000 7f30e73ff700 1 Processor -- start 2023-03-29T10:58:37.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.354+0000 7f30e73ff700 1 -- start start 2023-03-29T10:58:37.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.355+0000 7f30e73ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f30e00b87a0 0x7f30e00cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:37.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.355+0000 7f30e73ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e00cdc60 con 0x7f30e01601e0 2023-03-29T10:58:37.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.355+0000 7f30e73ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e00cd6a0 con 0x7f30e0160d40 2023-03-29T10:58:37.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.355+0000 7f30e73ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00d02d0 con 0x7f30e00b87a0 2023-03-29T10:58:37.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.356+0000 7f30e499a700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f30e01601e0 0x7f30e0160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43602/0 (socket says 172.21.15.111:43602) 2023-03-29T10:58:37.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.356+0000 7f30e499a700 1 -- 172.21.15.111:0/4284436211 learned_addr learned my addr 172.21.15.111:0/4284436211 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:37.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.356+0000 7f30e519b700 1 --2- 172.21.15.111:0/4284436211 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f30e00b87a0 0x7f30e00cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:37.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.356+0000 7f30d7fff700 1 -- 172.21.15.111:0/4284436211 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322258635 0 0) 0x7f30e00cdc60 con 0x7f30e01601e0 2023-03-29T10:58:37.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.356+0000 7f30d7fff700 1 -- 172.21.15.111:0/4284436211 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30cc0035b0 con 0x7f30e01601e0 2023-03-29T10:58:37.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.356+0000 7f30d7fff700 1 -- 172.21.15.111:0/4284436211 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2101944941 0 0) 0x7f30e00cd6a0 con 0x7f30e0160d40 2023-03-29T10:58:37.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.356+0000 7f30d7fff700 1 -- 172.21.15.111:0/4284436211 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30e00cdc60 con 0x7f30e0160d40 2023-03-29T10:58:37.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.356+0000 7f30d7fff700 1 -- 172.21.15.111:0/4284436211 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1427602087 0 0) 0x7f30cc0035b0 con 0x7f30e01601e0 2023-03-29T10:58:37.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.356+0000 7f30d7fff700 1 -- 172.21.15.111:0/4284436211 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f30e00b87a0 msgr2=0x7f30e00cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:37.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.356+0000 7f30d7fff700 1 --2- 172.21.15.111:0/4284436211 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f30e00b87a0 0x7f30e00cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.356+0000 7f30d7fff700 1 -- 172.21.15.111:0/4284436211 >> v1:172.21.15.119:6789/0 conn(0x7f30e0160d40 legacy=0x7f30e00d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:37.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.356+0000 7f30d7fff700 1 -- 172.21.15.111:0/4284436211 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e00cd980 con 0x7f30e01601e0 2023-03-29T10:58:37.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.356+0000 7f30d7fff700 1 -- 172.21.15.111:0/4284436211 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f30d0001670 con 0x7f30e01601e0 2023-03-29T10:58:37.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.356+0000 7f30e519b700 1 --2- 172.21.15.111:0/4284436211 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f30e00b87a0 0x7f30e00cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:58:37.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.357+0000 7f30d7fff700 1 -- 172.21.15.111:0/4284436211 >> v1:172.21.15.111:6789/0 conn(0x7f30e01601e0 legacy=0x7f30e0160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:37.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.357+0000 7f30d7fff700 1 --2- 172.21.15.111:0/4284436211 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f30e00b87a0 0x7f30cc003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:37.361 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.357+0000 7f30d7fff700 1 --2- 172.21.15.111:0/4284436211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30cc004150 0x7f30cc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:37.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.357+0000 7f30d7fff700 1 --2- 172.21.15.111:0/4284436211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc004d90 0x7f30cc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:37.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.357+0000 7f30d7fff700 1 -- 172.21.15.111:0/4284436211 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00d02d0 con 0x7f30cc004150 2023-03-29T10:58:37.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.357+0000 7f30d7fff700 1 -- 172.21.15.111:0/4284436211 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30cc00baf0 con 0x7f30cc004d90 2023-03-29T10:58:37.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.357+0000 7f30d7fff700 1 -- 172.21.15.111:0/4284436211 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30cc00bcb0 con 0x7f30e00b87a0 2023-03-29T10:58:37.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.357+0000 7f30e499a700 1 --2- 172.21.15.111:0/4284436211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30cc004150 0x7f30cc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:37.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.357+0000 7f30e519b700 1 --2- 172.21.15.111:0/4284436211 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f30e00b87a0 0x7f30cc003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:37.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.357+0000 7f30e599c700 1 --2- 172.21.15.111:0/4284436211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc004d90 0x7f30cc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:37.362 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.357+0000 7f30e499a700 1 -- 172.21.15.111:0/4284436211 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f30e00b87a0 msgr2=0x7f30cc003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:37.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.357+0000 7f30e499a700 1 --2- 172.21.15.111:0/4284436211 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f30e00b87a0 0x7f30cc003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.357+0000 7f30e499a700 1 -- 172.21.15.111:0/4284436211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc004d90 msgr2=0x7f30cc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:37.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.357+0000 7f30e499a700 1 --2- 172.21.15.111:0/4284436211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc004d90 0x7f30cc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.357+0000 7f30e499a700 1 -- 172.21.15.111:0/4284436211 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc0094a0 con 0x7f30cc004150 2023-03-29T10:58:37.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.358+0000 7f30e499a700 1 --2- 172.21.15.111:0/4284436211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30cc004150 0x7f30cc004850 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f30d000c110 tx=0x7f30cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49438f8ec035dee server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:37.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.358+0000 7f30d7fff700 1 -- 172.21.15.111:0/4284436211 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d000ccb0 con 0x7f30cc004150 2023-03-29T10:58:37.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.358+0000 7f30d7fff700 1 -- 172.21.15.111:0/4284436211 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d000ce10 con 0x7f30cc004150 2023-03-29T10:58:37.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.358+0000 7f30d7fff700 1 -- 172.21.15.111:0/4284436211 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0005ba0 con 0x7f30cc004150 2023-03-29T10:58:37.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.358+0000 7f30e73ff700 1 -- 172.21.15.111:0/4284436211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30cc004150 msgr2=0x7f30cc004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:37.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.358+0000 7f30e73ff700 1 --2- 172.21.15.111:0/4284436211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30cc004150 0x7f30cc004850 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f30d000c110 tx=0x7f30cc005880 comp rx=0 tx=0).stop 2023-03-29T10:58:37.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.358+0000 7f30e599c700 1 -- 172.21.15.111:0/4284436211 reap_dead start 2023-03-29T10:58:37.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.358+0000 7f30e73ff700 1 -- 172.21.15.111:0/4284436211 shutdown_connections 2023-03-29T10:58:37.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.358+0000 7f30e73ff700 1 -- 172.21.15.111:0/4284436211 >> 172.21.15.111:0/4284436211 conn(0x7f30e0071b10 msgr2=0x7f30e0071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:37.363 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.358+0000 7f30e73ff700 1 -- 172.21.15.111:0/4284436211 shutdown_connections 2023-03-29T10:58:37.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.358+0000 7f30e73ff700 1 -- 172.21.15.111:0/4284436211 wait complete. 2023-03-29T10:58:37.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.361+0000 7f30e73ff700 1 Processor -- start 2023-03-29T10:58:37.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.361+0000 7f30e73ff700 1 -- start start 2023-03-29T10:58:37.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.362+0000 7f30e73ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc004150 0x7f30e0154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:37.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.362+0000 7f30e73ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f30e015deb0 0x7f30e015e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:37.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.362+0000 7f30e73ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30e00d2780 0x7f30e00d2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:37.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.362+0000 7f30e73ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00d3160 con 0x7f30e00d2780 2023-03-29T10:58:37.365 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.362+0000 7f30e73ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e01548a0 con 0x7f30cc004150 2023-03-29T10:58:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.362+0000 7f30e73ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e015fa10 con 0x7f30e015deb0 2023-03-29T10:58:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.362+0000 7f30e599c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30e00d2780 0x7f30e00d2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.363+0000 7f30e599c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30e00d2780 0x7f30e00d2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44936/0 (socket says 172.21.15.111:44936) 2023-03-29T10:58:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.363+0000 7f30e599c700 1 -- 172.21.15.111:0/1309803737 learned_addr learned my addr 172.21.15.111:0/1309803737 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.363+0000 7f30e499a700 1 --2- 172.21.15.111:0/1309803737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f30e015deb0 0x7f30e015e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.363+0000 7f30e519b700 1 --2- 172.21.15.111:0/1309803737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc004150 0x7f30e0154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.363+0000 7f30e599c700 1 -- 172.21.15.111:0/1309803737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f30e015deb0 msgr2=0x7f30e015e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.363+0000 7f30e599c700 1 --2- 172.21.15.111:0/1309803737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f30e015deb0 0x7f30e015e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.363+0000 7f30e599c700 1 -- 172.21.15.111:0/1309803737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc004150 msgr2=0x7f30e0154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.363+0000 7f30e599c700 1 --2- 172.21.15.111:0/1309803737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc004150 0x7f30e0154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.363+0000 7f30e599c700 1 -- 172.21.15.111:0/1309803737 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e00cfd80 con 0x7f30e00d2780 2023-03-29T10:58:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.363+0000 7f30e599c700 1 --2- 172.21.15.111:0/1309803737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30e00d2780 0x7f30e00d2c20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f30e00d0820 tx=0x7f30d800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.363+0000 7f30d5ffb700 1 -- 172.21.15.111:0/1309803737 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d8003cd0 con 0x7f30e00d2780 2023-03-29T10:58:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.363+0000 7f30d5ffb700 1 -- 172.21.15.111:0/1309803737 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d8003e30 con 0x7f30e00d2780 2023-03-29T10:58:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.363+0000 7f30d5ffb700 1 -- 172.21.15.111:0/1309803737 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d800dbd0 con 0x7f30e00d2780 2023-03-29T10:58:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.363+0000 7f30e73ff700 1 -- 172.21.15.111:0/1309803737 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e00cfbf0 con 0x7f30e00d2780 2023-03-29T10:58:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.364+0000 7f30e73ff700 1 -- 172.21.15.111:0/1309803737 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e00cff10 con 0x7f30e00d2780 2023-03-29T10:58:37.366 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.365+0000 7f30e73ff700 1 -- 172.21.15.111:0/1309803737 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e00b52f0 con 0x7f30e00d2780 2023-03-29T10:58:37.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.365+0000 7f30d5ffb700 1 -- 172.21.15.111:0/1309803737 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f30d800dd30 con 0x7f30e00d2780 2023-03-29T10:58:37.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.366+0000 7f30d5ffb700 1 --2- 172.21.15.111:0/1309803737 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f30c4069de0 0x7f30c406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:37.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.366+0000 7f30d5ffb700 1 -- 172.21.15.111:0/1309803737 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f30d8088800 con 0x7f30e00d2780 2023-03-29T10:58:37.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.366+0000 7f30e519b700 1 --2- 172.21.15.111:0/1309803737 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f30c4069de0 0x7f30c406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:37.367 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.367+0000 7f30e519b700 1 --2- 172.21.15.111:0/1309803737 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f30c4069de0 0x7f30c406c290 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009c50 tx=0x7f30dc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:37.370 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.370+0000 7f30d5ffb700 1 -- 172.21.15.111:0/1309803737 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f30d8053c60 con 0x7f30e00d2780 2023-03-29T10:58:37.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.563+0000 7f30e73ff700 1 -- 172.21.15.111:0/1309803737 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f30e00b5e80 con 0x7f30e00d2780 2023-03-29T10:58:37.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.563+0000 7f30d5ffb700 1 -- 172.21.15.111:0/1309803737 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f30d8057280 con 0x7f30e00d2780 2023-03-29T10:58:37.563 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-03-29T10:58:37.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.566+0000 7f30e73ff700 1 -- 172.21.15.111:0/1309803737 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f30c4069de0 msgr2=0x7f30c406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:37.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.566+0000 7f30e73ff700 1 --2- 172.21.15.111:0/1309803737 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f30c4069de0 0x7f30c406c290 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009c50 tx=0x7f30dc008040 comp rx=0 tx=0).stop 2023-03-29T10:58:37.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.566+0000 7f30e73ff700 1 -- 172.21.15.111:0/1309803737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30e00d2780 msgr2=0x7f30e00d2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:37.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.566+0000 7f30e73ff700 1 --2- 172.21.15.111:0/1309803737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30e00d2780 0x7f30e00d2c20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f30e00d0820 tx=0x7f30d800d430 comp rx=0 tx=0).stop 2023-03-29T10:58:37.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.566+0000 7f30e73ff700 1 -- 172.21.15.111:0/1309803737 shutdown_connections 2023-03-29T10:58:37.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.566+0000 7f30e73ff700 1 --2- 172.21.15.111:0/1309803737 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f30c4069de0 0x7f30c406c290 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.566+0000 7f30e73ff700 1 --2- 172.21.15.111:0/1309803737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f30e00d2780 0x7f30e00d2c20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.566+0000 7f30e73ff700 1 --2- 172.21.15.111:0/1309803737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f30cc004150 0x7f30e0154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.566+0000 7f30e73ff700 1 --2- 172.21.15.111:0/1309803737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f30e015deb0 0x7f30e015e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.566+0000 7f30e73ff700 1 -- 172.21.15.111:0/1309803737 >> 172.21.15.111:0/1309803737 conn(0x7f30e0071b10 msgr2=0x7f30e00cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:37.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.566+0000 7f30e73ff700 1 -- 172.21.15.111:0/1309803737 shutdown_connections 2023-03-29T10:58:37.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.566+0000 7f30e73ff700 1 -- 172.21.15.111:0/1309803737 wait complete. 2023-03-29T10:58:37.582 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-29T10:58:37.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.770+0000 7f3b57fff700 1 Processor -- start 2023-03-29T10:58:37.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.770+0000 7f3b57fff700 1 -- start start 2023-03-29T10:58:37.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.771+0000 7f3b57fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3b500b87a0 0x7f3b500cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:37.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.771+0000 7f3b57fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b50160d40 0x7f3b500d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:37.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.771+0000 7f3b57fff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b500cdc60 con 0x7f3b501601e0 2023-03-29T10:58:37.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.771+0000 7f3b57fff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b500d02d0 con 0x7f3b500b87a0 2023-03-29T10:58:37.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.771+0000 7f3b57fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b500d0cb0 con 0x7f3b50160d40 2023-03-29T10:58:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.771+0000 7f3b5559a700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f3b501601e0 0x7f3b50160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:43608/0 (socket says 172.21.15.111:43608) 2023-03-29T10:58:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.771+0000 7f3b55d9b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3b500b87a0 0x7f3b500cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.771+0000 7f3b5559a700 1 -- 172.21.15.111:0/1899005505 learned_addr learned my addr 172.21.15.111:0/1899005505 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.771+0000 7f3b5659c700 1 --2- 172.21.15.111:0/1899005505 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b50160d40 0x7f3b500d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.772+0000 7f3b55d9b700 1 -- 172.21.15.111:0/1899005505 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b50160d40 msgr2=0x7f3b500d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.772+0000 7f3b55d9b700 1 --2- 172.21.15.111:0/1899005505 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b50160d40 0x7f3b500d1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.772+0000 7f3b55d9b700 1 -- 172.21.15.111:0/1899005505 >> v1:172.21.15.111:6789/0 conn(0x7f3b501601e0 legacy=0x7f3b50160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.772+0000 7f3b55d9b700 1 -- 172.21.15.111:0/1899005505 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b500cd6a0 con 0x7f3b500b87a0 2023-03-29T10:58:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.772+0000 7f3b54d99700 1 -- 172.21.15.111:0/1899005505 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2065709267 0 0) 0x7f3b500cdc60 con 0x7f3b501601e0 2023-03-29T10:58:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.772+0000 7f3b55d9b700 1 --2- 172.21.15.111:0/1899005505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b500b87a0 0x7f3b500cfbf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009840 tx=0x7f3b4c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8695a4557a5b139 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.772+0000 7f3b54d99700 1 -- 172.21.15.111:0/1899005505 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c011830 con 0x7f3b500b87a0 2023-03-29T10:58:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.772+0000 7f3b54d99700 1 -- 172.21.15.111:0/1899005505 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4c011990 con 0x7f3b500b87a0 2023-03-29T10:58:37.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.772+0000 7f3b54d99700 1 -- 172.21.15.111:0/1899005505 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c011d20 con 0x7f3b500b87a0 2023-03-29T10:58:37.773 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.773+0000 7f3b57fff700 1 -- 172.21.15.111:0/1899005505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b500b87a0 msgr2=0x7f3b500cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:37.773 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.773+0000 7f3b57fff700 1 --2- 172.21.15.111:0/1899005505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b500b87a0 0x7f3b500cfbf0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c009840 tx=0x7f3b4c009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:37.773 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.773+0000 7f3b57fff700 1 -- 172.21.15.111:0/1899005505 shutdown_connections 2023-03-29T10:58:37.773 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.773+0000 7f3b57fff700 1 --2- 172.21.15.111:0/1899005505 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b50160d40 0x7f3b500d1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.773 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.773+0000 7f3b57fff700 1 --2- 172.21.15.111:0/1899005505 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b500b87a0 0x7f3b500cfbf0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.773 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.773+0000 7f3b57fff700 1 -- 172.21.15.111:0/1899005505 >> 172.21.15.111:0/1899005505 conn(0x7f3b50071b10 msgr2=0x7f3b50071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:37.774 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.773+0000 7f3b57fff700 1 -- 172.21.15.111:0/1899005505 shutdown_connections 2023-03-29T10:58:37.774 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.773+0000 7f3b57fff700 1 -- 172.21.15.111:0/1899005505 wait complete. 2023-03-29T10:58:37.774 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.774+0000 7f3b57fff700 1 Processor -- start 2023-03-29T10:58:37.774 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.775+0000 7f3b57fff700 1 -- start start 2023-03-29T10:58:37.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.775+0000 7f3b57fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b500b87a0 0x7f3b50150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:37.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.776+0000 7f3b57fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b501601e0 0x7f3b500c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:37.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.776+0000 7f3b57fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b50160d40 0x7f3b5015cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:37.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.776+0000 7f3b57fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b500c9480 con 0x7f3b500b87a0 2023-03-29T10:58:37.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.776+0000 7f3b57fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b501e6480 con 0x7f3b50160d40 2023-03-29T10:58:37.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.776+0000 7f3b57fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b500c8ec0 con 0x7f3b501601e0 2023-03-29T10:58:37.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.776+0000 7f3b55d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b500b87a0 0x7f3b50150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:37.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.776+0000 7f3b55d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b500b87a0 0x7f3b50150cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:44956/0 (socket says 172.21.15.111:44956) 2023-03-29T10:58:37.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.776+0000 7f3b55d9b700 1 -- 172.21.15.111:0/309255351 learned_addr learned my addr 172.21.15.111:0/309255351 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:37.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.776+0000 7f3b5559a700 1 --2- 172.21.15.111:0/309255351 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b501601e0 0x7f3b500c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:37.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.776+0000 7f3b5659c700 1 --2- 172.21.15.111:0/309255351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b50160d40 0x7f3b5015cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:37.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.776+0000 7f3b55d9b700 1 -- 172.21.15.111:0/309255351 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b501601e0 msgr2=0x7f3b500c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:37.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.776+0000 7f3b55d9b700 1 --2- 172.21.15.111:0/309255351 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b501601e0 0x7f3b500c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.776+0000 7f3b55d9b700 1 -- 172.21.15.111:0/309255351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b50160d40 msgr2=0x7f3b5015cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:37.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.776+0000 7f3b55d9b700 1 --2- 172.21.15.111:0/309255351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b50160d40 0x7f3b5015cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.776+0000 7f3b55d9b700 1 -- 172.21.15.111:0/309255351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c0094a0 con 0x7f3b500b87a0 2023-03-29T10:58:37.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.777+0000 7f3b55d9b700 1 --2- 172.21.15.111:0/309255351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b500b87a0 0x7f3b50150cb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0090b0 tx=0x7f3b4c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:37.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.777+0000 7f3b467fc700 1 -- 172.21.15.111:0/309255351 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c0036a0 con 0x7f3b500b87a0 2023-03-29T10:58:37.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.777+0000 7f3b467fc700 1 -- 172.21.15.111:0/309255351 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4c01f070 con 0x7f3b500b87a0 2023-03-29T10:58:37.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.777+0000 7f3b57fff700 1 -- 172.21.15.111:0/309255351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b500d1640 con 0x7f3b500b87a0 2023-03-29T10:58:37.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.777+0000 7f3b467fc700 1 -- 172.21.15.111:0/309255351 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4c003a60 con 0x7f3b500b87a0 2023-03-29T10:58:37.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.777+0000 7f3b57fff700 1 -- 172.21.15.111:0/309255351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b500cd980 con 0x7f3b500b87a0 2023-03-29T10:58:37.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.778+0000 7f3b57fff700 1 -- 172.21.15.111:0/309255351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b500d17d0 con 0x7f3b500b87a0 2023-03-29T10:58:37.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.779+0000 7f3b467fc700 1 -- 172.21.15.111:0/309255351 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f3b4c003bc0 con 0x7f3b500b87a0 2023-03-29T10:58:37.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.780+0000 7f3b467fc700 1 --2- 172.21.15.111:0/309255351 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3b34069de0 0x7f3b3406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:37.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.780+0000 7f3b467fc700 1 -- 172.21.15.111:0/309255351 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3b4c021070 con 0x7f3b500b87a0 2023-03-29T10:58:37.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.780+0000 7f3b5559a700 1 --2- 172.21.15.111:0/309255351 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3b34069de0 0x7f3b3406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:37.780 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.780+0000 7f3b5559a700 1 --2- 172.21.15.111:0/309255351 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3b34069de0 0x7f3b3406c290 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3b501511f0 tx=0x7f3b40007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:37.783 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.784+0000 7f3b467fc700 1 -- 172.21.15.111:0/309255351 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f3b4c0552d0 con 0x7f3b500b87a0 2023-03-29T10:58:37.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.979+0000 7f3b57fff700 1 -- 172.21.15.111:0/309255351 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3b50074430 con 0x7f3b500b87a0 2023-03-29T10:58:37.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.979+0000 7f3b467fc700 1 -- 172.21.15.111:0/309255351 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3b4c0150a0 con 0x7f3b500b87a0 2023-03-29T10:58:37.979 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:37.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.981+0000 7f3b57fff700 1 -- 172.21.15.111:0/309255351 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3b34069de0 msgr2=0x7f3b3406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:37.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.981+0000 7f3b57fff700 1 --2- 172.21.15.111:0/309255351 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3b34069de0 0x7f3b3406c290 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3b501511f0 tx=0x7f3b40007de0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.981+0000 7f3b57fff700 1 -- 172.21.15.111:0/309255351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b500b87a0 msgr2=0x7f3b50150cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:37.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.981+0000 7f3b57fff700 1 --2- 172.21.15.111:0/309255351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b500b87a0 0x7f3b50150cb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0090b0 tx=0x7f3b4c004750 comp rx=0 tx=0).stop 2023-03-29T10:58:37.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.981+0000 7f3b57fff700 1 -- 172.21.15.111:0/309255351 shutdown_connections 2023-03-29T10:58:37.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.981+0000 7f3b57fff700 1 --2- 172.21.15.111:0/309255351 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3b34069de0 0x7f3b3406c290 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.981+0000 7f3b57fff700 1 --2- 172.21.15.111:0/309255351 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b500b87a0 0x7f3b50150cb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.981+0000 7f3b57fff700 1 --2- 172.21.15.111:0/309255351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b50160d40 0x7f3b5015cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.982+0000 7f3b57fff700 1 --2- 172.21.15.111:0/309255351 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b501601e0 0x7f3b500c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:37.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.982+0000 7f3b57fff700 1 -- 172.21.15.111:0/309255351 >> 172.21.15.111:0/309255351 conn(0x7f3b50071b10 msgr2=0x7f3b500d1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:37.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.982+0000 7f3b57fff700 1 -- 172.21.15.111:0/309255351 shutdown_connections 2023-03-29T10:58:37.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:37.982+0000 7f3b57fff700 1 -- 172.21.15.111:0/309255351 wait complete. 2023-03-29T10:58:37.998 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-03-29T10:58:38.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.184+0000 7fbf439ff700 1 Processor -- start 2023-03-29T10:58:38.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.185+0000 7fbf439ff700 1 -- start start 2023-03-29T10:58:38.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.186+0000 7fbf439ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbf3c0aff90 0x7fbf3c0d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:38.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.186+0000 7fbf439ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbf3c0d0a30 0x7fbf3c0cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:38.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.186+0000 7fbf439ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbf3c0cef40 0x7fbf3c06b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:38.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.186+0000 7fbf439ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c15cef0 con 0x7fbf3c0d0a30 2023-03-29T10:58:38.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.186+0000 7fbf439ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c0cf450 con 0x7fbf3c0aff90 2023-03-29T10:58:38.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.186+0000 7fbf439ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0cf730 con 0x7fbf3c0cef40 2023-03-29T10:58:38.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.186+0000 7fbf40f9a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbf3c0d0a30 0x7fbf3c0cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:38.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.186+0000 7fbf40f9a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbf3c0d0a30 0x7fbf3c0cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58868/0 (socket says 172.21.15.111:58868) 2023-03-29T10:58:38.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.186+0000 7fbf40f9a700 1 -- 172.21.15.111:0/4055375671 learned_addr learned my addr 172.21.15.111:0/4055375671 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:38.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.186+0000 7fbf41f9c700 1 --2- 172.21.15.111:0/4055375671 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbf3c0cef40 0x7fbf3c06b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:38.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.186+0000 7fbf4179b700 1 --2- 172.21.15.111:0/4055375671 >> v2:172.21.15.119:3300/0 conn(0x7fbf3c0aff90 0x7fbf3c0d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:38.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.186+0000 7fbf40f9a700 1 -- 172.21.15.111:0/4055375671 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbf3c0cef40 msgr2=0x7fbf3c06b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:38.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.186+0000 7fbf40f9a700 1 --2- 172.21.15.111:0/4055375671 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbf3c0cef40 0x7fbf3c06b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.186+0000 7fbf40f9a700 1 -- 172.21.15.111:0/4055375671 >> v2:172.21.15.119:3300/0 conn(0x7fbf3c0aff90 msgr2=0x7fbf3c0d04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:38.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.186+0000 7fbf40f9a700 1 --2- 172.21.15.111:0/4055375671 >> v2:172.21.15.119:3300/0 conn(0x7fbf3c0aff90 0x7fbf3c0d04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.186+0000 7fbf40f9a700 1 -- 172.21.15.111:0/4055375671 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c0cfa10 con 0x7fbf3c0d0a30 2023-03-29T10:58:38.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.187+0000 7fbf40f9a700 1 --2- 172.21.15.111:0/4055375671 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf3c0d0a30 0x7fbf3c0cea00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c071ee0 tx=0x7fbf2c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592bf20d43924278 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:38.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.187+0000 7fbf33fff700 1 -- 172.21.15.111:0/4055375671 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c011830 con 0x7fbf3c0d0a30 2023-03-29T10:58:38.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.187+0000 7fbf33fff700 1 -- 172.21.15.111:0/4055375671 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2c011990 con 0x7fbf3c0d0a30 2023-03-29T10:58:38.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.187+0000 7fbf33fff700 1 -- 172.21.15.111:0/4055375671 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c011d20 con 0x7fbf3c0d0a30 2023-03-29T10:58:38.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.187+0000 7fbf439ff700 1 -- 172.21.15.111:0/4055375671 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf3c0d0a30 msgr2=0x7fbf3c0cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:38.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.187+0000 7fbf439ff700 1 --2- 172.21.15.111:0/4055375671 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf3c0d0a30 0x7fbf3c0cea00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c071ee0 tx=0x7fbf2c009bd0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.187+0000 7fbf439ff700 1 -- 172.21.15.111:0/4055375671 shutdown_connections 2023-03-29T10:58:38.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.187+0000 7fbf439ff700 1 --2- 172.21.15.111:0/4055375671 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbf3c0cef40 0x7fbf3c06b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.187+0000 7fbf439ff700 1 --2- 172.21.15.111:0/4055375671 >> v2:172.21.15.119:3300/0 conn(0x7fbf3c0aff90 0x7fbf3c0d04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.187+0000 7fbf439ff700 1 --2- 172.21.15.111:0/4055375671 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf3c0d0a30 0x7fbf3c0cea00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.187+0000 7fbf439ff700 1 -- 172.21.15.111:0/4055375671 >> 172.21.15.111:0/4055375671 conn(0x7fbf3c09a100 msgr2=0x7fbf3c0d0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:38.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.188+0000 7fbf439ff700 1 -- 172.21.15.111:0/4055375671 shutdown_connections 2023-03-29T10:58:38.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.188+0000 7fbf439ff700 1 -- 172.21.15.111:0/4055375671 wait complete. 2023-03-29T10:58:38.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.189+0000 7fbf439ff700 1 Processor -- start 2023-03-29T10:58:38.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.189+0000 7fbf439ff700 1 -- start start 2023-03-29T10:58:38.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.190+0000 7fbf439ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf3c0aff90 0x7fbf3c1583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.190+0000 7fbf439ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf3c0cef40 0x7fbf3c0b6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.190+0000 7fbf439ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbf3c0d0a30 0x7fbf3c0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.190+0000 7fbf439ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0c9d30 con 0x7fbf3c0aff90 2023-03-29T10:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.190+0000 7fbf439ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c158910 con 0x7fbf3c0cef40 2023-03-29T10:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.190+0000 7fbf439ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0b60a0 con 0x7fbf3c0d0a30 2023-03-29T10:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.190+0000 7fbf4179b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf3c0aff90 0x7fbf3c1583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.190+0000 7fbf4179b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf3c0aff90 0x7fbf3c1583d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58884/0 (socket says 172.21.15.111:58884) 2023-03-29T10:58:38.192 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.191+0000 7fbf4179b700 1 -- 172.21.15.111:0/3814195331 learned_addr learned my addr 172.21.15.111:0/3814195331 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:38.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.191+0000 7fbf41f9c700 1 --2- 172.21.15.111:0/3814195331 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbf3c0d0a30 0x7fbf3c0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:38.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.191+0000 7fbf40f9a700 1 --2- 172.21.15.111:0/3814195331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf3c0cef40 0x7fbf3c0b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:38.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.191+0000 7fbf4179b700 1 -- 172.21.15.111:0/3814195331 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbf3c0d0a30 msgr2=0x7fbf3c0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:38.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.191+0000 7fbf4179b700 1 --2- 172.21.15.111:0/3814195331 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbf3c0d0a30 0x7fbf3c0b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.191+0000 7fbf4179b700 1 -- 172.21.15.111:0/3814195331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf3c0cef40 msgr2=0x7fbf3c0b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:38.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.191+0000 7fbf4179b700 1 --2- 172.21.15.111:0/3814195331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf3c0cef40 0x7fbf3c0b6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.191+0000 7fbf4179b700 1 -- 172.21.15.111:0/3814195331 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c009550 con 0x7fbf3c0aff90 2023-03-29T10:58:38.193 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.191+0000 7fbf4179b700 1 --2- 172.21.15.111:0/3814195331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf3c0aff90 0x7fbf3c1583d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800e930 tx=0x7fbf3800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:38.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.191+0000 7fbf31ffb700 1 -- 172.21.15.111:0/3814195331 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38013680 con 0x7fbf3c0aff90 2023-03-29T10:58:38.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.191+0000 7fbf31ffb700 1 -- 172.21.15.111:0/3814195331 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf380137e0 con 0x7fbf3c0aff90 2023-03-29T10:58:38.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.191+0000 7fbf31ffb700 1 -- 172.21.15.111:0/3814195331 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf38013b70 con 0x7fbf3c0aff90 2023-03-29T10:58:38.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.191+0000 7fbf439ff700 1 -- 172.21.15.111:0/3814195331 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3c0cffd0 con 0x7fbf3c0aff90 2023-03-29T10:58:38.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.192+0000 7fbf439ff700 1 -- 172.21.15.111:0/3814195331 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3c0cfcf0 con 0x7fbf3c0aff90 2023-03-29T10:58:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.193+0000 7fbf439ff700 1 -- 172.21.15.111:0/3814195331 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3c074430 con 0x7fbf3c0aff90 2023-03-29T10:58:38.195 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.193+0000 7fbf31ffb700 1 -- 172.21.15.111:0/3814195331 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fbf38013cd0 con 0x7fbf3c0aff90 2023-03-29T10:58:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.194+0000 7fbf31ffb700 1 --2- 172.21.15.111:0/3814195331 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbf20069de0 0x7fbf2006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.194+0000 7fbf31ffb700 1 -- 172.21.15.111:0/3814195331 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbf380886d0 con 0x7fbf3c0aff90 2023-03-29T10:58:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.195+0000 7fbf40f9a700 1 --2- 172.21.15.111:0/3814195331 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbf20069de0 0x7fbf2006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:38.196 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.195+0000 7fbf40f9a700 1 --2- 172.21.15.111:0/3814195331 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbf20069de0 0x7fbf2006c290 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0d0e00 tx=0x7fbf2c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:38.198 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.198+0000 7fbf31ffb700 1 -- 172.21.15.111:0/3814195331 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fbf38053ac0 con 0x7fbf3c0aff90 2023-03-29T10:58:38.404 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.404+0000 7fbf439ff700 1 -- 172.21.15.111:0/3814195331 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbf3c0b1750 con 0x7fbf3c0aff90 2023-03-29T10:58:38.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.405+0000 7fbf31ffb700 1 -- 172.21.15.111:0/3814195331 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbf380570e0 con 0x7fbf3c0aff90 2023-03-29T10:58:38.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.407+0000 7fbf439ff700 1 -- 172.21.15.111:0/3814195331 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbf20069de0 msgr2=0x7fbf2006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:38.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.407+0000 7fbf439ff700 1 --2- 172.21.15.111:0/3814195331 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbf20069de0 0x7fbf2006c290 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0d0e00 tx=0x7fbf2c008030 comp rx=0 tx=0).stop 2023-03-29T10:58:38.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.407+0000 7fbf439ff700 1 -- 172.21.15.111:0/3814195331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf3c0aff90 msgr2=0x7fbf3c1583d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:38.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.407+0000 7fbf439ff700 1 --2- 172.21.15.111:0/3814195331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf3c0aff90 0x7fbf3c1583d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbf3800e930 tx=0x7fbf3800ec40 comp rx=0 tx=0).stop 2023-03-29T10:58:38.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.407+0000 7fbf439ff700 1 -- 172.21.15.111:0/3814195331 shutdown_connections 2023-03-29T10:58:38.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.407+0000 7fbf439ff700 1 --2- 172.21.15.111:0/3814195331 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbf20069de0 0x7fbf2006c290 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.407+0000 7fbf439ff700 1 --2- 172.21.15.111:0/3814195331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbf3c0aff90 0x7fbf3c1583d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.407+0000 7fbf439ff700 1 --2- 172.21.15.111:0/3814195331 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbf3c0d0a30 0x7fbf3c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.407+0000 7fbf439ff700 1 --2- 172.21.15.111:0/3814195331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf3c0cef40 0x7fbf3c0b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.407+0000 7fbf439ff700 1 -- 172.21.15.111:0/3814195331 >> 172.21.15.111:0/3814195331 conn(0x7fbf3c09a100 msgr2=0x7fbf3c0704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:38.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.407+0000 7fbf439ff700 1 -- 172.21.15.111:0/3814195331 shutdown_connections 2023-03-29T10:58:38.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.407+0000 7fbf439ff700 1 -- 172.21.15.111:0/3814195331 wait complete. 2023-03-29T10:58:38.408 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-03-29T10:58:38.424 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:58:38.424 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-29T10:58:38.471 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:58:38.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.658+0000 7fd4b19ff700 1 Processor -- start 2023-03-29T10:58:38.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.659+0000 7fd4b19ff700 1 -- start start 2023-03-29T10:58:38.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.660+0000 7fd4b19ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd4ac0ce100 0x7fd4ac0cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:38.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.660+0000 7fd4b19ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ac0b0fc0 con 0x7fd4ac1594c0 2023-03-29T10:58:38.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.660+0000 7fd4b19ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ac0cc9b0 con 0x7fd4ac15dce0 2023-03-29T10:58:38.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.660+0000 7fd4b19ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ac0b0dd0 con 0x7fd4ac0ce100 2023-03-29T10:58:38.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.660+0000 7fd4ab7fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd4ac0ce100 0x7fd4ac0cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:38.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.660+0000 7fd4aaffd700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fd4ac1594c0 0x7fd4ac0b06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54516/0 (socket says 172.21.15.111:54516) 2023-03-29T10:58:38.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.660+0000 7fd4aaffd700 1 -- 172.21.15.111:0/3790990095 learned_addr learned my addr 172.21.15.111:0/3790990095 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:38.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.660+0000 7fd4aa7fc700 1 -- 172.21.15.111:0/3790990095 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441873961 0 0) 0x7fd4ac0b0fc0 con 0x7fd4ac1594c0 2023-03-29T10:58:38.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.661+0000 7fd4aa7fc700 1 -- 172.21.15.111:0/3790990095 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4980057c0 con 0x7fd4ac1594c0 2023-03-29T10:58:38.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.661+0000 7fd4ab7fe700 1 -- 172.21.15.111:0/3790990095 >> v1:172.21.15.119:6789/0 conn(0x7fd4ac15dce0 legacy=0x7fd4ac0cd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:38.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.661+0000 7fd4ab7fe700 1 -- 172.21.15.111:0/3790990095 >> v1:172.21.15.111:6789/0 conn(0x7fd4ac1594c0 legacy=0x7fd4ac0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:38.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.661+0000 7fd4ab7fe700 1 -- 172.21.15.111:0/3790990095 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac0ccc90 con 0x7fd4ac0ce100 2023-03-29T10:58:38.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.661+0000 7fd4ab7fe700 1 --2- 172.21.15.111:0/3790990095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac0ce100 0x7fd4ac0cd470 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009d90 tx=0x7fd4a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7656be8398921e24 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:38.662 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.661+0000 7fd4aa7fc700 1 -- 172.21.15.111:0/3790990095 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a400e070 con 0x7fd4ac0ce100 2023-03-29T10:58:38.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.661+0000 7fd4aa7fc700 1 -- 172.21.15.111:0/3790990095 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a400bb20 con 0x7fd4ac0ce100 2023-03-29T10:58:38.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.661+0000 7fd4aa7fc700 1 -- 172.21.15.111:0/3790990095 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4012430 con 0x7fd4ac0ce100 2023-03-29T10:58:38.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.662+0000 7fd4b19ff700 1 -- 172.21.15.111:0/3790990095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac0ce100 msgr2=0x7fd4ac0cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:38.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.662+0000 7fd4b19ff700 1 --2- 172.21.15.111:0/3790990095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac0ce100 0x7fd4ac0cd470 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4009d90 tx=0x7fd4a400b330 comp rx=0 tx=0).stop 2023-03-29T10:58:38.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.662+0000 7fd4b19ff700 1 -- 172.21.15.111:0/3790990095 shutdown_connections 2023-03-29T10:58:38.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.662+0000 7fd4b19ff700 1 --2- 172.21.15.111:0/3790990095 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac0ce100 0x7fd4ac0cd470 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.662+0000 7fd4b19ff700 1 -- 172.21.15.111:0/3790990095 >> 172.21.15.111:0/3790990095 conn(0x7fd4ac0cb6c0 msgr2=0x7fd4ac0cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:38.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.662+0000 7fd4b19ff700 1 -- 172.21.15.111:0/3790990095 shutdown_connections 2023-03-29T10:58:38.663 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.662+0000 7fd4b19ff700 1 -- 172.21.15.111:0/3790990095 wait complete. 2023-03-29T10:58:38.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.663+0000 7fd4b19ff700 1 Processor -- start 2023-03-29T10:58:38.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.664+0000 7fd4b19ff700 1 -- start start 2023-03-29T10:58:38.666 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.664+0000 7fd4b19ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4ac0ce100 0x7fd4ac071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:38.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.664+0000 7fd4b19ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac1594c0 0x7fd4ac0c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:38.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.664+0000 7fd4b19ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4ac15dce0 0x7fd4ac0c4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:38.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.665+0000 7fd4b19ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac159bb0 con 0x7fd4ac1594c0 2023-03-29T10:58:38.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.665+0000 7fd4b19ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac1f2270 con 0x7fd4ac15dce0 2023-03-29T10:58:38.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.665+0000 7fd4b19ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac159e00 con 0x7fd4ac0ce100 2023-03-29T10:58:38.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.665+0000 7fd4a3fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac1594c0 0x7fd4ac0c3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:38.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.665+0000 7fd4a3fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac1594c0 0x7fd4ac0c3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58900/0 (socket says 172.21.15.111:58900) 2023-03-29T10:58:38.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.665+0000 7fd4a3fff700 1 -- 172.21.15.111:0/1940978736 learned_addr learned my addr 172.21.15.111:0/1940978736 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:38.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.665+0000 7fd4aaffd700 1 --2- 172.21.15.111:0/1940978736 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4ac0ce100 0x7fd4ac071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:38.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.665+0000 7fd4ab7fe700 1 --2- 172.21.15.111:0/1940978736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4ac15dce0 0x7fd4ac0c4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:38.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.665+0000 7fd4a3fff700 1 -- 172.21.15.111:0/1940978736 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4ac0ce100 msgr2=0x7fd4ac071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:38.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.665+0000 7fd4a3fff700 1 --2- 172.21.15.111:0/1940978736 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4ac0ce100 0x7fd4ac071470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.665+0000 7fd4a3fff700 1 -- 172.21.15.111:0/1940978736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4ac15dce0 msgr2=0x7fd4ac0c4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:38.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.665+0000 7fd4a3fff700 1 --2- 172.21.15.111:0/1940978736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4ac15dce0 0x7fd4ac0c4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.665+0000 7fd4a3fff700 1 -- 172.21.15.111:0/1940978736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a4009a70 con 0x7fd4ac1594c0 2023-03-29T10:58:38.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.665+0000 7fd4ab7fe700 1 --2- 172.21.15.111:0/1940978736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4ac15dce0 0x7fd4ac0c4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:38.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.665+0000 7fd4a3fff700 1 --2- 172.21.15.111:0/1940978736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac1594c0 0x7fd4ac0c3d10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd49400c430 tx=0x7fd49400c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:38.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.666+0000 7fd4a8ff9700 1 -- 172.21.15.111:0/1940978736 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4940052d0 con 0x7fd4ac1594c0 2023-03-29T10:58:38.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.666+0000 7fd4a8ff9700 1 -- 172.21.15.111:0/1940978736 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd494005430 con 0x7fd4ac1594c0 2023-03-29T10:58:38.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.666+0000 7fd4a8ff9700 1 -- 172.21.15.111:0/1940978736 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4940038c0 con 0x7fd4ac1594c0 2023-03-29T10:58:38.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.666+0000 7fd4b19ff700 1 -- 172.21.15.111:0/1940978736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ac159890 con 0x7fd4ac1594c0 2023-03-29T10:58:38.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.666+0000 7fd4b19ff700 1 -- 172.21.15.111:0/1940978736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac0ccf70 con 0x7fd4ac1594c0 2023-03-29T10:58:38.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.667+0000 7fd4b19ff700 1 -- 172.21.15.111:0/1940978736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ac159a20 con 0x7fd4ac1594c0 2023-03-29T10:58:38.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.668+0000 7fd4a8ff9700 1 -- 172.21.15.111:0/1940978736 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fd494003a20 con 0x7fd4ac1594c0 2023-03-29T10:58:38.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.669+0000 7fd4a8ff9700 1 --2- 172.21.15.111:0/1940978736 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd48c069de0 0x7fd48c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:38.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.669+0000 7fd4a8ff9700 1 -- 172.21.15.111:0/1940978736 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd494087640 con 0x7fd4ac1594c0 2023-03-29T10:58:38.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.669+0000 7fd4aaffd700 1 --2- 172.21.15.111:0/1940978736 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd48c069de0 0x7fd48c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:38.670 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.669+0000 7fd4aaffd700 1 --2- 172.21.15.111:0/1940978736 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd48c069de0 0x7fd48c06c290 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd49c007fd0 tx=0x7fd49c007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:38.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.672+0000 7fd4a8ff9700 1 -- 172.21.15.111:0/1940978736 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fd494052bc0 con 0x7fd4ac1594c0 2023-03-29T10:58:38.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.870+0000 7fd4b19ff700 1 -- 172.21.15.111:0/1940978736 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4ac0cc2e0 con 0x7fd4ac1594c0 2023-03-29T10:58:38.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.872+0000 7fd4a8ff9700 1 -- 172.21.15.111:0/1940978736 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19892 (secure 0 0 0) 0x7fd4940561e0 con 0x7fd4ac1594c0 2023-03-29T10:58:38.872 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:38.872 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":30,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:58:09.476334+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.26","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:58:38.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.874+0000 7fd4b19ff700 1 -- 172.21.15.111:0/1940978736 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd48c069de0 msgr2=0x7fd48c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:38.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.874+0000 7fd4b19ff700 1 --2- 172.21.15.111:0/1940978736 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd48c069de0 0x7fd48c06c290 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd49c007fd0 tx=0x7fd49c007dc0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.874+0000 7fd4b19ff700 1 -- 172.21.15.111:0/1940978736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac1594c0 msgr2=0x7fd4ac0c3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:38.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.874+0000 7fd4b19ff700 1 --2- 172.21.15.111:0/1940978736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac1594c0 0x7fd4ac0c3d10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd49400c430 tx=0x7fd49400c740 comp rx=0 tx=0).stop 2023-03-29T10:58:38.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.874+0000 7fd4b19ff700 1 -- 172.21.15.111:0/1940978736 shutdown_connections 2023-03-29T10:58:38.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.874+0000 7fd4b19ff700 1 --2- 172.21.15.111:0/1940978736 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd48c069de0 0x7fd48c06c290 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.874+0000 7fd4b19ff700 1 --2- 172.21.15.111:0/1940978736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4ac15dce0 0x7fd4ac0c4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.874 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.874+0000 7fd4b19ff700 1 --2- 172.21.15.111:0/1940978736 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd4ac0ce100 0x7fd4ac071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.874+0000 7fd4b19ff700 1 --2- 172.21.15.111:0/1940978736 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd4ac1594c0 0x7fd4ac0c3d10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:38.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.874+0000 7fd4b19ff700 1 -- 172.21.15.111:0/1940978736 >> 172.21.15.111:0/1940978736 conn(0x7fd4ac0cb6c0 msgr2=0x7fd4ac0cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:38.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.874+0000 7fd4b19ff700 1 -- 172.21.15.111:0/1940978736 shutdown_connections 2023-03-29T10:58:38.875 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:38.874+0000 7fd4b19ff700 1 -- 172.21.15.111:0/1940978736 wait complete. 2023-03-29T10:58:38.891 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-29T10:57:16.183979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-29T10:57:21.239809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-29T10:57:26.192124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-29T10:57:26.796777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-29T10:57:29.874795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-29T10:58:38.892 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-29T10:58:39.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.078+0000 7f8476bff700 1 Processor -- start 2023-03-29T10:58:39.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.078+0000 7f8476bff700 1 -- start start 2023-03-29T10:58:39.079 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.079+0000 7f8476bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f84700cfbf0 0x7f84701608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.079 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.079+0000 7f8476bff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f84700d1100 0x7f84700d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.079 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.079+0000 7f8476bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84700d0020 con 0x7f84700b87a0 2023-03-29T10:58:39.079 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.079+0000 7f8476bff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84700d02d0 con 0x7f84700d1100 2023-03-29T10:58:39.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.079+0000 7f8476bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84700d0cb0 con 0x7f84700cfbf0 2023-03-29T10:58:39.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.080+0000 7f847519c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f84700d1100 0x7f84700d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.080+0000 7f847519c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f84700d1100 0x7f84700d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58904/0 (socket says 172.21.15.111:58904) 2023-03-29T10:58:39.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.080+0000 7f847519c700 1 -- 172.21.15.111:0/2553718030 learned_addr learned my addr 172.21.15.111:0/2553718030 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:39.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.080+0000 7f846ffff700 1 --2- 172.21.15.111:0/2553718030 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f84700cfbf0 0x7f84701608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.080+0000 7f847519c700 1 -- 172.21.15.111:0/2553718030 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f84700cfbf0 msgr2=0x7f84701608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:39.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.080+0000 7f847519c700 1 --2- 172.21.15.111:0/2553718030 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f84700cfbf0 0x7f84701608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.080+0000 7f847519c700 1 -- 172.21.15.111:0/2553718030 >> v1:172.21.15.119:6789/0 conn(0x7f84700b87a0 legacy=0x7f84701601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:39.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.080+0000 7f847519c700 1 -- 172.21.15.111:0/2553718030 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84700cdc60 con 0x7f84700d1100 2023-03-29T10:58:39.080 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.080+0000 7f846ffff700 1 --2- 172.21.15.111:0/2553718030 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f84700cfbf0 0x7f84701608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:39.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.080+0000 7f847519c700 1 --2- 172.21.15.111:0/2553718030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84700d1100 0x7f84700d1580 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8468009e50 tx=0x7f846800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb07ef83c005a03 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:39.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.080+0000 7f846f7fe700 1 -- 172.21.15.111:0/2553718030 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846800e070 con 0x7f84700d1100 2023-03-29T10:58:39.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.080+0000 7f846f7fe700 1 -- 172.21.15.111:0/2553718030 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846800b9c0 con 0x7f84700d1100 2023-03-29T10:58:39.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.080+0000 7f846f7fe700 1 -- 172.21.15.111:0/2553718030 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846800bd70 con 0x7f84700d1100 2023-03-29T10:58:39.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.081+0000 7f8476bff700 1 -- 172.21.15.111:0/2553718030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84700d1100 msgr2=0x7f84700d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:39.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.081+0000 7f8476bff700 1 --2- 172.21.15.111:0/2553718030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84700d1100 0x7f84700d1580 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8468009e50 tx=0x7f846800b330 comp rx=0 tx=0).stop 2023-03-29T10:58:39.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.081+0000 7f8476bff700 1 -- 172.21.15.111:0/2553718030 shutdown_connections 2023-03-29T10:58:39.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.081+0000 7f8476bff700 1 --2- 172.21.15.111:0/2553718030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84700d1100 0x7f84700d1580 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.081+0000 7f8476bff700 1 --2- 172.21.15.111:0/2553718030 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f84700cfbf0 0x7f84701608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.081 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.081+0000 7f8476bff700 1 -- 172.21.15.111:0/2553718030 >> 172.21.15.111:0/2553718030 conn(0x7f8470071b10 msgr2=0x7f8470071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:39.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.081+0000 7f8476bff700 1 -- 172.21.15.111:0/2553718030 shutdown_connections 2023-03-29T10:58:39.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.081+0000 7f8476bff700 1 -- 172.21.15.111:0/2553718030 wait complete. 2023-03-29T10:58:39.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.083+0000 7f8476bff700 1 Processor -- start 2023-03-29T10:58:39.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.083+0000 7f8476bff700 1 -- start start 2023-03-29T10:58:39.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.084+0000 7f8476bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f84700b87a0 0x7f84700c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.084+0000 7f8476bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84700cfbf0 0x7f84700b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.084+0000 7f8476bff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84700d1100 0x7f847014fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.084+0000 7f8476bff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84700cd4f0 con 0x7f84700d1100 2023-03-29T10:58:39.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.084+0000 7f8476bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84701f2670 con 0x7f84700cfbf0 2023-03-29T10:58:39.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.084+0000 7f8476bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84700ccc90 con 0x7f84700b87a0 2023-03-29T10:58:39.084 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.084+0000 7f847519c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84700d1100 0x7f847014fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.084+0000 7f847519c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84700d1100 0x7f847014fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58916/0 (socket says 172.21.15.111:58916) 2023-03-29T10:58:39.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.084+0000 7f847519c700 1 -- 172.21.15.111:0/241178458 learned_addr learned my addr 172.21.15.111:0/241178458 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:39.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.084+0000 7f846ffff700 1 --2- 172.21.15.111:0/241178458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84700cfbf0 0x7f84700b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.084+0000 7f847499b700 1 --2- 172.21.15.111:0/241178458 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f84700b87a0 0x7f84700c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.084+0000 7f847519c700 1 -- 172.21.15.111:0/241178458 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f84700b87a0 msgr2=0x7f84700c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:39.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.084+0000 7f847519c700 1 --2- 172.21.15.111:0/241178458 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f84700b87a0 0x7f84700c73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.084+0000 7f847519c700 1 -- 172.21.15.111:0/241178458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84700cfbf0 msgr2=0x7f84700b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:39.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.084+0000 7f847519c700 1 --2- 172.21.15.111:0/241178458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84700cfbf0 0x7f84700b2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.084+0000 7f847519c700 1 -- 172.21.15.111:0/241178458 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8468009b30 con 0x7f84700d1100 2023-03-29T10:58:39.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.085+0000 7f847519c700 1 --2- 172.21.15.111:0/241178458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84700d1100 0x7f847014fed0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8470072760 tx=0x7f8468004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:39.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.085+0000 7f846d7fa700 1 -- 172.21.15.111:0/241178458 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8468004800 con 0x7f84700d1100 2023-03-29T10:58:39.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.085+0000 7f846d7fa700 1 -- 172.21.15.111:0/241178458 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8468004960 con 0x7f84700d1100 2023-03-29T10:58:39.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.085+0000 7f846d7fa700 1 -- 172.21.15.111:0/241178458 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846801c680 con 0x7f84700d1100 2023-03-29T10:58:39.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.085+0000 7f8476bff700 1 -- 172.21.15.111:0/241178458 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84700cd980 con 0x7f84700d1100 2023-03-29T10:58:39.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.085+0000 7f8476bff700 1 -- 172.21.15.111:0/241178458 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84700cd6a0 con 0x7f84700d1100 2023-03-29T10:58:39.086 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.086+0000 7f8476bff700 1 -- 172.21.15.111:0/241178458 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8470160e30 con 0x7f84700d1100 2023-03-29T10:58:39.086 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.087+0000 7f846d7fa700 1 -- 172.21.15.111:0/241178458 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f846801c7e0 con 0x7f84700d1100 2023-03-29T10:58:39.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.088+0000 7f846d7fa700 1 --2- 172.21.15.111:0/241178458 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8454069de0 0x7f845406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.088+0000 7f846d7fa700 1 -- 172.21.15.111:0/241178458 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8468089e70 con 0x7f84700d1100 2023-03-29T10:58:39.088 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.088+0000 7f847499b700 1 --2- 172.21.15.111:0/241178458 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8454069de0 0x7f845406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.088 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.089+0000 7f847499b700 1 --2- 172.21.15.111:0/241178458 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8454069de0 0x7f845406c290 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8464007fd0 tx=0x7f8464009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:39.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.092+0000 7f846d7fa700 1 -- 172.21.15.111:0/241178458 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f84680553e0 con 0x7f84700d1100 2023-03-29T10:58:39.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.287+0000 7f8476bff700 1 -- 172.21.15.111:0/241178458 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f847006d2c0 con 0x7f84700d1100 2023-03-29T10:58:39.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.287+0000 7f846d7fa700 1 -- 172.21.15.111:0/241178458 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8468058a00 con 0x7f84700d1100 2023-03-29T10:58:39.287 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-03-29T10:58:39.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.289+0000 7f8476bff700 1 -- 172.21.15.111:0/241178458 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8454069de0 msgr2=0x7f845406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:39.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.289+0000 7f8476bff700 1 --2- 172.21.15.111:0/241178458 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8454069de0 0x7f845406c290 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8464007fd0 tx=0x7f8464009040 comp rx=0 tx=0).stop 2023-03-29T10:58:39.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.289+0000 7f8476bff700 1 -- 172.21.15.111:0/241178458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84700d1100 msgr2=0x7f847014fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:39.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.289+0000 7f8476bff700 1 --2- 172.21.15.111:0/241178458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84700d1100 0x7f847014fed0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8470072760 tx=0x7f8468004ad0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.289+0000 7f8476bff700 1 -- 172.21.15.111:0/241178458 shutdown_connections 2023-03-29T10:58:39.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.289+0000 7f8476bff700 1 --2- 172.21.15.111:0/241178458 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8454069de0 0x7f845406c290 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.289+0000 7f8476bff700 1 --2- 172.21.15.111:0/241178458 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f84700d1100 0x7f847014fed0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.289+0000 7f8476bff700 1 --2- 172.21.15.111:0/241178458 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f84700b87a0 0x7f84700c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.289+0000 7f8476bff700 1 --2- 172.21.15.111:0/241178458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f84700cfbf0 0x7f84700b2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.289+0000 7f8476bff700 1 -- 172.21.15.111:0/241178458 >> 172.21.15.111:0/241178458 conn(0x7f8470071b10 msgr2=0x7f847015c400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:39.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.290+0000 7f8476bff700 1 -- 172.21.15.111:0/241178458 shutdown_connections 2023-03-29T10:58:39.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.290+0000 7f8476bff700 1 -- 172.21.15.111:0/241178458 wait complete. 2023-03-29T10:58:39.307 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-29T10:58:39.494 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.495+0000 7f99c73ff700 1 Processor -- start 2023-03-29T10:58:39.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.495+0000 7f99c73ff700 1 -- start start 2023-03-29T10:58:39.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.496+0000 7f99c73ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f99c00cff50 0x7f99c0160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.496+0000 7f99c73ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c00cd520 con 0x7f99c00cf010 2023-03-29T10:58:39.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.496+0000 7f99c73ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c00d2640 con 0x7f99c015c590 2023-03-29T10:58:39.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.496+0000 7f99c73ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c01610d0 con 0x7f99c00cff50 2023-03-29T10:58:39.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.496+0000 7f99c499a700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f99c00cf010 0x7f99c00d2ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54530/0 (socket says 172.21.15.111:54530) 2023-03-29T10:58:39.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.496+0000 7f99c499a700 1 -- 172.21.15.111:0/1292083591 learned_addr learned my addr 172.21.15.111:0/1292083591 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:39.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.497+0000 7f99c599c700 1 --2- 172.21.15.111:0/1292083591 >> v2:172.21.15.119:3300/0 conn(0x7f99c00cff50 0x7f99c0160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.497+0000 7f99b7fff700 1 -- 172.21.15.111:0/1292083591 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3692717191 0 0) 0x7f99c00cd520 con 0x7f99c00cf010 2023-03-29T10:58:39.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.497+0000 7f99b7fff700 1 -- 172.21.15.111:0/1292083591 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99ac0035b0 con 0x7f99c00cf010 2023-03-29T10:58:39.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.497+0000 7f99b7fff700 1 -- 172.21.15.111:0/1292083591 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2467231852 0 0) 0x7f99c00d2640 con 0x7f99c015c590 2023-03-29T10:58:39.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.497+0000 7f99b7fff700 1 -- 172.21.15.111:0/1292083591 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99c00cd520 con 0x7f99c015c590 2023-03-29T10:58:39.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.497+0000 7f99b7fff700 1 -- 172.21.15.111:0/1292083591 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 385753658 0 0) 0x7f99ac0035b0 con 0x7f99c00cf010 2023-03-29T10:58:39.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.497+0000 7f99b7fff700 1 -- 172.21.15.111:0/1292083591 >> v2:172.21.15.119:3300/0 conn(0x7f99c00cff50 msgr2=0x7f99c0160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:39.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.497+0000 7f99b7fff700 1 --2- 172.21.15.111:0/1292083591 >> v2:172.21.15.119:3300/0 conn(0x7f99c00cff50 0x7f99c0160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.497+0000 7f99b7fff700 1 -- 172.21.15.111:0/1292083591 >> v1:172.21.15.119:6789/0 conn(0x7f99c015c590 legacy=0x7f99c00cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:39.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.497+0000 7f99b7fff700 1 -- 172.21.15.111:0/1292083591 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c00b23c0 con 0x7f99c00cf010 2023-03-29T10:58:39.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.497+0000 7f99b7fff700 1 -- 172.21.15.111:0/1292083591 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f99b0001670 con 0x7f99c00cf010 2023-03-29T10:58:39.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.497+0000 7f99b7fff700 1 -- 172.21.15.111:0/1292083591 >> v1:172.21.15.111:6789/0 conn(0x7f99c00cf010 legacy=0x7f99c00d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:39.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.497+0000 7f99b7fff700 1 --2- 172.21.15.111:0/1292083591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99ac003b90 0x7f99ac003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.497+0000 7f99b7fff700 1 --2- 172.21.15.111:0/1292083591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f99ac004850 0x7f99ac004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.497+0000 7f99b7fff700 1 --2- 172.21.15.111:0/1292083591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ac005200 0x7f99ac00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.497+0000 7f99b7fff700 1 -- 172.21.15.111:0/1292083591 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c01610d0 con 0x7f99ac005200 2023-03-29T10:58:39.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.497+0000 7f99b7fff700 1 -- 172.21.15.111:0/1292083591 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac00c060 con 0x7f99ac003b90 2023-03-29T10:58:39.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.497+0000 7f99b7fff700 1 -- 172.21.15.111:0/1292083591 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac00c220 con 0x7f99ac004850 2023-03-29T10:58:39.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.498+0000 7f99c599c700 1 --2- 172.21.15.111:0/1292083591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ac005200 0x7f99ac00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.499 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.498+0000 7f99c519b700 1 --2- 172.21.15.111:0/1292083591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99ac003b90 0x7f99ac003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.498+0000 7f99c499a700 1 --2- 172.21.15.111:0/1292083591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f99ac004850 0x7f99ac004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.498+0000 7f99c599c700 1 -- 172.21.15.111:0/1292083591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f99ac004850 msgr2=0x7f99ac004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:39.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.498+0000 7f99c599c700 1 --2- 172.21.15.111:0/1292083591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f99ac004850 0x7f99ac004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.498+0000 7f99c599c700 1 -- 172.21.15.111:0/1292083591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99ac003b90 msgr2=0x7f99ac003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:39.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.498+0000 7f99c599c700 1 --2- 172.21.15.111:0/1292083591 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99ac003b90 0x7f99ac003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.498+0000 7f99c599c700 1 -- 172.21.15.111:0/1292083591 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac0035b0 con 0x7f99ac005200 2023-03-29T10:58:39.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.498+0000 7f99c499a700 1 --2- 172.21.15.111:0/1292083591 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f99ac004850 0x7f99ac004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:39.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.498+0000 7f99c599c700 1 -- 172.21.15.111:0/1292083591 reap_dead start 2023-03-29T10:58:39.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.498+0000 7f99c599c700 1 --2- 172.21.15.111:0/1292083591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ac005200 0x7f99ac00ba10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009fc0 tx=0x7f99ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2d19e50d4cbd1b server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:39.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.498+0000 7f99b7fff700 1 -- 172.21.15.111:0/1292083591 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8003a20 con 0x7f99ac005200 2023-03-29T10:58:39.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.498+0000 7f99b7fff700 1 -- 172.21.15.111:0/1292083591 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b800f040 con 0x7f99ac005200 2023-03-29T10:58:39.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.498+0000 7f99b7fff700 1 -- 172.21.15.111:0/1292083591 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8003de0 con 0x7f99ac005200 2023-03-29T10:58:39.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.499+0000 7f99c73ff700 1 -- 172.21.15.111:0/1292083591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ac005200 msgr2=0x7f99ac00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:39.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.499+0000 7f99c73ff700 1 --2- 172.21.15.111:0/1292083591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ac005200 0x7f99ac00ba10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009fc0 tx=0x7f99ac005880 comp rx=0 tx=0).stop 2023-03-29T10:58:39.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.499+0000 7f99c73ff700 1 -- 172.21.15.111:0/1292083591 shutdown_connections 2023-03-29T10:58:39.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.499+0000 7f99c73ff700 1 --2- 172.21.15.111:0/1292083591 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ac005200 0x7f99ac00ba10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.499+0000 7f99c73ff700 1 -- 172.21.15.111:0/1292083591 >> 172.21.15.111:0/1292083591 conn(0x7f99c0071b10 msgr2=0x7f99c0071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:39.501 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.499+0000 7f99c73ff700 1 -- 172.21.15.111:0/1292083591 shutdown_connections 2023-03-29T10:58:39.501 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.499+0000 7f99c73ff700 1 -- 172.21.15.111:0/1292083591 wait complete. 2023-03-29T10:58:39.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.501+0000 7f99c73ff700 1 Processor -- start 2023-03-29T10:58:39.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.501+0000 7f99c73ff700 1 -- start start 2023-03-29T10:58:39.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.502+0000 7f99c73ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ac005200 0x7f99c01603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.502+0000 7f99c73ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c01e5f80 0x7f99c01e75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.502+0000 7f99c73ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f99c01e7b30 0x7f99c015dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.502+0000 7f99c73ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c01f55c0 con 0x7f99ac005200 2023-03-29T10:58:39.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.502+0000 7f99c73ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c01f58a0 con 0x7f99c01e5f80 2023-03-29T10:58:39.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.502+0000 7f99c73ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c01f5b80 con 0x7f99c01e7b30 2023-03-29T10:58:39.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.502+0000 7f99c519b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ac005200 0x7f99c01603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.502+0000 7f99c519b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ac005200 0x7f99c01603d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58948/0 (socket says 172.21.15.111:58948) 2023-03-29T10:58:39.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.502+0000 7f99c519b700 1 -- 172.21.15.111:0/3273661253 learned_addr learned my addr 172.21.15.111:0/3273661253 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:39.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.502+0000 7f99c499a700 1 --2- 172.21.15.111:0/3273661253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c01e5f80 0x7f99c01e75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.502+0000 7f99c599c700 1 --2- 172.21.15.111:0/3273661253 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f99c01e7b30 0x7f99c015dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.502+0000 7f99c519b700 1 -- 172.21.15.111:0/3273661253 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f99c01e7b30 msgr2=0x7f99c015dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:39.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.502+0000 7f99c519b700 1 --2- 172.21.15.111:0/3273661253 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f99c01e7b30 0x7f99c015dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.502+0000 7f99c519b700 1 -- 172.21.15.111:0/3273661253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c01e5f80 msgr2=0x7f99c01e75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:39.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.502+0000 7f99c519b700 1 --2- 172.21.15.111:0/3273661253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c01e5f80 0x7f99c01e75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.502+0000 7f99c519b700 1 -- 172.21.15.111:0/3273661253 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c00cd770 con 0x7f99ac005200 2023-03-29T10:58:39.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.502+0000 7f99c499a700 1 --2- 172.21.15.111:0/3273661253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c01e5f80 0x7f99c01e75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:39.503 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.503+0000 7f99c519b700 1 --2- 172.21.15.111:0/3273661253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ac005200 0x7f99c01603d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00f770 tx=0x7f99bc00fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:39.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.503+0000 7f99b5ffb700 1 -- 172.21.15.111:0/3273661253 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc005320 con 0x7f99ac005200 2023-03-29T10:58:39.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.503+0000 7f99b5ffb700 1 -- 172.21.15.111:0/3273661253 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99bc0054a0 con 0x7f99ac005200 2023-03-29T10:58:39.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.503+0000 7f99b5ffb700 1 -- 172.21.15.111:0/3273661253 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc006aa0 con 0x7f99ac005200 2023-03-29T10:58:39.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.503+0000 7f99c73ff700 1 -- 172.21.15.111:0/3273661253 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c00cf560 con 0x7f99ac005200 2023-03-29T10:58:39.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.503+0000 7f99c73ff700 1 -- 172.21.15.111:0/3273661253 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c015c960 con 0x7f99ac005200 2023-03-29T10:58:39.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.504+0000 7f99c73ff700 1 -- 172.21.15.111:0/3273661253 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c00b51f0 con 0x7f99ac005200 2023-03-29T10:58:39.504 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.505+0000 7f99b5ffb700 1 -- 172.21.15.111:0/3273661253 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f99bc006c00 con 0x7f99ac005200 2023-03-29T10:58:39.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.506+0000 7f99b5ffb700 1 --2- 172.21.15.111:0/3273661253 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f99a4069de0 0x7f99a406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.506+0000 7f99b5ffb700 1 -- 172.21.15.111:0/3273661253 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f99bc087850 con 0x7f99ac005200 2023-03-29T10:58:39.505 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.506+0000 7f99c499a700 1 --2- 172.21.15.111:0/3273661253 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f99a4069de0 0x7f99a406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.506 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.507+0000 7f99c499a700 1 --2- 172.21.15.111:0/3273661253 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f99a4069de0 0x7f99a406c290 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f99b0001010 tx=0x7f99b0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:39.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.510+0000 7f99b5ffb700 1 -- 172.21.15.111:0/3273661253 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f99bc052d30 con 0x7f99ac005200 2023-03-29T10:58:39.705 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.706+0000 7f99c73ff700 1 -- 172.21.15.111:0/3273661253 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f99c00721c0 con 0x7f99ac005200 2023-03-29T10:58:39.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.706+0000 7f99b5ffb700 1 -- 172.21.15.111:0/3273661253 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f99bc056350 con 0x7f99ac005200 2023-03-29T10:58:39.706 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 8 2023-03-29T10:58:39.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.708+0000 7f99c73ff700 1 -- 172.21.15.111:0/3273661253 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f99a4069de0 msgr2=0x7f99a406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:39.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.708+0000 7f99c73ff700 1 --2- 172.21.15.111:0/3273661253 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f99a4069de0 0x7f99a406c290 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f99b0001010 tx=0x7f99b0009040 comp rx=0 tx=0).stop 2023-03-29T10:58:39.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.708+0000 7f99c73ff700 1 -- 172.21.15.111:0/3273661253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ac005200 msgr2=0x7f99c01603d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:39.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.708+0000 7f99c73ff700 1 --2- 172.21.15.111:0/3273661253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ac005200 0x7f99c01603d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00f770 tx=0x7f99bc00fb30 comp rx=0 tx=0).stop 2023-03-29T10:58:39.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.708+0000 7f99c73ff700 1 -- 172.21.15.111:0/3273661253 shutdown_connections 2023-03-29T10:58:39.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.708+0000 7f99c73ff700 1 --2- 172.21.15.111:0/3273661253 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f99a4069de0 0x7f99a406c290 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.708+0000 7f99c73ff700 1 --2- 172.21.15.111:0/3273661253 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f99ac005200 0x7f99c01603d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.708+0000 7f99c73ff700 1 --2- 172.21.15.111:0/3273661253 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f99c01e7b30 0x7f99c015dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.708+0000 7f99c73ff700 1 --2- 172.21.15.111:0/3273661253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c01e5f80 0x7f99c01e75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.708+0000 7f99c73ff700 1 -- 172.21.15.111:0/3273661253 >> 172.21.15.111:0/3273661253 conn(0x7f99c0071b10 msgr2=0x7f99c00b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:39.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.708+0000 7f99c73ff700 1 -- 172.21.15.111:0/3273661253 shutdown_connections 2023-03-29T10:58:39.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.708+0000 7f99c73ff700 1 -- 172.21.15.111:0/3273661253 wait complete. 2023-03-29T10:58:39.726 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-29T10:58:39.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.917+0000 7f1cb53ff700 1 Processor -- start 2023-03-29T10:58:39.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.917+0000 7f1cb53ff700 1 -- start start 2023-03-29T10:58:39.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.918+0000 7f1cb53ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1cb00cd760 0x7f1cb00cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.918+0000 7f1cb53ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cb00adf70 0x7f1cb00cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.918+0000 7f1cb53ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb00cc9e0 con 0x7f1cb015cf10 2023-03-29T10:58:39.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.918+0000 7f1cb53ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cb015de20 con 0x7f1cb00cd760 2023-03-29T10:58:39.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.918+0000 7f1cb53ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00ce070 con 0x7f1cb00adf70 2023-03-29T10:58:39.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.918+0000 7f1caf7fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f1cb015cf10 0x7f1cb006c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54540/0 (socket says 172.21.15.111:54540) 2023-03-29T10:58:39.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.918+0000 7f1caf7fe700 1 -- 172.21.15.111:0/3333048930 learned_addr learned my addr 172.21.15.111:0/3333048930 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:39.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.919+0000 7f1caeffd700 1 --2- 172.21.15.111:0/3333048930 >> v2:172.21.15.119:3300/0 conn(0x7f1cb00cd760 0x7f1cb00cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.919+0000 7f1cae7fc700 1 --2- 172.21.15.111:0/3333048930 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cb00adf70 0x7f1cb00cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.919+0000 7f1cadffb700 1 -- 172.21.15.111:0/3333048930 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1964788798 0 0) 0x7f1cb00cc9e0 con 0x7f1cb015cf10 2023-03-29T10:58:39.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.919+0000 7f1cadffb700 1 -- 172.21.15.111:0/3333048930 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c980035b0 con 0x7f1cb015cf10 2023-03-29T10:58:39.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.919+0000 7f1cadffb700 1 -- 172.21.15.111:0/3333048930 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 528511509 0 0) 0x7f1c980035b0 con 0x7f1cb015cf10 2023-03-29T10:58:39.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.919+0000 7f1cadffb700 1 -- 172.21.15.111:0/3333048930 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cb00adf70 msgr2=0x7f1cb00cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:39.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.919+0000 7f1cadffb700 1 --2- 172.21.15.111:0/3333048930 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cb00adf70 0x7f1cb00cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.919+0000 7f1cadffb700 1 -- 172.21.15.111:0/3333048930 >> v2:172.21.15.119:3300/0 conn(0x7f1cb00cd760 msgr2=0x7f1cb00cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:39.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.919+0000 7f1cadffb700 1 --2- 172.21.15.111:0/3333048930 >> v2:172.21.15.119:3300/0 conn(0x7f1cb00cd760 0x7f1cb00cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.919+0000 7f1cadffb700 1 -- 172.21.15.111:0/3333048930 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb015d860 con 0x7f1cb015cf10 2023-03-29T10:58:39.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.919+0000 7f1cadffb700 1 -- 172.21.15.111:0/3333048930 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f1ca80019f0 con 0x7f1cb015cf10 2023-03-29T10:58:39.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.919+0000 7f1cadffb700 1 -- 172.21.15.111:0/3333048930 >> v1:172.21.15.111:6789/0 conn(0x7f1cb015cf10 legacy=0x7f1cb006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:39.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.919+0000 7f1cadffb700 1 --2- 172.21.15.111:0/3333048930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c98003a00 0x7f1c98003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.919+0000 7f1cadffb700 1 --2- 172.21.15.111:0/3333048930 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cb00adf70 0x7f1c98004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.919+0000 7f1cadffb700 1 --2- 172.21.15.111:0/3333048930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c98004d90 0x7f1c98005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.919+0000 7f1cadffb700 1 -- 172.21.15.111:0/3333048930 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb015de20 con 0x7f1c98004d90 2023-03-29T10:58:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.919+0000 7f1cadffb700 1 -- 172.21.15.111:0/3333048930 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00ce070 con 0x7f1c98003a00 2023-03-29T10:58:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.919+0000 7f1cadffb700 1 -- 172.21.15.111:0/3333048930 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9800dc20 con 0x7f1cb00adf70 2023-03-29T10:58:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.920+0000 7f1caf7fe700 1 --2- 172.21.15.111:0/3333048930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c98004d90 0x7f1c98005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.920+0000 7f1caeffd700 1 --2- 172.21.15.111:0/3333048930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c98003a00 0x7f1c98003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.920+0000 7f1cae7fc700 1 --2- 172.21.15.111:0/3333048930 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cb00adf70 0x7f1c98004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.920+0000 7f1caf7fe700 1 -- 172.21.15.111:0/3333048930 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cb00adf70 msgr2=0x7f1c98004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.920+0000 7f1caf7fe700 1 --2- 172.21.15.111:0/3333048930 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1cb00adf70 0x7f1c98004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.920+0000 7f1caf7fe700 1 -- 172.21.15.111:0/3333048930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c98003a00 msgr2=0x7f1c98003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.920+0000 7f1caf7fe700 1 --2- 172.21.15.111:0/3333048930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c98003a00 0x7f1c98003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.920+0000 7f1caf7fe700 1 -- 172.21.15.111:0/3333048930 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c980035b0 con 0x7f1c98004d90 2023-03-29T10:58:39.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.920+0000 7f1caf7fe700 1 --2- 172.21.15.111:0/3333048930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c98004d90 0x7f1c98005200 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800a8d0 tx=0x7f1c98005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633cc791c9b2d298 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.920+0000 7f1cadffb700 1 -- 172.21.15.111:0/3333048930 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca800a340 con 0x7f1c98004d90 2023-03-29T10:58:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.920+0000 7f1cadffb700 1 -- 172.21.15.111:0/3333048930 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca8012600 con 0x7f1c98004d90 2023-03-29T10:58:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.920+0000 7f1cadffb700 1 -- 172.21.15.111:0/3333048930 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca80129b0 con 0x7f1c98004d90 2023-03-29T10:58:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.921+0000 7f1cb53ff700 1 -- 172.21.15.111:0/3333048930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c98004d90 msgr2=0x7f1c98005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.921+0000 7f1cb53ff700 1 --2- 172.21.15.111:0/3333048930 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1c98004d90 0x7f1c98005200 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800a8d0 tx=0x7f1c98005880 comp rx=0 tx=0).stop 2023-03-29T10:58:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.921+0000 7f1caf7fe700 1 -- 172.21.15.111:0/3333048930 reap_dead start 2023-03-29T10:58:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.921+0000 7f1cb53ff700 1 -- 172.21.15.111:0/3333048930 shutdown_connections 2023-03-29T10:58:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.921+0000 7f1cb53ff700 1 -- 172.21.15.111:0/3333048930 >> 172.21.15.111:0/3333048930 conn(0x7f1cb00cf720 msgr2=0x7f1cb00cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.921+0000 7f1cb53ff700 1 -- 172.21.15.111:0/3333048930 shutdown_connections 2023-03-29T10:58:39.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.921+0000 7f1cb53ff700 1 -- 172.21.15.111:0/3333048930 wait complete. 2023-03-29T10:58:39.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.923+0000 7f1cb53ff700 1 Processor -- start 2023-03-29T10:58:39.923 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.923+0000 7f1cb53ff700 1 -- start start 2023-03-29T10:58:39.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.924+0000 7f1cb53ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1c98004d90 0x7f1cb00aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.924+0000 7f1cb53ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb0157b80 0x7f1cb00c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.924+0000 7f1cb53ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cb00b08f0 0x7f1cb01f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.924+0000 7f1cb53ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00c8a10 con 0x7f1cb00b08f0 2023-03-29T10:58:39.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.924+0000 7f1cb53ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00c7430 con 0x7f1cb0157b80 2023-03-29T10:58:39.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.924+0000 7f1cb53ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00c7710 con 0x7f1c98004d90 2023-03-29T10:58:39.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.924+0000 7f1caf7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cb00b08f0 0x7f1cb01f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.924+0000 7f1caf7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cb00b08f0 0x7f1cb01f4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58972/0 (socket says 172.21.15.111:58972) 2023-03-29T10:58:39.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.924+0000 7f1caf7fe700 1 -- 172.21.15.111:0/666537943 learned_addr learned my addr 172.21.15.111:0/666537943 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:39.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.924+0000 7f1cae7fc700 1 --2- 172.21.15.111:0/666537943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb0157b80 0x7f1cb00c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.924+0000 7f1caeffd700 1 --2- 172.21.15.111:0/666537943 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1c98004d90 0x7f1cb00aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.925 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.924+0000 7f1caf7fe700 1 -- 172.21.15.111:0/666537943 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1c98004d90 msgr2=0x7f1cb00aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:39.926 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.924+0000 7f1caf7fe700 1 --2- 172.21.15.111:0/666537943 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1c98004d90 0x7f1cb00aed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.926 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.924+0000 7f1caf7fe700 1 -- 172.21.15.111:0/666537943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb0157b80 msgr2=0x7f1cb00c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:39.926 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.924+0000 7f1caf7fe700 1 --2- 172.21.15.111:0/666537943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb0157b80 0x7f1cb00c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:39.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.925+0000 7f1caf7fe700 1 -- 172.21.15.111:0/666537943 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb00cc670 con 0x7f1cb00b08f0 2023-03-29T10:58:39.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.925+0000 7f1caf7fe700 1 --2- 172.21.15.111:0/666537943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cb00b08f0 0x7f1cb01f4d00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8001cb0 tx=0x7f1ca8001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:39.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.925+0000 7f1c977fe700 1 -- 172.21.15.111:0/666537943 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca800a340 con 0x7f1cb00b08f0 2023-03-29T10:58:39.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.925+0000 7f1c977fe700 1 -- 172.21.15.111:0/666537943 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca80019c0 con 0x7f1cb00b08f0 2023-03-29T10:58:39.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.925+0000 7f1c977fe700 1 -- 172.21.15.111:0/666537943 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8026640 con 0x7f1cb00b08f0 2023-03-29T10:58:39.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.925+0000 7f1cb53ff700 1 -- 172.21.15.111:0/666537943 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb015db40 con 0x7f1cb00b08f0 2023-03-29T10:58:39.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.925+0000 7f1cb53ff700 1 -- 172.21.15.111:0/666537943 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb0071600 con 0x7f1cb00b08f0 2023-03-29T10:58:39.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.926+0000 7f1cb53ff700 1 -- 172.21.15.111:0/666537943 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb00adc70 con 0x7f1cb00b08f0 2023-03-29T10:58:39.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.927+0000 7f1c977fe700 1 -- 172.21.15.111:0/666537943 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f1ca8019030 con 0x7f1cb00b08f0 2023-03-29T10:58:39.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.928+0000 7f1c977fe700 1 --2- 172.21.15.111:0/666537943 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1c9c069de0 0x7f1c9c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:39.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.928+0000 7f1c977fe700 1 -- 172.21.15.111:0/666537943 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1ca8089b60 con 0x7f1cb00b08f0 2023-03-29T10:58:39.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.928+0000 7f1caeffd700 1 --2- 172.21.15.111:0/666537943 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1c9c069de0 0x7f1c9c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:39.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.929+0000 7f1caeffd700 1 --2- 172.21.15.111:0/666537943 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1c9c069de0 0x7f1c9c06c290 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009aa0 tx=0x7f1ca0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:39.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:39.932+0000 7f1c977fe700 1 -- 172.21.15.111:0/666537943 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f1ca8055220 con 0x7f1cb00b08f0 2023-03-29T10:58:40.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.125+0000 7f1cb53ff700 1 -- 172.21.15.111:0/666537943 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1cb0071b50 con 0x7f1cb00b08f0 2023-03-29T10:58:40.125 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.126+0000 7f1c977fe700 1 -- 172.21.15.111:0/666537943 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1ca80170a0 con 0x7f1cb00b08f0 2023-03-29T10:58:40.125 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:40.128 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.128+0000 7f1cb53ff700 1 -- 172.21.15.111:0/666537943 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1c9c069de0 msgr2=0x7f1c9c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:40.128 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.128+0000 7f1cb53ff700 1 --2- 172.21.15.111:0/666537943 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1c9c069de0 0x7f1c9c06c290 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009aa0 tx=0x7f1ca0008040 comp rx=0 tx=0).stop 2023-03-29T10:58:40.128 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.128+0000 7f1cb53ff700 1 -- 172.21.15.111:0/666537943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cb00b08f0 msgr2=0x7f1cb01f4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:40.128 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.128+0000 7f1cb53ff700 1 --2- 172.21.15.111:0/666537943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cb00b08f0 0x7f1cb01f4d00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8001cb0 tx=0x7f1ca8001360 comp rx=0 tx=0).stop 2023-03-29T10:58:40.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.128+0000 7f1cb53ff700 1 -- 172.21.15.111:0/666537943 shutdown_connections 2023-03-29T10:58:40.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.128+0000 7f1cb53ff700 1 --2- 172.21.15.111:0/666537943 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1c9c069de0 0x7f1c9c06c290 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.128+0000 7f1cb53ff700 1 --2- 172.21.15.111:0/666537943 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1cb00b08f0 0x7f1cb01f4d00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.128+0000 7f1cb53ff700 1 --2- 172.21.15.111:0/666537943 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1c98004d90 0x7f1cb00aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.128+0000 7f1cb53ff700 1 --2- 172.21.15.111:0/666537943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1cb0157b80 0x7f1cb00c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.128+0000 7f1cb53ff700 1 -- 172.21.15.111:0/666537943 >> 172.21.15.111:0/666537943 conn(0x7f1cb00cf720 msgr2=0x7f1cb006c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:40.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.128+0000 7f1cb53ff700 1 -- 172.21.15.111:0/666537943 shutdown_connections 2023-03-29T10:58:40.129 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.128+0000 7f1cb53ff700 1 -- 172.21.15.111:0/666537943 wait complete. 2023-03-29T10:58:40.145 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-29T10:58:40.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.331+0000 7f60a51ff700 1 Processor -- start 2023-03-29T10:58:40.331 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.332+0000 7f60a51ff700 1 -- start start 2023-03-29T10:58:40.332 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.333+0000 7f60a51ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60a00b87a0 0x7f60a00cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:40.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.333+0000 7f60a51ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f60a0160d40 0x7f60a00d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:40.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.333+0000 7f60a51ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60a00cdc60 con 0x7f60a01601e0 2023-03-29T10:58:40.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.333+0000 7f60a51ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60a00d02d0 con 0x7f60a0160d40 2023-03-29T10:58:40.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.333+0000 7f60a51ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a00d0cb0 con 0x7f60a00b87a0 2023-03-29T10:58:40.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.333+0000 7f6097fff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f60a01601e0 0x7f60a0160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54546/0 (socket says 172.21.15.111:54546) 2023-03-29T10:58:40.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.333+0000 7f6097fff700 1 -- 172.21.15.111:0/3440699965 learned_addr learned my addr 172.21.15.111:0/3440699965 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:40.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.333+0000 7f609f59e700 1 --2- 172.21.15.111:0/3440699965 >> v2:172.21.15.111:3300/0 conn(0x7f60a0160d40 0x7f60a00d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:40.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.333+0000 7f609ed9d700 1 --2- 172.21.15.111:0/3440699965 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60a00b87a0 0x7f60a00cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:40.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.333+0000 7f609e59c700 1 -- 172.21.15.111:0/3440699965 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2892317131 0 0) 0x7f60a00cdc60 con 0x7f60a01601e0 2023-03-29T10:58:40.334 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.333+0000 7f609e59c700 1 -- 172.21.15.111:0/3440699965 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f608c0035b0 con 0x7f60a01601e0 2023-03-29T10:58:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.333+0000 7f609f59e700 1 -- 172.21.15.111:0/3440699965 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60a00b87a0 msgr2=0x7f60a00cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.333+0000 7f609f59e700 1 --2- 172.21.15.111:0/3440699965 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60a00b87a0 0x7f60a00cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.333+0000 7f609f59e700 1 -- 172.21.15.111:0/3440699965 >> v1:172.21.15.111:6789/0 conn(0x7f60a01601e0 legacy=0x7f60a0160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.333+0000 7f609f59e700 1 -- 172.21.15.111:0/3440699965 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a00cd6a0 con 0x7f60a0160d40 2023-03-29T10:58:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.334+0000 7f609f59e700 1 --2- 172.21.15.111:0/3440699965 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a0160d40 0x7f60a00d1100 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6098009d90 tx=0x7f609800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05b0fcf736d371a server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.334+0000 7f609e59c700 1 -- 172.21.15.111:0/3440699965 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609800e070 con 0x7f60a0160d40 2023-03-29T10:58:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.334+0000 7f609e59c700 1 -- 172.21.15.111:0/3440699965 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609800bb20 con 0x7f60a0160d40 2023-03-29T10:58:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.334+0000 7f609e59c700 1 -- 172.21.15.111:0/3440699965 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6098012430 con 0x7f60a0160d40 2023-03-29T10:58:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.334+0000 7f60a51ff700 1 -- 172.21.15.111:0/3440699965 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a0160d40 msgr2=0x7f60a00d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.334+0000 7f60a51ff700 1 --2- 172.21.15.111:0/3440699965 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a0160d40 0x7f60a00d1100 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6098009d90 tx=0x7f609800b330 comp rx=0 tx=0).stop 2023-03-29T10:58:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.335+0000 7f60a51ff700 1 -- 172.21.15.111:0/3440699965 shutdown_connections 2023-03-29T10:58:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.335+0000 7f60a51ff700 1 --2- 172.21.15.111:0/3440699965 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a0160d40 0x7f60a00d1100 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.335+0000 7f60a51ff700 1 --2- 172.21.15.111:0/3440699965 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60a00b87a0 0x7f60a00cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.335 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.335+0000 7f60a51ff700 1 -- 172.21.15.111:0/3440699965 >> 172.21.15.111:0/3440699965 conn(0x7f60a0071b10 msgr2=0x7f60a0071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:40.336 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.335+0000 7f60a51ff700 1 -- 172.21.15.111:0/3440699965 shutdown_connections 2023-03-29T10:58:40.336 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.335+0000 7f60a51ff700 1 -- 172.21.15.111:0/3440699965 wait complete. 2023-03-29T10:58:40.336 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.336+0000 7f60a51ff700 1 Processor -- start 2023-03-29T10:58:40.336 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.336+0000 7f60a51ff700 1 -- start start 2023-03-29T10:58:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.337+0000 7f60a51ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60a00b87a0 0x7f60a01f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.337+0000 7f60a51ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a01601e0 0x7f60a015e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.337+0000 7f60a51ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60a0160d40 0x7f60a01591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.337+0000 7f60a51ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a00caec0 con 0x7f60a01601e0 2023-03-29T10:58:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.337+0000 7f60a51ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a00cb1a0 con 0x7f60a0160d40 2023-03-29T10:58:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.337+0000 7f60a51ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a00cb480 con 0x7f60a00b87a0 2023-03-29T10:58:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.338+0000 7f6097fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a01601e0 0x7f60a015e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.338+0000 7f6097fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a01601e0 0x7f60a015e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:58994/0 (socket says 172.21.15.111:58994) 2023-03-29T10:58:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.338+0000 7f6097fff700 1 -- 172.21.15.111:0/1114319859 learned_addr learned my addr 172.21.15.111:0/1114319859 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.338+0000 7f609ed9d700 1 --2- 172.21.15.111:0/1114319859 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60a00b87a0 0x7f60a01f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.338+0000 7f609f59e700 1 --2- 172.21.15.111:0/1114319859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60a0160d40 0x7f60a01591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.338+0000 7f6097fff700 1 -- 172.21.15.111:0/1114319859 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60a00b87a0 msgr2=0x7f60a01f4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.338+0000 7f6097fff700 1 --2- 172.21.15.111:0/1114319859 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60a00b87a0 0x7f60a01f4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.338 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.338+0000 7f6097fff700 1 -- 172.21.15.111:0/1114319859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60a0160d40 msgr2=0x7f60a01591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:40.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.338+0000 7f6097fff700 1 --2- 172.21.15.111:0/1114319859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60a0160d40 0x7f60a01591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.338+0000 7f6097fff700 1 -- 172.21.15.111:0/1114319859 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6098009a70 con 0x7f60a01601e0 2023-03-29T10:58:40.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.338+0000 7f6097fff700 1 --2- 172.21.15.111:0/1114319859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a01601e0 0x7f60a015e8d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f608800c700 tx=0x7f608800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:40.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.338+0000 7f60977fe700 1 -- 172.21.15.111:0/1114319859 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6088005d70 con 0x7f60a01601e0 2023-03-29T10:58:40.339 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.339+0000 7f60977fe700 1 -- 172.21.15.111:0/1114319859 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60880070d0 con 0x7f60a01601e0 2023-03-29T10:58:40.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.339+0000 7f60977fe700 1 -- 172.21.15.111:0/1114319859 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6088007460 con 0x7f60a01601e0 2023-03-29T10:58:40.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.339+0000 7f60a51ff700 1 -- 172.21.15.111:0/1114319859 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a00d1640 con 0x7f60a01601e0 2023-03-29T10:58:40.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.339+0000 7f60a51ff700 1 -- 172.21.15.111:0/1114319859 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a00cd980 con 0x7f60a01601e0 2023-03-29T10:58:40.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.340+0000 7f60a51ff700 1 -- 172.21.15.111:0/1114319859 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a00d17d0 con 0x7f60a01601e0 2023-03-29T10:58:40.340 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.341+0000 7f60977fe700 1 -- 172.21.15.111:0/1114319859 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f60880075c0 con 0x7f60a01601e0 2023-03-29T10:58:40.341 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.341+0000 7f60977fe700 1 --2- 172.21.15.111:0/1114319859 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6080069de0 0x7f608006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:40.341 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.342+0000 7f60977fe700 1 -- 172.21.15.111:0/1114319859 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6088087880 con 0x7f60a01601e0 2023-03-29T10:58:40.341 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.342+0000 7f609ed9d700 1 --2- 172.21.15.111:0/1114319859 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6080069de0 0x7f608006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:40.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.342+0000 7f609ed9d700 1 --2- 172.21.15.111:0/1114319859 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6080069de0 0x7f608006c290 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6090006b20 tx=0x7f6090008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:40.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.345+0000 7f60977fe700 1 -- 172.21.15.111:0/1114319859 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f6088052c70 con 0x7f60a01601e0 2023-03-29T10:58:40.535 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.536+0000 7f60a51ff700 1 -- 172.21.15.111:0/1114319859 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f60a01f21c0 con 0x7f60a01601e0 2023-03-29T10:58:40.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.536+0000 7f60977fe700 1 -- 172.21.15.111:0/1114319859 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f60880170a0 con 0x7f60a01601e0 2023-03-29T10:58:40.536 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-03-29T10:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.538+0000 7f60a51ff700 1 -- 172.21.15.111:0/1114319859 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6080069de0 msgr2=0x7f608006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.538+0000 7f60a51ff700 1 --2- 172.21.15.111:0/1114319859 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6080069de0 0x7f608006c290 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6090006b20 tx=0x7f6090008040 comp rx=0 tx=0).stop 2023-03-29T10:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.538+0000 7f60a51ff700 1 -- 172.21.15.111:0/1114319859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a01601e0 msgr2=0x7f60a015e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.538+0000 7f60a51ff700 1 --2- 172.21.15.111:0/1114319859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a01601e0 0x7f60a015e8d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f608800c700 tx=0x7f608800ca10 comp rx=0 tx=0).stop 2023-03-29T10:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.538+0000 7f60a51ff700 1 -- 172.21.15.111:0/1114319859 shutdown_connections 2023-03-29T10:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.538+0000 7f60a51ff700 1 --2- 172.21.15.111:0/1114319859 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6080069de0 0x7f608006c290 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.538+0000 7f60a51ff700 1 --2- 172.21.15.111:0/1114319859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60a0160d40 0x7f60a01591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.538+0000 7f60a51ff700 1 --2- 172.21.15.111:0/1114319859 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60a00b87a0 0x7f60a01f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.538+0000 7f60a51ff700 1 --2- 172.21.15.111:0/1114319859 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60a01601e0 0x7f60a015e8d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.538+0000 7f60a51ff700 1 -- 172.21.15.111:0/1114319859 >> 172.21.15.111:0/1114319859 conn(0x7f60a0071b10 msgr2=0x7f60a00b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:40.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.538+0000 7f60a51ff700 1 -- 172.21.15.111:0/1114319859 shutdown_connections 2023-03-29T10:58:40.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.538+0000 7f60a51ff700 1 -- 172.21.15.111:0/1114319859 wait complete. 2023-03-29T10:58:40.555 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-29T10:58:40.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.741+0000 7f85b29ff700 1 Processor -- start 2023-03-29T10:58:40.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.741+0000 7f85b29ff700 1 -- start start 2023-03-29T10:58:40.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.742+0000 7f85b29ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85ac0cfff0 0x7f85ac0b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:40.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.742+0000 7f85b29ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85ac15d660 con 0x7f85ac0cd520 2023-03-29T10:58:40.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.742+0000 7f85b29ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85ac15cd70 con 0x7f85ac15a140 2023-03-29T10:58:40.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.742+0000 7f85b29ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac15d010 con 0x7f85ac0cfff0 2023-03-29T10:58:40.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.743+0000 7f85ab7fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f85ac0cd520 0x7f85ac0cf8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54562/0 (socket says 172.21.15.111:54562) 2023-03-29T10:58:40.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.743+0000 7f85ab7fe700 1 -- 172.21.15.111:0/3076790865 learned_addr learned my addr 172.21.15.111:0/3076790865 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:40.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.743+0000 7f85b0f9c700 1 --2- 172.21.15.111:0/3076790865 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85ac0cfff0 0x7f85ac0b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:40.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.743+0000 7f85aaffd700 1 -- 172.21.15.111:0/3076790865 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78159329 0 0) 0x7f85ac15d660 con 0x7f85ac0cd520 2023-03-29T10:58:40.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.743+0000 7f85aaffd700 1 -- 172.21.15.111:0/3076790865 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85940035b0 con 0x7f85ac0cd520 2023-03-29T10:58:40.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.743+0000 7f85aaffd700 1 -- 172.21.15.111:0/3076790865 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 798923194 0 0) 0x7f85ac15cd70 con 0x7f85ac15a140 2023-03-29T10:58:40.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.743+0000 7f85aaffd700 1 -- 172.21.15.111:0/3076790865 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85ac15d660 con 0x7f85ac15a140 2023-03-29T10:58:40.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.743+0000 7f85b0f9c700 1 -- 172.21.15.111:0/3076790865 >> v1:172.21.15.119:6789/0 conn(0x7f85ac15a140 legacy=0x7f85ac160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:40.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.743+0000 7f85b0f9c700 1 -- 172.21.15.111:0/3076790865 >> v1:172.21.15.111:6789/0 conn(0x7f85ac0cd520 legacy=0x7f85ac0cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:40.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.743+0000 7f85b0f9c700 1 -- 172.21.15.111:0/3076790865 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ac15d2f0 con 0x7f85ac0cfff0 2023-03-29T10:58:40.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.744+0000 7f85b0f9c700 1 --2- 172.21.15.111:0/3076790865 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85ac0cfff0 0x7f85ac0b1b70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009dc0 tx=0x7f85a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7afe713f41820143 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:40.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.744+0000 7f85aaffd700 1 -- 172.21.15.111:0/3076790865 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a400e070 con 0x7f85ac0cfff0 2023-03-29T10:58:40.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.744+0000 7f85aaffd700 1 -- 172.21.15.111:0/3076790865 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a400bb20 con 0x7f85ac0cfff0 2023-03-29T10:58:40.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.744+0000 7f85aaffd700 1 -- 172.21.15.111:0/3076790865 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4012430 con 0x7f85ac0cfff0 2023-03-29T10:58:40.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.744+0000 7f85b29ff700 1 -- 172.21.15.111:0/3076790865 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85ac0cfff0 msgr2=0x7f85ac0b1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:40.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.744+0000 7f85b29ff700 1 --2- 172.21.15.111:0/3076790865 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85ac0cfff0 0x7f85ac0b1b70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009dc0 tx=0x7f85a400b330 comp rx=0 tx=0).stop 2023-03-29T10:58:40.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.744+0000 7f85b29ff700 1 -- 172.21.15.111:0/3076790865 shutdown_connections 2023-03-29T10:58:40.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.744+0000 7f85b29ff700 1 --2- 172.21.15.111:0/3076790865 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85ac0cfff0 0x7f85ac0b1b70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.744+0000 7f85b29ff700 1 -- 172.21.15.111:0/3076790865 >> 172.21.15.111:0/3076790865 conn(0x7f85ac071b10 msgr2=0x7f85ac071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:40.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.745+0000 7f85b29ff700 1 -- 172.21.15.111:0/3076790865 shutdown_connections 2023-03-29T10:58:40.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.745+0000 7f85b29ff700 1 -- 172.21.15.111:0/3076790865 wait complete. 2023-03-29T10:58:40.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.746+0000 7f85b29ff700 1 Processor -- start 2023-03-29T10:58:40.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.746+0000 7f85b29ff700 1 -- start start 2023-03-29T10:58:40.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.747+0000 7f85b29ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85ac0cd520 0x7f85ac0c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:40.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.747+0000 7f85b29ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85ac0cfff0 0x7f85ac144aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:40.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.747+0000 7f85b29ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85ac15a140 0x7f85ac0b8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:40.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.747+0000 7f85b29ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac0b7dc0 con 0x7f85ac0cd520 2023-03-29T10:58:40.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.747+0000 7f85b29ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac0b74d0 con 0x7f85ac0cfff0 2023-03-29T10:58:40.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.747+0000 7f85b29ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac0cba80 con 0x7f85ac15a140 2023-03-29T10:58:40.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.747+0000 7f85abfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85ac0cd520 0x7f85ac0c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:40.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.748+0000 7f85abfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85ac0cd520 0x7f85ac0c8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59000/0 (socket says 172.21.15.111:59000) 2023-03-29T10:58:40.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.748+0000 7f85abfff700 1 -- 172.21.15.111:0/3453677737 learned_addr learned my addr 172.21.15.111:0/3453677737 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:40.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.748+0000 7f85ab7fe700 1 --2- 172.21.15.111:0/3453677737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85ac0cfff0 0x7f85ac144aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:40.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.748+0000 7f85b0f9c700 1 --2- 172.21.15.111:0/3453677737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85ac15a140 0x7f85ac0b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:40.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.748+0000 7f85abfff700 1 -- 172.21.15.111:0/3453677737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85ac15a140 msgr2=0x7f85ac0b8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:40.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.748+0000 7f85abfff700 1 --2- 172.21.15.111:0/3453677737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85ac15a140 0x7f85ac0b8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.748+0000 7f85abfff700 1 -- 172.21.15.111:0/3453677737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85ac0cfff0 msgr2=0x7f85ac144aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:40.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.748+0000 7f85abfff700 1 --2- 172.21.15.111:0/3453677737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85ac0cfff0 0x7f85ac144aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.748+0000 7f85abfff700 1 -- 172.21.15.111:0/3453677737 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a4009a70 con 0x7f85ac0cd520 2023-03-29T10:58:40.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.748+0000 7f85abfff700 1 --2- 172.21.15.111:0/3453677737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85ac0cd520 0x7f85ac0c8df0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f859c00c500 tx=0x7f859c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:40.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.748+0000 7f85a8ff9700 1 -- 172.21.15.111:0/3453677737 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c0052d0 con 0x7f85ac0cd520 2023-03-29T10:58:40.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.748+0000 7f85a8ff9700 1 -- 172.21.15.111:0/3453677737 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859c005450 con 0x7f85ac0cd520 2023-03-29T10:58:40.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.748+0000 7f85a8ff9700 1 -- 172.21.15.111:0/3453677737 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c0038c0 con 0x7f85ac0cd520 2023-03-29T10:58:40.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.748+0000 7f85b29ff700 1 -- 172.21.15.111:0/3453677737 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ac0cdb00 con 0x7f85ac0cd520 2023-03-29T10:58:40.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.749+0000 7f85b29ff700 1 -- 172.21.15.111:0/3453677737 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ac15daf0 con 0x7f85ac0cd520 2023-03-29T10:58:40.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.749+0000 7f85b29ff700 1 -- 172.21.15.111:0/3453677737 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85ac0b80a0 con 0x7f85ac0cd520 2023-03-29T10:58:40.750 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.750+0000 7f85a8ff9700 1 -- 172.21.15.111:0/3453677737 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f859c003a20 con 0x7f85ac0cd520 2023-03-29T10:58:40.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.751+0000 7f85a8ff9700 1 --2- 172.21.15.111:0/3453677737 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8598069de0 0x7f859806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:40.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.751+0000 7f85a8ff9700 1 -- 172.21.15.111:0/3453677737 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f859c087820 con 0x7f85ac0cd520 2023-03-29T10:58:40.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.752+0000 7f85ab7fe700 1 --2- 172.21.15.111:0/3453677737 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8598069de0 0x7f859806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:40.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.752+0000 7f85ab7fe700 1 --2- 172.21.15.111:0/3453677737 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8598069de0 0x7f859806c290 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0b72c0 tx=0x7f85a0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:40.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.755+0000 7f85a8ff9700 1 -- 172.21.15.111:0/3453677737 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f859c052da0 con 0x7f85ac0cd520 2023-03-29T10:58:40.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.946+0000 7f85b29ff700 1 -- 172.21.15.111:0/3453677737 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f85ac0d34a0 con 0x7f85ac0cd520 2023-03-29T10:58:40.946 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.947+0000 7f85a8ff9700 1 -- 172.21.15.111:0/3453677737 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f859c0563c0 con 0x7f85ac0cd520 2023-03-29T10:58:40.947 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:40.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.949+0000 7f85b29ff700 1 -- 172.21.15.111:0/3453677737 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8598069de0 msgr2=0x7f859806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:40.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.949+0000 7f85b29ff700 1 --2- 172.21.15.111:0/3453677737 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8598069de0 0x7f859806c290 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f85ac0b72c0 tx=0x7f85a0009040 comp rx=0 tx=0).stop 2023-03-29T10:58:40.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.949+0000 7f85b29ff700 1 -- 172.21.15.111:0/3453677737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85ac0cd520 msgr2=0x7f85ac0c8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:40.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.949+0000 7f85b29ff700 1 --2- 172.21.15.111:0/3453677737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85ac0cd520 0x7f85ac0c8df0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f859c00c500 tx=0x7f859c00c8c0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.949+0000 7f85b29ff700 1 -- 172.21.15.111:0/3453677737 shutdown_connections 2023-03-29T10:58:40.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.949+0000 7f85b29ff700 1 --2- 172.21.15.111:0/3453677737 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8598069de0 0x7f859806c290 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.949 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.949+0000 7f85b29ff700 1 --2- 172.21.15.111:0/3453677737 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f85ac0cd520 0x7f85ac0c8df0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.950 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.949+0000 7f85b29ff700 1 --2- 172.21.15.111:0/3453677737 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f85ac15a140 0x7f85ac0b8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.950 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.949+0000 7f85b29ff700 1 --2- 172.21.15.111:0/3453677737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85ac0cfff0 0x7f85ac144aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:40.950 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.949+0000 7f85b29ff700 1 -- 172.21.15.111:0/3453677737 >> 172.21.15.111:0/3453677737 conn(0x7f85ac071b10 msgr2=0x7f85ac0cf5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:40.950 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.950+0000 7f85b29ff700 1 -- 172.21.15.111:0/3453677737 shutdown_connections 2023-03-29T10:58:40.950 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:40.950+0000 7f85b29ff700 1 -- 172.21.15.111:0/3453677737 wait complete. 2023-03-29T10:58:40.966 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-03-29T10:58:41.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.150+0000 7f9ad9dff700 1 Processor -- start 2023-03-29T10:58:41.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.150+0000 7f9ad9dff700 1 -- start start 2023-03-29T10:58:41.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.151+0000 7f9ad9dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9ad40ca210 0x7f9ad40b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:41.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.151+0000 7f9ad9dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9ad40b1b80 0x7f9ad406b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:41.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.151+0000 7f9ad9dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad406a5f0 con 0x7f9ad40cd790 2023-03-29T10:58:41.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.151+0000 7f9ad9dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad40ca680 con 0x7f9ad40b1b80 2023-03-29T10:58:41.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.151+0000 7f9ad9dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad406a840 con 0x7f9ad40ca210 2023-03-29T10:58:41.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.152+0000 7f9ad37fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9ad40cd790 0x7f9ad40cdb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.111:47510/0 (socket says 172.21.15.111:47510) 2023-03-29T10:58:41.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.152+0000 7f9ad2ffd700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9ad40ca210 0x7f9ad40b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:41.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.152+0000 7f9ad37fe700 1 -- 172.21.15.111:0/1814691145 learned_addr learned my addr 172.21.15.111:0/1814691145 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:41.153 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.152+0000 7f9ad3fff700 1 --2- 172.21.15.111:0/1814691145 >> v2:172.21.15.119:3300/0 conn(0x7f9ad40b1b80 0x7f9ad406b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.152+0000 7f9ad27fc700 1 -- 172.21.15.111:0/1814691145 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993704625 0 0) 0x7f9ad406a5f0 con 0x7f9ad40cd790 2023-03-29T10:58:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.152+0000 7f9ad27fc700 1 -- 172.21.15.111:0/1814691145 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9abc0035b0 con 0x7f9ad40cd790 2023-03-29T10:58:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.152+0000 7f9ad2ffd700 1 -- 172.21.15.111:0/1814691145 >> v2:172.21.15.119:3300/0 conn(0x7f9ad40b1b80 msgr2=0x7f9ad406b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.152+0000 7f9ad2ffd700 1 --2- 172.21.15.111:0/1814691145 >> v2:172.21.15.119:3300/0 conn(0x7f9ad40b1b80 0x7f9ad406b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.152+0000 7f9ad2ffd700 1 -- 172.21.15.111:0/1814691145 >> v1:172.21.15.119:6789/0 conn(0x7f9ad40cd790 legacy=0x7f9ad40cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.152+0000 7f9ad2ffd700 1 -- 172.21.15.111:0/1814691145 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad406ab20 con 0x7f9ad40ca210 2023-03-29T10:58:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.152+0000 7f9ad2ffd700 1 --2- 172.21.15.111:0/1814691145 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9ad40ca210 0x7f9ad40b1640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009840 tx=0x7f9ac8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcd6ec591114ed44 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.153+0000 7f9ad27fc700 1 -- 172.21.15.111:0/1814691145 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8011830 con 0x7f9ad40ca210 2023-03-29T10:58:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.153+0000 7f9ad27fc700 1 -- 172.21.15.111:0/1814691145 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac8011990 con 0x7f9ad40ca210 2023-03-29T10:58:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.153+0000 7f9ad27fc700 1 -- 172.21.15.111:0/1814691145 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8011d20 con 0x7f9ad40ca210 2023-03-29T10:58:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.153+0000 7f9ad9dff700 1 -- 172.21.15.111:0/1814691145 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9ad40ca210 msgr2=0x7f9ad40b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.153+0000 7f9ad9dff700 1 --2- 172.21.15.111:0/1814691145 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9ad40ca210 0x7f9ad40b1640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009840 tx=0x7f9ac8009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.153+0000 7f9ad9dff700 1 -- 172.21.15.111:0/1814691145 shutdown_connections 2023-03-29T10:58:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.153+0000 7f9ad9dff700 1 --2- 172.21.15.111:0/1814691145 >> v2:172.21.15.119:3300/0 conn(0x7f9ad40b1b80 0x7f9ad406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.153+0000 7f9ad9dff700 1 --2- 172.21.15.111:0/1814691145 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9ad40ca210 0x7f9ad40b1640 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.153+0000 7f9ad9dff700 1 -- 172.21.15.111:0/1814691145 >> 172.21.15.111:0/1814691145 conn(0x7f9ad406e980 msgr2=0x7f9ad40cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.153+0000 7f9ad9dff700 1 -- 172.21.15.111:0/1814691145 shutdown_connections 2023-03-29T10:58:41.154 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.153+0000 7f9ad9dff700 1 -- 172.21.15.111:0/1814691145 wait complete. 2023-03-29T10:58:41.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.155+0000 7f9ad9dff700 1 Processor -- start 2023-03-29T10:58:41.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.155+0000 7f9ad9dff700 1 -- start start 2023-03-29T10:58:41.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.156+0000 7f9ad9dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ad40b1b80 0x7f9ad4148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:41.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.156+0000 7f9ad9dff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ad40ca210 0x7f9ad41f5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:41.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.156+0000 7f9ad9dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9ad40cd790 0x7f9ad41f6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:41.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.156+0000 7f9ad9dff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad40c8700 con 0x7f9ad40ca210 2023-03-29T10:58:41.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.156+0000 7f9ad9dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad40c8910 con 0x7f9ad40b1b80 2023-03-29T10:58:41.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.156+0000 7f9ad9dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad40c8bf0 con 0x7f9ad40cd790 2023-03-29T10:58:41.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.156+0000 7f9ad2ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ad40ca210 0x7f9ad41f5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:41.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.156+0000 7f9ad2ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ad40ca210 0x7f9ad41f5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59014/0 (socket says 172.21.15.111:59014) 2023-03-29T10:58:41.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.156+0000 7f9ad2ffd700 1 -- 172.21.15.111:0/26718410 learned_addr learned my addr 172.21.15.111:0/26718410 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:41.156 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.156+0000 7f9ad37fe700 1 --2- 172.21.15.111:0/26718410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ad40b1b80 0x7f9ad4148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:41.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.156+0000 7f9ad3fff700 1 --2- 172.21.15.111:0/26718410 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9ad40cd790 0x7f9ad41f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:41.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.157+0000 7f9ad2ffd700 1 -- 172.21.15.111:0/26718410 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9ad40cd790 msgr2=0x7f9ad41f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:41.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.157+0000 7f9ad2ffd700 1 --2- 172.21.15.111:0/26718410 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9ad40cd790 0x7f9ad41f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:41.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.157+0000 7f9ad2ffd700 1 -- 172.21.15.111:0/26718410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ad40b1b80 msgr2=0x7f9ad4148ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:41.157 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.157+0000 7f9ad2ffd700 1 --2- 172.21.15.111:0/26718410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ad40b1b80 0x7f9ad4148ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:41.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.157+0000 7f9ad2ffd700 1 -- 172.21.15.111:0/26718410 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac80094a0 con 0x7f9ad40ca210 2023-03-29T10:58:41.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.157+0000 7f9ad2ffd700 1 --2- 172.21.15.111:0/26718410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ad40ca210 0x7f9ad41f5ed0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8000c00 tx=0x7f9ac8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:41.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.157+0000 7f9abbfff700 1 -- 172.21.15.111:0/26718410 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8004590 con 0x7f9ad40ca210 2023-03-29T10:58:41.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.157+0000 7f9abbfff700 1 -- 172.21.15.111:0/26718410 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac801f070 con 0x7f9ad40ca210 2023-03-29T10:58:41.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.157+0000 7f9ad9dff700 1 -- 172.21.15.111:0/26718410 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad406b0e0 con 0x7f9ad40ca210 2023-03-29T10:58:41.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.157+0000 7f9abbfff700 1 -- 172.21.15.111:0/26718410 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac80038e0 con 0x7f9ad40ca210 2023-03-29T10:58:41.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.157+0000 7f9ad9dff700 1 -- 172.21.15.111:0/26718410 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad406ae00 con 0x7f9ad40ca210 2023-03-29T10:58:41.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.158+0000 7f9ad9dff700 1 -- 172.21.15.111:0/26718410 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad40b2000 con 0x7f9ad40ca210 2023-03-29T10:58:41.158 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.159+0000 7f9abbfff700 1 -- 172.21.15.111:0/26718410 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f9ac8003a40 con 0x7f9ad40ca210 2023-03-29T10:58:41.159 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.160+0000 7f9abbfff700 1 --2- 172.21.15.111:0/26718410 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9ac0069de0 0x7f9ac006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:41.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.160+0000 7f9abbfff700 1 -- 172.21.15.111:0/26718410 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9ac8021070 con 0x7f9ad40ca210 2023-03-29T10:58:41.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.160+0000 7f9ad37fe700 1 --2- 172.21.15.111:0/26718410 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9ac0069de0 0x7f9ac006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:41.160 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.161+0000 7f9ad37fe700 1 --2- 172.21.15.111:0/26718410 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9ac0069de0 0x7f9ac006c290 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4007fd0 tx=0x7f9ac4007dc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:41.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.164+0000 7f9abbfff700 1 -- 172.21.15.111:0/26718410 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f9ac8055100 con 0x7f9ad40ca210 2023-03-29T10:58:41.373 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.374+0000 7f9ad9dff700 1 -- 172.21.15.111:0/26718410 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9ad40c9ff0 con 0x7f9ad40ca210 2023-03-29T10:58:41.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.374+0000 7f9abbfff700 1 -- 172.21.15.111:0/26718410 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9ac8058720 con 0x7f9ad40ca210 2023-03-29T10:58:41.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.376+0000 7f9ad9dff700 1 -- 172.21.15.111:0/26718410 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9ac0069de0 msgr2=0x7f9ac006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:41.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.376+0000 7f9ad9dff700 1 --2- 172.21.15.111:0/26718410 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9ac0069de0 0x7f9ac006c290 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4007fd0 tx=0x7f9ac4007dc0 comp rx=0 tx=0).stop 2023-03-29T10:58:41.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.376+0000 7f9ad9dff700 1 -- 172.21.15.111:0/26718410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ad40ca210 msgr2=0x7f9ad41f5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:41.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.376+0000 7f9ad9dff700 1 --2- 172.21.15.111:0/26718410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ad40ca210 0x7f9ad41f5ed0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8000c00 tx=0x7f9ac8004750 comp rx=0 tx=0).stop 2023-03-29T10:58:41.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.376+0000 7f9ad9dff700 1 -- 172.21.15.111:0/26718410 shutdown_connections 2023-03-29T10:58:41.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.376+0000 7f9ad9dff700 1 --2- 172.21.15.111:0/26718410 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9ac0069de0 0x7f9ac006c290 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:41.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.376+0000 7f9ad9dff700 1 --2- 172.21.15.111:0/26718410 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9ad40cd790 0x7f9ad41f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:41.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.376+0000 7f9ad9dff700 1 --2- 172.21.15.111:0/26718410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ad40b1b80 0x7f9ad4148ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:41.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.376+0000 7f9ad9dff700 1 --2- 172.21.15.111:0/26718410 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9ad40ca210 0x7f9ad41f5ed0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:41.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.377+0000 7f9ad9dff700 1 -- 172.21.15.111:0/26718410 >> 172.21.15.111:0/26718410 conn(0x7f9ad406e980 msgr2=0x7f9ad40cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:41.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.377+0000 7f9ad9dff700 1 -- 172.21.15.111:0/26718410 shutdown_connections 2023-03-29T10:58:41.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.377+0000 7f9ad9dff700 1 -- 172.21.15.111:0/26718410 wait complete. 2023-03-29T10:58:41.377 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-03-29T10:58:41.393 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi119.front.sepia.ceph.com... 2023-03-29T10:58:41.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:41.393 DEBUG:teuthology.orchestra.run.smithi119:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-03-29T10:58:41.464 INFO:teuthology.orchestra.run:waiting for 300 2023-03-29T10:58:41.464 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-03-29T10:58:41.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:41.464 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-03-29T10:58:41.587 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:58:41.587 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-29T10:58:41.629 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:58:41.629 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-29T10:58:41.709 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:58:41.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.931+0000 7fb0cebff700 1 Processor -- start 2023-03-29T10:58:41.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.932+0000 7fb0cebff700 1 -- start start 2023-03-29T10:58:41.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.933+0000 7fb0cebff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb0c8160c00 0x7fb0c80b5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:41.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.933+0000 7fb0cebff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb0c80cf190 0x7fb0c80cf610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:41.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.933+0000 7fb0cebff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c81611d0 con 0x7fb0c80b88e0 2023-03-29T10:58:41.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.933+0000 7fb0cebff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c80d2e60 con 0x7fb0c80cf190 2023-03-29T10:58:41.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.933+0000 7fb0cebff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c80cd830 con 0x7fb0c8160c00 2023-03-29T10:58:41.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.933+0000 7fb0cd19c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb0c80cf190 0x7fb0c80cf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:41.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.933+0000 7fb0cc99b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb0c80b88e0 0x7fb0c80b4a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54568/0 (socket says 172.21.15.111:54568) 2023-03-29T10:58:41.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.933+0000 7fb0cc99b700 1 -- 172.21.15.111:0/2648977008 learned_addr learned my addr 172.21.15.111:0/2648977008 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:41.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.933+0000 7fb0c7fff700 1 --2- 172.21.15.111:0/2648977008 >> v2:172.21.15.119:3300/0 conn(0x7fb0c8160c00 0x7fb0c80b5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:41.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.933+0000 7fb0cd19c700 1 -- 172.21.15.111:0/2648977008 >> v2:172.21.15.119:3300/0 conn(0x7fb0c8160c00 msgr2=0x7fb0c80b5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:41.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.933+0000 7fb0cd19c700 1 --2- 172.21.15.111:0/2648977008 >> v2:172.21.15.119:3300/0 conn(0x7fb0c8160c00 0x7fb0c80b5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:41.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.933+0000 7fb0cd19c700 1 -- 172.21.15.111:0/2648977008 >> v1:172.21.15.111:6789/0 conn(0x7fb0c80b88e0 legacy=0x7fb0c80b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:41.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.933+0000 7fb0cd19c700 1 -- 172.21.15.111:0/2648977008 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c80cdc50 con 0x7fb0c80cf190 2023-03-29T10:58:41.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.933+0000 7fb0c77fe700 1 -- 172.21.15.111:0/2648977008 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935104964 0 0) 0x7fb0bc001110 con 0x7fb0c80b88e0 2023-03-29T10:58:41.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.934+0000 7fb0cd19c700 1 --2- 172.21.15.111:0/2648977008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c80cf190 0x7fb0c80cf610 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8066ae0 tx=0x7fb0c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e026a292906af67 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:41.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.934+0000 7fb0c77fe700 1 -- 172.21.15.111:0/2648977008 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c000e070 con 0x7fb0c80cf190 2023-03-29T10:58:41.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.934+0000 7fb0c77fe700 1 -- 172.21.15.111:0/2648977008 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c000bbd0 con 0x7fb0c80cf190 2023-03-29T10:58:41.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.934+0000 7fb0c77fe700 1 -- 172.21.15.111:0/2648977008 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0012430 con 0x7fb0c80cf190 2023-03-29T10:58:41.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.934+0000 7fb0cebff700 1 -- 172.21.15.111:0/2648977008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c80cf190 msgr2=0x7fb0c80cf610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:41.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.934+0000 7fb0cebff700 1 --2- 172.21.15.111:0/2648977008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c80cf190 0x7fb0c80cf610 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb0c8066ae0 tx=0x7fb0c000b330 comp rx=0 tx=0).stop 2023-03-29T10:58:41.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.935+0000 7fb0cebff700 1 -- 172.21.15.111:0/2648977008 shutdown_connections 2023-03-29T10:58:41.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.935+0000 7fb0cebff700 1 --2- 172.21.15.111:0/2648977008 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c80cf190 0x7fb0c80cf610 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:41.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.935+0000 7fb0cebff700 1 --2- 172.21.15.111:0/2648977008 >> v2:172.21.15.119:3300/0 conn(0x7fb0c8160c00 0x7fb0c80b5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:41.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.935+0000 7fb0cebff700 1 -- 172.21.15.111:0/2648977008 >> 172.21.15.111:0/2648977008 conn(0x7fb0c80666b0 msgr2=0x7fb0c8071b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:41.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.935+0000 7fb0cebff700 1 -- 172.21.15.111:0/2648977008 shutdown_connections 2023-03-29T10:58:41.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.935+0000 7fb0cebff700 1 -- 172.21.15.111:0/2648977008 wait complete. 2023-03-29T10:58:41.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.936+0000 7fb0cebff700 1 Processor -- start 2023-03-29T10:58:41.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.937+0000 7fb0cebff700 1 -- start start 2023-03-29T10:58:41.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.937+0000 7fb0cebff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0c80cf190 0x7fb0c8155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:41.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.937+0000 7fb0cebff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb0c8160c00 0x7fb0c81483c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:41.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.937+0000 7fb0cebff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c80b2930 0x7fb0c80b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:41.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.937+0000 7fb0cebff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c8155cb0 con 0x7fb0c80b2930 2023-03-29T10:58:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.937+0000 7fb0cebff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c81f30e0 con 0x7fb0c80cf190 2023-03-29T10:58:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.937+0000 7fb0cebff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c81588f0 con 0x7fb0c8160c00 2023-03-29T10:58:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.938+0000 7fb0cd19c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c80b2930 0x7fb0c80b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.938+0000 7fb0cd19c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c80b2930 0x7fb0c80b2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59032/0 (socket says 172.21.15.111:59032) 2023-03-29T10:58:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.938+0000 7fb0cd19c700 1 -- 172.21.15.111:0/227342219 learned_addr learned my addr 172.21.15.111:0/227342219 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.938+0000 7fb0cc99b700 1 --2- 172.21.15.111:0/227342219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0c80cf190 0x7fb0c8155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:41.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.938+0000 7fb0c7fff700 1 --2- 172.21.15.111:0/227342219 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb0c8160c00 0x7fb0c81483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:41.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.938+0000 7fb0cd19c700 1 -- 172.21.15.111:0/227342219 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb0c8160c00 msgr2=0x7fb0c81483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:41.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.938+0000 7fb0cd19c700 1 --2- 172.21.15.111:0/227342219 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb0c8160c00 0x7fb0c81483c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:41.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.938+0000 7fb0cd19c700 1 -- 172.21.15.111:0/227342219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0c80cf190 msgr2=0x7fb0c8155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:41.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.938+0000 7fb0cd19c700 1 --2- 172.21.15.111:0/227342219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0c80cf190 0x7fb0c8155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:41.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.938+0000 7fb0cd19c700 1 -- 172.21.15.111:0/227342219 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c0009a70 con 0x7fb0c80b2930 2023-03-29T10:58:41.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.938+0000 7fb0c7fff700 1 --2- 172.21.15.111:0/227342219 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb0c8160c00 0x7fb0c81483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:41.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.938+0000 7fb0cd19c700 1 --2- 172.21.15.111:0/227342219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c80b2930 0x7fb0c80b2db0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80b5bc0 tx=0x7fb0c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:41.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.939+0000 7fb0c57fa700 1 -- 172.21.15.111:0/227342219 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0012430 con 0x7fb0c80b2930 2023-03-29T10:58:41.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.939+0000 7fb0c57fa700 1 -- 172.21.15.111:0/227342219 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c0020070 con 0x7fb0c80b2930 2023-03-29T10:58:41.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.939+0000 7fb0c57fa700 1 -- 172.21.15.111:0/227342219 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0004280 con 0x7fb0c80b2930 2023-03-29T10:58:41.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.939+0000 7fb0cebff700 1 -- 172.21.15.111:0/227342219 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c80d2b80 con 0x7fb0c80b2930 2023-03-29T10:58:41.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.939+0000 7fb0cebff700 1 -- 172.21.15.111:0/227342219 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c80d28a0 con 0x7fb0c80b2930 2023-03-29T10:58:41.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.940+0000 7fb0cebff700 1 -- 172.21.15.111:0/227342219 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c80d0c30 con 0x7fb0c80b2930 2023-03-29T10:58:41.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.941+0000 7fb0c57fa700 1 -- 172.21.15.111:0/227342219 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fb0c00043e0 con 0x7fb0c80b2930 2023-03-29T10:58:41.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.942+0000 7fb0c57fa700 1 --2- 172.21.15.111:0/227342219 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb0ac069de0 0x7fb0ac06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:41.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.942+0000 7fb0c57fa700 1 -- 172.21.15.111:0/227342219 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb0c0022070 con 0x7fb0c80b2930 2023-03-29T10:58:41.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.942+0000 7fb0cc99b700 1 --2- 172.21.15.111:0/227342219 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb0ac069de0 0x7fb0ac06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:41.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.942+0000 7fb0cc99b700 1 --2- 172.21.15.111:0/227342219 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb0ac069de0 0x7fb0ac06c290 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc000f60 tx=0x7fb0bc007c20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:41.945 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:41.946+0000 7fb0c57fa700 1 -- 172.21.15.111:0/227342219 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fb0c0055af0 con 0x7fb0c80b2930 2023-03-29T10:58:42.141 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.141+0000 7fb0cebff700 1 -- 172.21.15.111:0/227342219 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0c81f3220 con 0x7fb0c80b2930 2023-03-29T10:58:42.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.143+0000 7fb0c57fa700 1 -- 172.21.15.111:0/227342219 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19892 (secure 0 0 0) 0x7fb0c0059110 con 0x7fb0c80b2930 2023-03-29T10:58:42.143 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:42.143 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":30,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:58:09.476334+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.26","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:58:42.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.145+0000 7fb0cebff700 1 -- 172.21.15.111:0/227342219 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb0ac069de0 msgr2=0x7fb0ac06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:42.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.145+0000 7fb0cebff700 1 --2- 172.21.15.111:0/227342219 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb0ac069de0 0x7fb0ac06c290 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc000f60 tx=0x7fb0bc007c20 comp rx=0 tx=0).stop 2023-03-29T10:58:42.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.145+0000 7fb0cebff700 1 -- 172.21.15.111:0/227342219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c80b2930 msgr2=0x7fb0c80b2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:42.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.145+0000 7fb0cebff700 1 --2- 172.21.15.111:0/227342219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c80b2930 0x7fb0c80b2db0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb0c80b5bc0 tx=0x7fb0c0004ad0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.145+0000 7fb0cebff700 1 -- 172.21.15.111:0/227342219 shutdown_connections 2023-03-29T10:58:42.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.145+0000 7fb0cebff700 1 --2- 172.21.15.111:0/227342219 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb0ac069de0 0x7fb0ac06c290 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.145+0000 7fb0cebff700 1 --2- 172.21.15.111:0/227342219 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb0c80b2930 0x7fb0c80b2db0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.145+0000 7fb0cebff700 1 --2- 172.21.15.111:0/227342219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb0c80cf190 0x7fb0c8155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.145+0000 7fb0cebff700 1 --2- 172.21.15.111:0/227342219 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb0c8160c00 0x7fb0c81483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.145+0000 7fb0cebff700 1 -- 172.21.15.111:0/227342219 >> 172.21.15.111:0/227342219 conn(0x7fb0c80666b0 msgr2=0x7fb0c815ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:42.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.146+0000 7fb0cebff700 1 -- 172.21.15.111:0/227342219 shutdown_connections 2023-03-29T10:58:42.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.146+0000 7fb0cebff700 1 -- 172.21.15.111:0/227342219 wait complete. 2023-03-29T10:58:42.163 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-29T10:57:16.183979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-29T10:57:21.239809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-29T10:57:26.192124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-29T10:57:26.796777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-29T10:57:29.874795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-29T10:58:42.163 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-29T10:58:42.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.349+0000 7f3eec3ff700 1 Processor -- start 2023-03-29T10:58:42.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.349+0000 7f3eec3ff700 1 -- start start 2023-03-29T10:58:42.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.351+0000 7f3eec3ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3ee406aaf0 0x7f3ee4070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:42.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.351+0000 7f3eec3ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee40cb280 con 0x7f3ee4071030 2023-03-29T10:58:42.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.351+0000 7f3eec3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee4071700 con 0x7f3ee415a5a0 2023-03-29T10:58:42.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.351+0000 7f3eec3ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee415a970 con 0x7f3ee406aaf0 2023-03-29T10:58:42.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.351+0000 7f3eea99c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f3ee4071030 0x7f3ee406d1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54576/0 (socket says 172.21.15.111:54576) 2023-03-29T10:58:42.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.351+0000 7f3ee999a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3ee406aaf0 0x7f3ee4070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:42.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.351+0000 7f3eea99c700 1 -- 172.21.15.111:0/285454097 learned_addr learned my addr 172.21.15.111:0/285454097 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:42.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.351+0000 7f3ee999a700 1 -- 172.21.15.111:0/285454097 >> v1:172.21.15.119:6789/0 conn(0x7f3ee415a5a0 legacy=0x7f3ee406a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:42.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.351+0000 7f3ee999a700 1 -- 172.21.15.111:0/285454097 >> v1:172.21.15.111:6789/0 conn(0x7f3ee4071030 legacy=0x7f3ee406d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:42.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.351+0000 7f3ee999a700 1 -- 172.21.15.111:0/285454097 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee40719e0 con 0x7f3ee406aaf0 2023-03-29T10:58:42.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.351+0000 7f3ee9199700 1 -- 172.21.15.111:0/285454097 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 605753120 0 0) 0x7f3edc001490 con 0x7f3ee4071030 2023-03-29T10:58:42.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.352+0000 7f3ee999a700 1 --2- 172.21.15.111:0/285454097 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ee406aaf0 0x7f3ee4070af0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009820 tx=0x7f3ed4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af03d6dd8e12353d server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:42.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.352+0000 7f3ee9199700 1 -- 172.21.15.111:0/285454097 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed4005e30 con 0x7f3ee406aaf0 2023-03-29T10:58:42.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.352+0000 7f3ee9199700 1 -- 172.21.15.111:0/285454097 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed4006030 con 0x7f3ee406aaf0 2023-03-29T10:58:42.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.352+0000 7f3ee9199700 1 -- 172.21.15.111:0/285454097 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed40063e0 con 0x7f3ee406aaf0 2023-03-29T10:58:42.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.352+0000 7f3eec3ff700 1 -- 172.21.15.111:0/285454097 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ee406aaf0 msgr2=0x7f3ee4070af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:42.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.352+0000 7f3eec3ff700 1 --2- 172.21.15.111:0/285454097 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ee406aaf0 0x7f3ee4070af0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009820 tx=0x7f3ed4009b30 comp rx=0 tx=0).stop 2023-03-29T10:58:42.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.353+0000 7f3eec3ff700 1 -- 172.21.15.111:0/285454097 shutdown_connections 2023-03-29T10:58:42.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.353+0000 7f3eec3ff700 1 --2- 172.21.15.111:0/285454097 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ee406aaf0 0x7f3ee4070af0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.353+0000 7f3eec3ff700 1 -- 172.21.15.111:0/285454097 >> 172.21.15.111:0/285454097 conn(0x7f3ee40cf720 msgr2=0x7f3ee40cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:42.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.353+0000 7f3eec3ff700 1 -- 172.21.15.111:0/285454097 shutdown_connections 2023-03-29T10:58:42.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.353+0000 7f3eec3ff700 1 -- 172.21.15.111:0/285454097 wait complete. 2023-03-29T10:58:42.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.354+0000 7f3eec3ff700 1 Processor -- start 2023-03-29T10:58:42.354 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.354+0000 7f3eec3ff700 1 -- start start 2023-03-29T10:58:42.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.355+0000 7f3eec3ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ee406aaf0 0x7f3ee4155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:42.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.355+0000 7f3eec3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ee4071030 0x7f3ee4155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:42.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.355+0000 7f3eec3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ee415a5a0 0x7f3ee415b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:42.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.355+0000 7f3eec3ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee41558b0 con 0x7f3ee406aaf0 2023-03-29T10:58:42.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.355+0000 7f3eec3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee415aab0 con 0x7f3ee4071030 2023-03-29T10:58:42.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.355+0000 7f3eec3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee40b3b50 con 0x7f3ee415a5a0 2023-03-29T10:58:42.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.355+0000 7f3eea19b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ee406aaf0 0x7f3ee4155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:42.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.355+0000 7f3eea19b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ee406aaf0 0x7f3ee4155d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59062/0 (socket says 172.21.15.111:59062) 2023-03-29T10:58:42.355 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.356+0000 7f3eea19b700 1 -- 172.21.15.111:0/3058804047 learned_addr learned my addr 172.21.15.111:0/3058804047 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:42.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.356+0000 7f3ee999a700 1 --2- 172.21.15.111:0/3058804047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ee4071030 0x7f3ee4155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:42.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.356+0000 7f3eea99c700 1 --2- 172.21.15.111:0/3058804047 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ee415a5a0 0x7f3ee415b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:42.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.356+0000 7f3eea19b700 1 -- 172.21.15.111:0/3058804047 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ee415a5a0 msgr2=0x7f3ee415b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:42.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.356+0000 7f3eea19b700 1 --2- 172.21.15.111:0/3058804047 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ee415a5a0 0x7f3ee415b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.356+0000 7f3eea19b700 1 -- 172.21.15.111:0/3058804047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ee4071030 msgr2=0x7f3ee4155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:42.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.356+0000 7f3eea19b700 1 --2- 172.21.15.111:0/3058804047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ee4071030 0x7f3ee4155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.356+0000 7f3eea19b700 1 -- 172.21.15.111:0/3058804047 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed4009430 con 0x7f3ee406aaf0 2023-03-29T10:58:42.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.356+0000 7f3eea99c700 1 --2- 172.21.15.111:0/3058804047 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ee415a5a0 0x7f3ee415b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:42.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.356+0000 7f3eea19b700 1 --2- 172.21.15.111:0/3058804047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ee406aaf0 0x7f3ee4155d70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000c4e0 tx=0x7f3ee000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:42.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.356+0000 7f3edaffd700 1 -- 172.21.15.111:0/3058804047 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee00052d0 con 0x7f3ee406aaf0 2023-03-29T10:58:42.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.356+0000 7f3edaffd700 1 -- 172.21.15.111:0/3058804047 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee0005430 con 0x7f3ee406aaf0 2023-03-29T10:58:42.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.356+0000 7f3eec3ff700 1 -- 172.21.15.111:0/3058804047 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee406c9a0 con 0x7f3ee406aaf0 2023-03-29T10:58:42.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.356+0000 7f3edaffd700 1 -- 172.21.15.111:0/3058804047 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee00038c0 con 0x7f3ee406aaf0 2023-03-29T10:58:42.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.357+0000 7f3eec3ff700 1 -- 172.21.15.111:0/3058804047 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee406c6c0 con 0x7f3ee406aaf0 2023-03-29T10:58:42.357 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.358+0000 7f3eec3ff700 1 -- 172.21.15.111:0/3058804047 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee406cc80 con 0x7f3ee406aaf0 2023-03-29T10:58:42.358 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.359+0000 7f3edaffd700 1 -- 172.21.15.111:0/3058804047 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f3ee0003a20 con 0x7f3ee406aaf0 2023-03-29T10:58:42.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.360+0000 7f3edaffd700 1 --2- 172.21.15.111:0/3058804047 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3ec8069de0 0x7f3ec806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:42.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.360+0000 7f3edaffd700 1 -- 172.21.15.111:0/3058804047 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3ee0087720 con 0x7f3ee406aaf0 2023-03-29T10:58:42.359 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.360+0000 7f3ee999a700 1 --2- 172.21.15.111:0/3058804047 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3ec8069de0 0x7f3ec806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:42.360 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.361+0000 7f3ee999a700 1 --2- 172.21.15.111:0/3058804047 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3ec8069de0 0x7f3ec806c290 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4013040 tx=0x7f3ed4008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:42.364 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.364+0000 7f3edaffd700 1 -- 172.21.15.111:0/3058804047 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f3ee0052d30 con 0x7f3ee406aaf0 2023-03-29T10:58:42.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.557+0000 7f3eec3ff700 1 -- 172.21.15.111:0/3058804047 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ee4157840 con 0x7f3ee406aaf0 2023-03-29T10:58:42.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.557+0000 7f3edaffd700 1 -- 172.21.15.111:0/3058804047 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3ee0056350 con 0x7f3ee406aaf0 2023-03-29T10:58:42.557 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-03-29T10:58:42.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.560+0000 7f3eec3ff700 1 -- 172.21.15.111:0/3058804047 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3ec8069de0 msgr2=0x7f3ec806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:42.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.560+0000 7f3eec3ff700 1 --2- 172.21.15.111:0/3058804047 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3ec8069de0 0x7f3ec806c290 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4013040 tx=0x7f3ed4008070 comp rx=0 tx=0).stop 2023-03-29T10:58:42.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.560+0000 7f3eec3ff700 1 -- 172.21.15.111:0/3058804047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ee406aaf0 msgr2=0x7f3ee4155d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:42.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.560+0000 7f3eec3ff700 1 --2- 172.21.15.111:0/3058804047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ee406aaf0 0x7f3ee4155d70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3ee000c4e0 tx=0x7f3ee000c8a0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.560+0000 7f3eec3ff700 1 -- 172.21.15.111:0/3058804047 shutdown_connections 2023-03-29T10:58:42.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.560+0000 7f3eec3ff700 1 --2- 172.21.15.111:0/3058804047 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3ec8069de0 0x7f3ec806c290 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.560+0000 7f3eec3ff700 1 --2- 172.21.15.111:0/3058804047 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3ee406aaf0 0x7f3ee4155d70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.560+0000 7f3eec3ff700 1 --2- 172.21.15.111:0/3058804047 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3ee415a5a0 0x7f3ee415b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.560+0000 7f3eec3ff700 1 --2- 172.21.15.111:0/3058804047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ee4071030 0x7f3ee4155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.560+0000 7f3eec3ff700 1 -- 172.21.15.111:0/3058804047 >> 172.21.15.111:0/3058804047 conn(0x7f3ee40cf720 msgr2=0x7f3ee4072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:42.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.560+0000 7f3eec3ff700 1 -- 172.21.15.111:0/3058804047 shutdown_connections 2023-03-29T10:58:42.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.560+0000 7f3eec3ff700 1 -- 172.21.15.111:0/3058804047 wait complete. 2023-03-29T10:58:42.577 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-29T10:58:42.763 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.763+0000 7f4870fff700 1 Processor -- start 2023-03-29T10:58:42.763 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.763+0000 7f4870fff700 1 -- start start 2023-03-29T10:58:42.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.764+0000 7f4870fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f486c15a5a0 0x7f486c06a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:42.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.764+0000 7f4870fff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486c0cb280 con 0x7f486c06a920 2023-03-29T10:58:42.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.764+0000 7f4870fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f486c071700 con 0x7f486c071200 2023-03-29T10:58:42.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.764+0000 7f4870fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f486c06adc0 con 0x7f486c15a5a0 2023-03-29T10:58:42.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.765+0000 7f4863fff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f486c06a920 0x7f486c070af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54582/0 (socket says 172.21.15.111:54582) 2023-03-29T10:58:42.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.765+0000 7f4863fff700 1 -- 172.21.15.111:0/2678362592 learned_addr learned my addr 172.21.15.111:0/2678362592 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:42.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.765+0000 7f4869d9b700 1 -- 172.21.15.111:0/2678362592 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686286260 0 0) 0x7f4854001290 con 0x7f486c06a920 2023-03-29T10:58:42.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.765+0000 7f4869d9b700 1 -- 172.21.15.111:0/2678362592 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48580035b0 con 0x7f486c06a920 2023-03-29T10:58:42.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.765+0000 7f486a59c700 1 --2- 172.21.15.111:0/2678362592 >> v2:172.21.15.119:3300/0 conn(0x7f486c15a5a0 0x7f486c06a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:42.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.765+0000 7f4869d9b700 1 -- 172.21.15.111:0/2678362592 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1404881277 0 0) 0x7f48580035b0 con 0x7f486c06a920 2023-03-29T10:58:42.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.765+0000 7f4869d9b700 1 -- 172.21.15.111:0/2678362592 >> v2:172.21.15.119:3300/0 conn(0x7f486c15a5a0 msgr2=0x7f486c06a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:42.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.765+0000 7f4869d9b700 1 --2- 172.21.15.111:0/2678362592 >> v2:172.21.15.119:3300/0 conn(0x7f486c15a5a0 0x7f486c06a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.765+0000 7f4869d9b700 1 -- 172.21.15.111:0/2678362592 >> v1:172.21.15.119:6789/0 conn(0x7f486c071200 legacy=0x7f486c06d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:42.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.765+0000 7f4869d9b700 1 -- 172.21.15.111:0/2678362592 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486c0719e0 con 0x7f486c06a920 2023-03-29T10:58:42.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.765+0000 7f4869d9b700 1 -- 172.21.15.111:0/2678362592 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f4854001980 con 0x7f486c06a920 2023-03-29T10:58:42.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.766+0000 7f4869d9b700 1 -- 172.21.15.111:0/2678362592 >> v1:172.21.15.111:6789/0 conn(0x7f486c06a920 legacy=0x7f486c070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:42.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.766+0000 7f4869d9b700 1 --2- 172.21.15.111:0/2678362592 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4858003a00 0x7f4858003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:42.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.766+0000 7f4869d9b700 1 --2- 172.21.15.111:0/2678362592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4858004850 0x7f4858004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:42.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.766+0000 7f4869d9b700 1 --2- 172.21.15.111:0/2678362592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4858005200 0x7f485800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:42.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.766+0000 7f4869d9b700 1 -- 172.21.15.111:0/2678362592 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485800c050 con 0x7f4858004850 2023-03-29T10:58:42.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.766+0000 7f4869d9b700 1 -- 172.21.15.111:0/2678362592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485800c1c0 con 0x7f4858005200 2023-03-29T10:58:42.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.766+0000 7f4869d9b700 1 -- 172.21.15.111:0/2678362592 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485800c380 con 0x7f4858003a00 2023-03-29T10:58:42.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.766+0000 7f4863fff700 1 --2- 172.21.15.111:0/2678362592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4858004850 0x7f4858004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:42.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.766+0000 7f486ad9d700 1 --2- 172.21.15.111:0/2678362592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4858005200 0x7f485800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:42.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.766+0000 7f486a59c700 1 --2- 172.21.15.111:0/2678362592 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4858003a00 0x7f4858003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:42.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.766+0000 7f4863fff700 1 -- 172.21.15.111:0/2678362592 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4858003a00 msgr2=0x7f4858003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:42.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.766+0000 7f4863fff700 1 --2- 172.21.15.111:0/2678362592 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4858003a00 0x7f4858003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.766+0000 7f4863fff700 1 -- 172.21.15.111:0/2678362592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4858005200 msgr2=0x7f485800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:42.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.766+0000 7f4863fff700 1 --2- 172.21.15.111:0/2678362592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4858005200 0x7f485800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.766+0000 7f4863fff700 1 -- 172.21.15.111:0/2678362592 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48580035b0 con 0x7f4858004850 2023-03-29T10:58:42.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.766+0000 7f486ad9d700 1 -- 172.21.15.111:0/2678362592 reap_dead start 2023-03-29T10:58:42.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.767+0000 7f4863fff700 1 --2- 172.21.15.111:0/2678362592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4858004850 0x7f4858004cc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f485400c210 tx=0x7f4858005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c491d0ea7011bd68 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:42.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.767+0000 7f4869d9b700 1 -- 172.21.15.111:0/2678362592 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485400cdb0 con 0x7f4858004850 2023-03-29T10:58:42.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.767+0000 7f4869d9b700 1 -- 172.21.15.111:0/2678362592 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4854007d40 con 0x7f4858004850 2023-03-29T10:58:42.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.767+0000 7f4869d9b700 1 -- 172.21.15.111:0/2678362592 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854007310 con 0x7f4858004850 2023-03-29T10:58:42.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.767+0000 7f4870fff700 1 -- 172.21.15.111:0/2678362592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4858004850 msgr2=0x7f4858004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:42.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.767+0000 7f4870fff700 1 --2- 172.21.15.111:0/2678362592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4858004850 0x7f4858004cc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f485400c210 tx=0x7f4858005880 comp rx=0 tx=0).stop 2023-03-29T10:58:42.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.767+0000 7f4870fff700 1 -- 172.21.15.111:0/2678362592 shutdown_connections 2023-03-29T10:58:42.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.767+0000 7f4870fff700 1 --2- 172.21.15.111:0/2678362592 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4858004850 0x7f4858004cc0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.767+0000 7f4870fff700 1 -- 172.21.15.111:0/2678362592 >> 172.21.15.111:0/2678362592 conn(0x7f486c0cf720 msgr2=0x7f486c0cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:42.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.767+0000 7f4870fff700 1 -- 172.21.15.111:0/2678362592 shutdown_connections 2023-03-29T10:58:42.768 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.767+0000 7f4870fff700 1 -- 172.21.15.111:0/2678362592 wait complete. 2023-03-29T10:58:42.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.770+0000 7f4870fff700 1 Processor -- start 2023-03-29T10:58:42.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.770+0000 7f4870fff700 1 -- start start 2023-03-29T10:58:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.771+0000 7f4870fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4858004850 0x7f486c152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.771+0000 7f4870fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f486c152ef0 0x7f486c0b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.771+0000 7f4870fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f486c0c2530 0x7f486c0c29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.771+0000 7f4870fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486c152c80 con 0x7f486c0c2530 2023-03-29T10:58:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.771+0000 7f4870fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486c0b4710 con 0x7f4858004850 2023-03-29T10:58:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.771+0000 7f4870fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486c15c670 con 0x7f486c152ef0 2023-03-29T10:58:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.771+0000 7f486ad9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f486c0c2530 0x7f486c0c29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.771+0000 7f486ad9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f486c0c2530 0x7f486c0c29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59070/0 (socket says 172.21.15.111:59070) 2023-03-29T10:58:42.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.771+0000 7f486ad9d700 1 -- 172.21.15.111:0/1513567558 learned_addr learned my addr 172.21.15.111:0/1513567558 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.772+0000 7f4863fff700 1 --2- 172.21.15.111:0/1513567558 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f486c152ef0 0x7f486c0b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.772+0000 7f486a59c700 1 --2- 172.21.15.111:0/1513567558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4858004850 0x7f486c152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.772+0000 7f486ad9d700 1 -- 172.21.15.111:0/1513567558 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f486c152ef0 msgr2=0x7f486c0b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.772+0000 7f486ad9d700 1 --2- 172.21.15.111:0/1513567558 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f486c152ef0 0x7f486c0b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.772+0000 7f486ad9d700 1 -- 172.21.15.111:0/1513567558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4858004850 msgr2=0x7f486c152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.772+0000 7f486ad9d700 1 --2- 172.21.15.111:0/1513567558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4858004850 0x7f486c152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.772+0000 7f486ad9d700 1 -- 172.21.15.111:0/1513567558 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486c06c9a0 con 0x7f486c0c2530 2023-03-29T10:58:42.772 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.772+0000 7f486ad9d700 1 --2- 172.21.15.111:0/1513567558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f486c0c2530 0x7f486c0c29d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f486c15df50 tx=0x7f486400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:42.773 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.772+0000 7f4862ffd700 1 -- 172.21.15.111:0/1513567558 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864003c10 con 0x7f486c0c2530 2023-03-29T10:58:42.773 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.772+0000 7f4862ffd700 1 -- 172.21.15.111:0/1513567558 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4864003d70 con 0x7f486c0c2530 2023-03-29T10:58:42.773 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.772+0000 7f4870fff700 1 -- 172.21.15.111:0/1513567558 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f486c06c6c0 con 0x7f486c0c2530 2023-03-29T10:58:42.773 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.772+0000 7f4862ffd700 1 -- 172.21.15.111:0/1513567558 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486400dbd0 con 0x7f486c0c2530 2023-03-29T10:58:42.773 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.772+0000 7f4870fff700 1 -- 172.21.15.111:0/1513567558 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f486c06cc80 con 0x7f486c0c2530 2023-03-29T10:58:42.773 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.773+0000 7f4870fff700 1 -- 172.21.15.111:0/1513567558 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f486c0ce3d0 con 0x7f486c0c2530 2023-03-29T10:58:42.774 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.774+0000 7f4862ffd700 1 -- 172.21.15.111:0/1513567558 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f486400dd30 con 0x7f486c0c2530 2023-03-29T10:58:42.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.775+0000 7f4862ffd700 1 --2- 172.21.15.111:0/1513567558 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f484c069de0 0x7f484c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:42.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.775+0000 7f4862ffd700 1 -- 172.21.15.111:0/1513567558 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4864088590 con 0x7f486c0c2530 2023-03-29T10:58:42.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.776+0000 7f486a59c700 1 --2- 172.21.15.111:0/1513567558 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f484c069de0 0x7f484c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:42.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.776+0000 7f486a59c700 1 --2- 172.21.15.111:0/1513567558 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f484c069de0 0x7f484c06c290 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f486c069690 tx=0x7f485c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:42.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.780+0000 7f4862ffd700 1 -- 172.21.15.111:0/1513567558 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f4864053b10 con 0x7f486c0c2530 2023-03-29T10:58:42.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.971+0000 7f4870fff700 1 -- 172.21.15.111:0/1513567558 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f486c0afed0 con 0x7f486c0c2530 2023-03-29T10:58:42.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.972+0000 7f4862ffd700 1 -- 172.21.15.111:0/1513567558 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4864057130 con 0x7f486c0c2530 2023-03-29T10:58:42.971 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 8 2023-03-29T10:58:42.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.974+0000 7f4870fff700 1 -- 172.21.15.111:0/1513567558 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f484c069de0 msgr2=0x7f484c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:42.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.974+0000 7f4870fff700 1 --2- 172.21.15.111:0/1513567558 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f484c069de0 0x7f484c06c290 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f486c069690 tx=0x7f485c008040 comp rx=0 tx=0).stop 2023-03-29T10:58:42.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.974+0000 7f4870fff700 1 -- 172.21.15.111:0/1513567558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f486c0c2530 msgr2=0x7f486c0c29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:42.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.974+0000 7f4870fff700 1 --2- 172.21.15.111:0/1513567558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f486c0c2530 0x7f486c0c29d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f486c15df50 tx=0x7f486400d430 comp rx=0 tx=0).stop 2023-03-29T10:58:42.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.974+0000 7f4870fff700 1 -- 172.21.15.111:0/1513567558 shutdown_connections 2023-03-29T10:58:42.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.974+0000 7f4870fff700 1 --2- 172.21.15.111:0/1513567558 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f484c069de0 0x7f484c06c290 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.974+0000 7f4870fff700 1 --2- 172.21.15.111:0/1513567558 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f486c0c2530 0x7f486c0c29d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.974+0000 7f4870fff700 1 --2- 172.21.15.111:0/1513567558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4858004850 0x7f486c152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.974+0000 7f4870fff700 1 --2- 172.21.15.111:0/1513567558 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f486c152ef0 0x7f486c0b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:42.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.975+0000 7f4870fff700 1 -- 172.21.15.111:0/1513567558 >> 172.21.15.111:0/1513567558 conn(0x7f486c0cf720 msgr2=0x7f486c070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:42.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.975+0000 7f4870fff700 1 -- 172.21.15.111:0/1513567558 shutdown_connections 2023-03-29T10:58:42.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:42.975+0000 7f4870fff700 1 -- 172.21.15.111:0/1513567558 wait complete. 2023-03-29T10:58:42.992 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-29T10:58:43.179 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.180+0000 7f5f7c3ff700 1 Processor -- start 2023-03-29T10:58:43.179 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.180+0000 7f5f7c3ff700 1 -- start start 2023-03-29T10:58:43.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.181+0000 7f5f7c3ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5f7406b4f0 0x7f5f7406b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:43.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.181+0000 7f5f7c3ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f740b26f0 con 0x7f5f740d0130 2023-03-29T10:58:43.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.181+0000 7f5f7c3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7406be00 con 0x7f5f740cbf90 2023-03-29T10:58:43.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.181+0000 7f5f7c3ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f740cfb60 con 0x7f5f7406b4f0 2023-03-29T10:58:43.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.181+0000 7f5f7999a700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f5f740d0130 0x7f5f740cf350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54586/0 (socket says 172.21.15.111:54586) 2023-03-29T10:58:43.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.181+0000 7f5f7999a700 1 -- 172.21.15.111:0/3945255628 learned_addr learned my addr 172.21.15.111:0/3945255628 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:43.182 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.181+0000 7f5f79199700 1 -- 172.21.15.111:0/3945255628 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 431907597 0 0) 0x7f5f740b26f0 con 0x7f5f740d0130 2023-03-29T10:58:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.181+0000 7f5f79199700 1 -- 172.21.15.111:0/3945255628 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f600035b0 con 0x7f5f740d0130 2023-03-29T10:58:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.181+0000 7f5f7a19b700 1 --2- 172.21.15.111:0/3945255628 >> v2:172.21.15.119:3300/0 conn(0x7f5f7406b4f0 0x7f5f7406b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.182+0000 7f5f79199700 1 -- 172.21.15.111:0/3945255628 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 319584959 0 0) 0x7f5f600035b0 con 0x7f5f740d0130 2023-03-29T10:58:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.182+0000 7f5f79199700 1 -- 172.21.15.111:0/3945255628 >> v2:172.21.15.119:3300/0 conn(0x7f5f7406b4f0 msgr2=0x7f5f7406b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.182+0000 7f5f79199700 1 --2- 172.21.15.111:0/3945255628 >> v2:172.21.15.119:3300/0 conn(0x7f5f7406b4f0 0x7f5f7406b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.182+0000 7f5f79199700 1 -- 172.21.15.111:0/3945255628 >> v1:172.21.15.119:6789/0 conn(0x7f5f740cbf90 legacy=0x7f5f740cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.182+0000 7f5f79199700 1 -- 172.21.15.111:0/3945255628 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f740cfe40 con 0x7f5f740d0130 2023-03-29T10:58:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.182+0000 7f5f79199700 1 -- 172.21.15.111:0/3945255628 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f5f64001670 con 0x7f5f740d0130 2023-03-29T10:58:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.182+0000 7f5f79199700 1 -- 172.21.15.111:0/3945255628 >> v1:172.21.15.111:6789/0 conn(0x7f5f740d0130 legacy=0x7f5f740cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.182+0000 7f5f79199700 1 --2- 172.21.15.111:0/3945255628 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5f60003a50 0x7f5f60003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.182+0000 7f5f79199700 1 --2- 172.21.15.111:0/3945255628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f60004850 0x7f5f60004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.182+0000 7f5f79199700 1 --2- 172.21.15.111:0/3945255628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f600052d0 0x7f5f6000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.182+0000 7f5f79199700 1 -- 172.21.15.111:0/3945255628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740cfb60 con 0x7f5f60004850 2023-03-29T10:58:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.182+0000 7f5f79199700 1 -- 172.21.15.111:0/3945255628 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6000c200 con 0x7f5f600052d0 2023-03-29T10:58:43.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.182+0000 7f5f79199700 1 -- 172.21.15.111:0/3945255628 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6000c3c0 con 0x7f5f60003a50 2023-03-29T10:58:43.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.182+0000 7f5f7999a700 1 --2- 172.21.15.111:0/3945255628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f60004850 0x7f5f60004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:43.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.182+0000 7f5f7a19b700 1 --2- 172.21.15.111:0/3945255628 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5f60003a50 0x7f5f60003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:43.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.182+0000 7f5f7a99c700 1 --2- 172.21.15.111:0/3945255628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f600052d0 0x7f5f6000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:43.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.183+0000 7f5f7999a700 1 -- 172.21.15.111:0/3945255628 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5f60003a50 msgr2=0x7f5f60003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:43.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.183+0000 7f5f7999a700 1 --2- 172.21.15.111:0/3945255628 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5f60003a50 0x7f5f60003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.183+0000 7f5f7999a700 1 -- 172.21.15.111:0/3945255628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f600052d0 msgr2=0x7f5f6000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:43.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.183+0000 7f5f7999a700 1 --2- 172.21.15.111:0/3945255628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f600052d0 0x7f5f6000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.183+0000 7f5f7999a700 1 -- 172.21.15.111:0/3945255628 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f600035b0 con 0x7f5f60004850 2023-03-29T10:58:43.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.183+0000 7f5f7a99c700 1 -- 172.21.15.111:0/3945255628 reap_dead start 2023-03-29T10:58:43.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.183+0000 7f5f7999a700 1 --2- 172.21.15.111:0/3945255628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f60004850 0x7f5f60004cc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400c160 tx=0x7f5f60005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cce5e76ee009fc9 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:43.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.183+0000 7f5f79199700 1 -- 172.21.15.111:0/3945255628 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64007b70 con 0x7f5f60004850 2023-03-29T10:58:43.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.183+0000 7f5f79199700 1 -- 172.21.15.111:0/3945255628 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6400cd20 con 0x7f5f60004850 2023-03-29T10:58:43.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.183+0000 7f5f79199700 1 -- 172.21.15.111:0/3945255628 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64005ba0 con 0x7f5f60004850 2023-03-29T10:58:43.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.184+0000 7f5f7c3ff700 1 -- 172.21.15.111:0/3945255628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f60004850 msgr2=0x7f5f60004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:43.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.184+0000 7f5f7c3ff700 1 --2- 172.21.15.111:0/3945255628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f60004850 0x7f5f60004cc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400c160 tx=0x7f5f60005880 comp rx=0 tx=0).stop 2023-03-29T10:58:43.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.184+0000 7f5f7c3ff700 1 -- 172.21.15.111:0/3945255628 shutdown_connections 2023-03-29T10:58:43.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.184+0000 7f5f7c3ff700 1 --2- 172.21.15.111:0/3945255628 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f60004850 0x7f5f60004cc0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.184+0000 7f5f7c3ff700 1 -- 172.21.15.111:0/3945255628 >> 172.21.15.111:0/3945255628 conn(0x7f5f7409a090 msgr2=0x7f5f740d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:43.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.184+0000 7f5f7c3ff700 1 -- 172.21.15.111:0/3945255628 shutdown_connections 2023-03-29T10:58:43.185 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.184+0000 7f5f7c3ff700 1 -- 172.21.15.111:0/3945255628 wait complete. 2023-03-29T10:58:43.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.185+0000 7f5f7c3ff700 1 Processor -- start 2023-03-29T10:58:43.186 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.186+0000 7f5f7c3ff700 1 -- start start 2023-03-29T10:58:43.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.186+0000 7f5f7c3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5f60004850 0x7f5f7415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:43.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.186+0000 7f5f7c3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f7415d410 0x7f5f7415ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:43.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.186+0000 7f5f7c3ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f7415f350 0x7f5f7415f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:43.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.187+0000 7f5f7c3ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7415d880 con 0x7f5f7415f350 2023-03-29T10:58:43.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.187+0000 7f5f7c3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7415fd10 con 0x7f5f7415d410 2023-03-29T10:58:43.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.187+0000 7f5f7c3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740b6910 con 0x7f5f60004850 2023-03-29T10:58:43.187 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.187+0000 7f5f7a99c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f7415f350 0x7f5f7415f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:43.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.187+0000 7f5f7a99c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f7415f350 0x7f5f7415f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59094/0 (socket says 172.21.15.111:59094) 2023-03-29T10:58:43.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.187+0000 7f5f7a99c700 1 -- 172.21.15.111:0/1390433179 learned_addr learned my addr 172.21.15.111:0/1390433179 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:43.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.187+0000 7f5f7a19b700 1 --2- 172.21.15.111:0/1390433179 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5f60004850 0x7f5f7415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:43.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.187+0000 7f5f7999a700 1 --2- 172.21.15.111:0/1390433179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f7415d410 0x7f5f7415ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:43.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.187+0000 7f5f7a99c700 1 -- 172.21.15.111:0/1390433179 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5f60004850 msgr2=0x7f5f7415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:43.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.187+0000 7f5f7a99c700 1 --2- 172.21.15.111:0/1390433179 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5f60004850 0x7f5f7415ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.187+0000 7f5f7a99c700 1 -- 172.21.15.111:0/1390433179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f7415d410 msgr2=0x7f5f7415ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:43.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.187+0000 7f5f7a99c700 1 --2- 172.21.15.111:0/1390433179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f7415d410 0x7f5f7415ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.187+0000 7f5f7a99c700 1 -- 172.21.15.111:0/1390433179 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f740cb790 con 0x7f5f7415f350 2023-03-29T10:58:43.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.187+0000 7f5f7a99c700 1 --2- 172.21.15.111:0/1390433179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f7415f350 0x7f5f7415f7d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5f74072650 tx=0x7f5f6c00d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:43.188 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.188+0000 7f5f6affd700 1 -- 172.21.15.111:0/1390433179 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c003cd0 con 0x7f5f7415f350 2023-03-29T10:58:43.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.188+0000 7f5f6affd700 1 -- 172.21.15.111:0/1390433179 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c003e70 con 0x7f5f7415f350 2023-03-29T10:58:43.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.188+0000 7f5f7c3ff700 1 -- 172.21.15.111:0/1390433179 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f740cb4b0 con 0x7f5f7415f350 2023-03-29T10:58:43.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.188+0000 7f5f6affd700 1 -- 172.21.15.111:0/1390433179 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c00d9f0 con 0x7f5f7415f350 2023-03-29T10:58:43.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.188+0000 7f5f7c3ff700 1 -- 172.21.15.111:0/1390433179 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f740cba70 con 0x7f5f7415f350 2023-03-29T10:58:43.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.189+0000 7f5f7c3ff700 1 -- 172.21.15.111:0/1390433179 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7406baf0 con 0x7f5f7415f350 2023-03-29T10:58:43.189 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.190+0000 7f5f6affd700 1 -- 172.21.15.111:0/1390433179 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f5f6c00db50 con 0x7f5f7415f350 2023-03-29T10:58:43.190 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.191+0000 7f5f6affd700 1 --2- 172.21.15.111:0/1390433179 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5f58069de0 0x7f5f5806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:43.190 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.191+0000 7f5f6affd700 1 -- 172.21.15.111:0/1390433179 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5f6c0885b0 con 0x7f5f7415f350 2023-03-29T10:58:43.190 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.191+0000 7f5f7a19b700 1 --2- 172.21.15.111:0/1390433179 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5f58069de0 0x7f5f5806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:43.191 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.192+0000 7f5f7a19b700 1 --2- 172.21.15.111:0/1390433179 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5f58069de0 0x7f5f5806c290 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5f70009c20 tx=0x7f5f70008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:43.194 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.195+0000 7f5f6affd700 1 -- 172.21.15.111:0/1390433179 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f5f6c053a90 con 0x7f5f7415f350 2023-03-29T10:58:43.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.386+0000 7f5f7c3ff700 1 -- 172.21.15.111:0/1390433179 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f740d1280 con 0x7f5f7415f350 2023-03-29T10:58:43.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.387+0000 7f5f6affd700 1 -- 172.21.15.111:0/1390433179 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5f6c0570b0 con 0x7f5f7415f350 2023-03-29T10:58:43.387 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:43.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.389+0000 7f5f7c3ff700 1 -- 172.21.15.111:0/1390433179 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5f58069de0 msgr2=0x7f5f5806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:43.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.389+0000 7f5f7c3ff700 1 --2- 172.21.15.111:0/1390433179 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5f58069de0 0x7f5f5806c290 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5f70009c20 tx=0x7f5f70008040 comp rx=0 tx=0).stop 2023-03-29T10:58:43.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.389+0000 7f5f7c3ff700 1 -- 172.21.15.111:0/1390433179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f7415f350 msgr2=0x7f5f7415f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:43.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.389+0000 7f5f7c3ff700 1 --2- 172.21.15.111:0/1390433179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f7415f350 0x7f5f7415f7d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5f74072650 tx=0x7f5f6c00d150 comp rx=0 tx=0).stop 2023-03-29T10:58:43.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.389+0000 7f5f7c3ff700 1 -- 172.21.15.111:0/1390433179 shutdown_connections 2023-03-29T10:58:43.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.389+0000 7f5f7c3ff700 1 --2- 172.21.15.111:0/1390433179 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5f58069de0 0x7f5f5806c290 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.389+0000 7f5f7c3ff700 1 --2- 172.21.15.111:0/1390433179 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5f7415f350 0x7f5f7415f7d0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.389+0000 7f5f7c3ff700 1 --2- 172.21.15.111:0/1390433179 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5f60004850 0x7f5f7415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.389+0000 7f5f7c3ff700 1 --2- 172.21.15.111:0/1390433179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f7415d410 0x7f5f7415ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.389+0000 7f5f7c3ff700 1 -- 172.21.15.111:0/1390433179 >> 172.21.15.111:0/1390433179 conn(0x7f5f7409a090 msgr2=0x7f5f740ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:43.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.389+0000 7f5f7c3ff700 1 -- 172.21.15.111:0/1390433179 shutdown_connections 2023-03-29T10:58:43.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.389+0000 7f5f7c3ff700 1 -- 172.21.15.111:0/1390433179 wait complete. 2023-03-29T10:58:43.407 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-29T10:58:43.591 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.592+0000 7f0f769ff700 1 Processor -- start 2023-03-29T10:58:43.592 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.592+0000 7f0f769ff700 1 -- start start 2023-03-29T10:58:43.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.593+0000 7f0f769ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0f7015c590 0x7f0f70160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:43.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.593+0000 7f0f769ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0f700cf010 0x7f0f700cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:43.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.593+0000 7f0f769ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f700cd520 con 0x7f0f700cfd80 2023-03-29T10:58:43.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.593+0000 7f0f769ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f700d2640 con 0x7f0f700cf010 2023-03-29T10:58:43.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.593+0000 7f0f769ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f701610d0 con 0x7f0f7015c590 2023-03-29T10:58:43.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.593+0000 7f0f74f9c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f0f700cfd80 0x7f0f700d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54602/0 (socket says 172.21.15.111:54602) 2023-03-29T10:58:43.593 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.593+0000 7f0f74f9c700 1 -- 172.21.15.111:0/2958162758 learned_addr learned my addr 172.21.15.111:0/2958162758 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:43.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.594+0000 7f0f6f7fe700 1 --2- 172.21.15.111:0/2958162758 >> v2:172.21.15.119:3300/0 conn(0x7f0f700cf010 0x7f0f700cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:43.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.594+0000 7f0f6effd700 1 -- 172.21.15.111:0/2958162758 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3611103194 0 0) 0x7f0f700cd520 con 0x7f0f700cfd80 2023-03-29T10:58:43.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.594+0000 7f0f6ffff700 1 --2- 172.21.15.111:0/2958162758 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0f7015c590 0x7f0f70160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.594+0000 7f0f6effd700 1 -- 172.21.15.111:0/2958162758 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f580035b0 con 0x7f0f700cfd80 2023-03-29T10:58:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.594+0000 7f0f6effd700 1 -- 172.21.15.111:0/2958162758 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2133188352 0 0) 0x7f0f580035b0 con 0x7f0f700cfd80 2023-03-29T10:58:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.594+0000 7f0f6effd700 1 -- 172.21.15.111:0/2958162758 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0f7015c590 msgr2=0x7f0f70160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.594+0000 7f0f6effd700 1 --2- 172.21.15.111:0/2958162758 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0f7015c590 0x7f0f70160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.594+0000 7f0f6effd700 1 -- 172.21.15.111:0/2958162758 >> v2:172.21.15.119:3300/0 conn(0x7f0f700cf010 msgr2=0x7f0f700cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.594+0000 7f0f6effd700 1 --2- 172.21.15.111:0/2958162758 >> v2:172.21.15.119:3300/0 conn(0x7f0f700cf010 0x7f0f700cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.594+0000 7f0f6effd700 1 -- 172.21.15.111:0/2958162758 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f700d0200 con 0x7f0f700cfd80 2023-03-29T10:58:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.594+0000 7f0f6effd700 1 -- 172.21.15.111:0/2958162758 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f0f680019f0 con 0x7f0f700cfd80 2023-03-29T10:58:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.594+0000 7f0f6effd700 1 -- 172.21.15.111:0/2958162758 >> v1:172.21.15.111:6789/0 conn(0x7f0f700cfd80 legacy=0x7f0f700d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.594+0000 7f0f6effd700 1 --2- 172.21.15.111:0/2958162758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f58003a00 0x7f0f58003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.594+0000 7f0f6effd700 1 --2- 172.21.15.111:0/2958162758 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0f7015c590 0x7f0f58004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.594+0000 7f0f6effd700 1 --2- 172.21.15.111:0/2958162758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f58004d90 0x7f0f58005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.594+0000 7f0f6effd700 1 -- 172.21.15.111:0/2958162758 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f700d2640 con 0x7f0f58004d90 2023-03-29T10:58:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.594+0000 7f0f6effd700 1 -- 172.21.15.111:0/2958162758 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f701610d0 con 0x7f0f58003a00 2023-03-29T10:58:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.594+0000 7f0f6effd700 1 -- 172.21.15.111:0/2958162758 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5800ba10 con 0x7f0f7015c590 2023-03-29T10:58:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.594+0000 7f0f74f9c700 1 --2- 172.21.15.111:0/2958162758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f58004d90 0x7f0f58005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:43.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.595+0000 7f0f6ffff700 1 --2- 172.21.15.111:0/2958162758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f58003a00 0x7f0f58003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:43.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.595+0000 7f0f6f7fe700 1 --2- 172.21.15.111:0/2958162758 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0f7015c590 0x7f0f58004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:43.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.595+0000 7f0f74f9c700 1 -- 172.21.15.111:0/2958162758 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0f7015c590 msgr2=0x7f0f58004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:43.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.595+0000 7f0f74f9c700 1 --2- 172.21.15.111:0/2958162758 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0f7015c590 0x7f0f58004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.595+0000 7f0f74f9c700 1 -- 172.21.15.111:0/2958162758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f58003a00 msgr2=0x7f0f58003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:43.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.595+0000 7f0f74f9c700 1 --2- 172.21.15.111:0/2958162758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f58003a00 0x7f0f58003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.595+0000 7f0f74f9c700 1 -- 172.21.15.111:0/2958162758 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f580035b0 con 0x7f0f58004d90 2023-03-29T10:58:43.596 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.595+0000 7f0f74f9c700 1 --2- 172.21.15.111:0/2958162758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f58004d90 0x7f0f58005200 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800a810 tx=0x7f0f58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e97cdd3c636c561 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:43.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.595+0000 7f0f6effd700 1 -- 172.21.15.111:0/2958162758 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6800a250 con 0x7f0f58004d90 2023-03-29T10:58:43.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.595+0000 7f0f6effd700 1 -- 172.21.15.111:0/2958162758 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f68012600 con 0x7f0f58004d90 2023-03-29T10:58:43.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.595+0000 7f0f6effd700 1 -- 172.21.15.111:0/2958162758 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f680129b0 con 0x7f0f58004d90 2023-03-29T10:58:43.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.596+0000 7f0f769ff700 1 -- 172.21.15.111:0/2958162758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f58004d90 msgr2=0x7f0f58005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:43.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.596+0000 7f0f769ff700 1 --2- 172.21.15.111:0/2958162758 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f58004d90 0x7f0f58005200 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800a810 tx=0x7f0f58005880 comp rx=0 tx=0).stop 2023-03-29T10:58:43.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.596+0000 7f0f74f9c700 1 -- 172.21.15.111:0/2958162758 reap_dead start 2023-03-29T10:58:43.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.596+0000 7f0f769ff700 1 -- 172.21.15.111:0/2958162758 shutdown_connections 2023-03-29T10:58:43.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.596+0000 7f0f769ff700 1 -- 172.21.15.111:0/2958162758 >> 172.21.15.111:0/2958162758 conn(0x7f0f70071b10 msgr2=0x7f0f70071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:43.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.596+0000 7f0f769ff700 1 -- 172.21.15.111:0/2958162758 shutdown_connections 2023-03-29T10:58:43.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.596+0000 7f0f769ff700 1 -- 172.21.15.111:0/2958162758 wait complete. 2023-03-29T10:58:43.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.598+0000 7f0f769ff700 1 Processor -- start 2023-03-29T10:58:43.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.598+0000 7f0f769ff700 1 -- start start 2023-03-29T10:58:43.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.600+0000 7f0f769ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f58004d90 0x7f0f701f1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:43.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.600+0000 7f0f769ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0f7015e250 0x7f0f701f5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:43.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.600+0000 7f0f769ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f701f5b50 0x7f0f700b6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:43.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.600+0000 7f0f769ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7015d720 con 0x7f0f701f5b50 2023-03-29T10:58:43.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.600+0000 7f0f769ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f701f5fc0 con 0x7f0f58004d90 2023-03-29T10:58:43.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.600+0000 7f0f769ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f701f1e60 con 0x7f0f7015e250 2023-03-29T10:58:43.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.600+0000 7f0f74f9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f701f5b50 0x7f0f700b6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:43.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.600+0000 7f0f74f9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f701f5b50 0x7f0f700b6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59114/0 (socket says 172.21.15.111:59114) 2023-03-29T10:58:43.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.600+0000 7f0f74f9c700 1 -- 172.21.15.111:0/3333494975 learned_addr learned my addr 172.21.15.111:0/3333494975 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:43.600 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.600+0000 7f0f6ffff700 1 --2- 172.21.15.111:0/3333494975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f58004d90 0x7f0f701f1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:43.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.600+0000 7f0f6f7fe700 1 --2- 172.21.15.111:0/3333494975 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0f7015e250 0x7f0f701f5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:43.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.600+0000 7f0f74f9c700 1 -- 172.21.15.111:0/3333494975 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0f7015e250 msgr2=0x7f0f701f5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:43.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.600+0000 7f0f74f9c700 1 --2- 172.21.15.111:0/3333494975 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0f7015e250 0x7f0f701f5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.600+0000 7f0f74f9c700 1 -- 172.21.15.111:0/3333494975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f58004d90 msgr2=0x7f0f701f1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:43.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.600+0000 7f0f74f9c700 1 --2- 172.21.15.111:0/3333494975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f58004d90 0x7f0f701f1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.600+0000 7f0f74f9c700 1 -- 172.21.15.111:0/3333494975 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f700cf560 con 0x7f0f701f5b50 2023-03-29T10:58:43.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.600+0000 7f0f6ffff700 1 --2- 172.21.15.111:0/3333494975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f58004d90 0x7f0f701f1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:43.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.601+0000 7f0f74f9c700 1 --2- 172.21.15.111:0/3333494975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f701f5b50 0x7f0f700b6d60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0f68012000 tx=0x7f0f700d25b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:43.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.601+0000 7f0f6cff9700 1 -- 172.21.15.111:0/3333494975 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6800a250 con 0x7f0f701f5b50 2023-03-29T10:58:43.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.601+0000 7f0f6cff9700 1 -- 172.21.15.111:0/3333494975 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6801e610 con 0x7f0f701f5b50 2023-03-29T10:58:43.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.601+0000 7f0f769ff700 1 -- 172.21.15.111:0/3333494975 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f700b23c0 con 0x7f0f701f5b50 2023-03-29T10:58:43.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.601+0000 7f0f6cff9700 1 -- 172.21.15.111:0/3333494975 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6801e7a0 con 0x7f0f701f5b50 2023-03-29T10:58:43.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.601+0000 7f0f769ff700 1 -- 172.21.15.111:0/3333494975 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f700cd770 con 0x7f0f701f5b50 2023-03-29T10:58:43.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.602+0000 7f0f769ff700 1 -- 172.21.15.111:0/3333494975 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f700b51f0 con 0x7f0f701f5b50 2023-03-29T10:58:43.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.603+0000 7f0f6cff9700 1 -- 172.21.15.111:0/3333494975 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f0f68019030 con 0x7f0f701f5b50 2023-03-29T10:58:43.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.604+0000 7f0f6cff9700 1 --2- 172.21.15.111:0/3333494975 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0f5c069de0 0x7f0f5c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:43.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.604+0000 7f0f6cff9700 1 -- 172.21.15.111:0/3333494975 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0f6808a1f0 con 0x7f0f701f5b50 2023-03-29T10:58:43.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.604+0000 7f0f6ffff700 1 --2- 172.21.15.111:0/3333494975 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0f5c069de0 0x7f0f5c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:43.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.605+0000 7f0f6ffff700 1 --2- 172.21.15.111:0/3333494975 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0f5c069de0 0x7f0f5c06c290 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009be0 tx=0x7f0f60008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:43.607 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.608+0000 7f0f6cff9700 1 -- 172.21.15.111:0/3333494975 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f0f680557a0 con 0x7f0f701f5b50 2023-03-29T10:58:43.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.801+0000 7f0f769ff700 1 -- 172.21.15.111:0/3333494975 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f700d1860 con 0x7f0f701f5b50 2023-03-29T10:58:43.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.801+0000 7f0f6cff9700 1 -- 172.21.15.111:0/3333494975 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f68058dc0 con 0x7f0f701f5b50 2023-03-29T10:58:43.801 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-03-29T10:58:43.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.803+0000 7f0f769ff700 1 -- 172.21.15.111:0/3333494975 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0f5c069de0 msgr2=0x7f0f5c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:43.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.803+0000 7f0f769ff700 1 --2- 172.21.15.111:0/3333494975 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0f5c069de0 0x7f0f5c06c290 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009be0 tx=0x7f0f60008040 comp rx=0 tx=0).stop 2023-03-29T10:58:43.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.804+0000 7f0f769ff700 1 -- 172.21.15.111:0/3333494975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f701f5b50 msgr2=0x7f0f700b6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:43.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.804+0000 7f0f769ff700 1 --2- 172.21.15.111:0/3333494975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f701f5b50 0x7f0f700b6d60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0f68012000 tx=0x7f0f700d25b0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.804+0000 7f0f769ff700 1 -- 172.21.15.111:0/3333494975 shutdown_connections 2023-03-29T10:58:43.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.804+0000 7f0f769ff700 1 --2- 172.21.15.111:0/3333494975 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0f5c069de0 0x7f0f5c06c290 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.804+0000 7f0f769ff700 1 --2- 172.21.15.111:0/3333494975 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0f701f5b50 0x7f0f700b6d60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.804+0000 7f0f769ff700 1 --2- 172.21.15.111:0/3333494975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f58004d90 0x7f0f701f1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.804+0000 7f0f769ff700 1 --2- 172.21.15.111:0/3333494975 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f0f7015e250 0x7f0f701f5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:43.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.804+0000 7f0f769ff700 1 -- 172.21.15.111:0/3333494975 >> 172.21.15.111:0/3333494975 conn(0x7f0f70071b10 msgr2=0x7f0f70072e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:43.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.804+0000 7f0f769ff700 1 -- 172.21.15.111:0/3333494975 shutdown_connections 2023-03-29T10:58:43.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:43.804+0000 7f0f769ff700 1 -- 172.21.15.111:0/3333494975 wait complete. 2023-03-29T10:58:43.821 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-29T10:58:44.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.011+0000 7f2491fff700 1 Processor -- start 2023-03-29T10:58:44.011 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.011+0000 7f2491fff700 1 -- start start 2023-03-29T10:58:44.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.012+0000 7f2491fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f248c0b87a0 0x7f248c0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:44.012 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.012+0000 7f2491fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f248c1601e0 0x7f248c160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:44.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.012+0000 7f2491fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248c0cdc60 con 0x7f248c160b70 2023-03-29T10:58:44.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.012+0000 7f2491fff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248c0d02d0 con 0x7f248c0b87a0 2023-03-29T10:58:44.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.012+0000 7f2491fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248c0d0cb0 con 0x7f248c1601e0 2023-03-29T10:58:44.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.012+0000 7f248b7fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f248c0b87a0 0x7f248c0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:44.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.012+0000 7f248b7fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f248c0b87a0 0x7f248c0cfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59120/0 (socket says 172.21.15.111:59120) 2023-03-29T10:58:44.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.012+0000 7f248b7fe700 1 -- 172.21.15.111:0/2579932784 learned_addr learned my addr 172.21.15.111:0/2579932784 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:44.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.013+0000 7f248affd700 1 --2- 172.21.15.111:0/2579932784 >> v2:172.21.15.119:3300/0 conn(0x7f248c1601e0 0x7f248c160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:44.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.013+0000 7f248b7fe700 1 -- 172.21.15.111:0/2579932784 >> v2:172.21.15.119:3300/0 conn(0x7f248c1601e0 msgr2=0x7f248c160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:44.013 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.013+0000 7f248b7fe700 1 --2- 172.21.15.111:0/2579932784 >> v2:172.21.15.119:3300/0 conn(0x7f248c1601e0 0x7f248c160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.013+0000 7f248b7fe700 1 -- 172.21.15.111:0/2579932784 >> v1:172.21.15.119:6789/0 conn(0x7f248c160b70 legacy=0x7f248c0d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:44.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.013+0000 7f248b7fe700 1 -- 172.21.15.111:0/2579932784 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c0cd6a0 con 0x7f248c0b87a0 2023-03-29T10:58:44.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.013+0000 7f248affd700 1 --2- 172.21.15.111:0/2579932784 >> v2:172.21.15.119:3300/0 conn(0x7f248c1601e0 0x7f248c160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:44.014 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.013+0000 7f248b7fe700 1 --2- 172.21.15.111:0/2579932784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f248c0b87a0 0x7f248c0cfbf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f248c0b8f60 tx=0x7f247c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1479a0e1287bfab1 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:44.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.013+0000 7f248a7fc700 1 -- 172.21.15.111:0/2579932784 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c004230 con 0x7f248c0b87a0 2023-03-29T10:58:44.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.013+0000 7f248a7fc700 1 -- 172.21.15.111:0/2579932784 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247c004390 con 0x7f248c0b87a0 2023-03-29T10:58:44.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.013+0000 7f248a7fc700 1 -- 172.21.15.111:0/2579932784 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c0036a0 con 0x7f248c0b87a0 2023-03-29T10:58:44.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.014+0000 7f2491fff700 1 -- 172.21.15.111:0/2579932784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f248c0b87a0 msgr2=0x7f248c0cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:44.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.014+0000 7f2491fff700 1 --2- 172.21.15.111:0/2579932784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f248c0b87a0 0x7f248c0cfbf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f248c0b8f60 tx=0x7f247c009c00 comp rx=0 tx=0).stop 2023-03-29T10:58:44.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.014+0000 7f2491fff700 1 -- 172.21.15.111:0/2579932784 shutdown_connections 2023-03-29T10:58:44.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.014+0000 7f2491fff700 1 --2- 172.21.15.111:0/2579932784 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f248c0b87a0 0x7f248c0cfbf0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.014+0000 7f2491fff700 1 --2- 172.21.15.111:0/2579932784 >> v2:172.21.15.119:3300/0 conn(0x7f248c1601e0 0x7f248c160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.014+0000 7f2491fff700 1 -- 172.21.15.111:0/2579932784 >> 172.21.15.111:0/2579932784 conn(0x7f248c071b10 msgr2=0x7f248c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:44.015 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.014+0000 7f2491fff700 1 -- 172.21.15.111:0/2579932784 shutdown_connections 2023-03-29T10:58:44.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.014+0000 7f2491fff700 1 -- 172.21.15.111:0/2579932784 wait complete. 2023-03-29T10:58:44.016 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.016+0000 7f2491fff700 1 Processor -- start 2023-03-29T10:58:44.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.016+0000 7f2491fff700 1 -- start start 2023-03-29T10:58:44.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.016+0000 7f2491fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f248c0b87a0 0x7f248c1580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:44.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.016+0000 7f2491fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f248c1601e0 0x7f248c0cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:44.017 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.017+0000 7f2491fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f248c160b70 0x7f248c150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:44.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.017+0000 7f2491fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c157ce0 con 0x7f248c1601e0 2023-03-29T10:58:44.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.017+0000 7f2491fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c0cb7c0 con 0x7f248c0b87a0 2023-03-29T10:58:44.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.017+0000 7f2491fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248c0cbaa0 con 0x7f248c160b70 2023-03-29T10:58:44.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.017+0000 7f248affd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f248c1601e0 0x7f248c0cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:44.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.017+0000 7f248affd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f248c1601e0 0x7f248c0cae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59124/0 (socket says 172.21.15.111:59124) 2023-03-29T10:58:44.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.017+0000 7f248affd700 1 -- 172.21.15.111:0/3309952813 learned_addr learned my addr 172.21.15.111:0/3309952813 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:44.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.017+0000 7f248b7fe700 1 --2- 172.21.15.111:0/3309952813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f248c0b87a0 0x7f248c1580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:44.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.017+0000 7f248affd700 1 -- 172.21.15.111:0/3309952813 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f248c160b70 msgr2=0x7f248c150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:44.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.017+0000 7f248affd700 1 --2- 172.21.15.111:0/3309952813 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f248c160b70 0x7f248c150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.017+0000 7f248affd700 1 -- 172.21.15.111:0/3309952813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f248c0b87a0 msgr2=0x7f248c1580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:44.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.017+0000 7f248affd700 1 --2- 172.21.15.111:0/3309952813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f248c0b87a0 0x7f248c1580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.017+0000 7f248affd700 1 -- 172.21.15.111:0/3309952813 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c009550 con 0x7f248c1601e0 2023-03-29T10:58:44.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.017+0000 7f248affd700 1 --2- 172.21.15.111:0/3309952813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f248c1601e0 0x7f248c0cae40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f248000c750 tx=0x7f248000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:44.018 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.018+0000 7f2473fff700 1 -- 172.21.15.111:0/3309952813 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480011800 con 0x7f248c1601e0 2023-03-29T10:58:44.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.018+0000 7f2473fff700 1 -- 172.21.15.111:0/3309952813 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2480011960 con 0x7f248c1601e0 2023-03-29T10:58:44.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.018+0000 7f2491fff700 1 -- 172.21.15.111:0/3309952813 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248c160f90 con 0x7f248c1601e0 2023-03-29T10:58:44.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.018+0000 7f2473fff700 1 -- 172.21.15.111:0/3309952813 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480011cd0 con 0x7f248c1601e0 2023-03-29T10:58:44.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.018+0000 7f2491fff700 1 -- 172.21.15.111:0/3309952813 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248c0cd980 con 0x7f248c1601e0 2023-03-29T10:58:44.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.019+0000 7f2491fff700 1 -- 172.21.15.111:0/3309952813 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f248c0d2110 con 0x7f248c1601e0 2023-03-29T10:58:44.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.020+0000 7f2473fff700 1 -- 172.21.15.111:0/3309952813 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f2480021350 con 0x7f248c1601e0 2023-03-29T10:58:44.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.020+0000 7f2473fff700 1 --2- 172.21.15.111:0/3309952813 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2478069de0 0x7f247806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:44.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.021+0000 7f2473fff700 1 -- 172.21.15.111:0/3309952813 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f24800882d0 con 0x7f248c1601e0 2023-03-29T10:58:44.020 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.021+0000 7f248b7fe700 1 --2- 172.21.15.111:0/3309952813 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2478069de0 0x7f247806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:44.021 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.021+0000 7f248b7fe700 1 --2- 172.21.15.111:0/3309952813 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2478069de0 0x7f247806c290 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f247c009130 tx=0x7f247c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:44.024 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.025+0000 7f2473fff700 1 -- 172.21.15.111:0/3309952813 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f2480057280 con 0x7f248c1601e0 2023-03-29T10:58:44.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.220+0000 7f2491fff700 1 -- 172.21.15.111:0/3309952813 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f248c074430 con 0x7f248c1601e0 2023-03-29T10:58:44.220 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.221+0000 7f2473fff700 1 -- 172.21.15.111:0/3309952813 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2480056e10 con 0x7f248c1601e0 2023-03-29T10:58:44.221 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:44.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.223+0000 7f2491fff700 1 -- 172.21.15.111:0/3309952813 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2478069de0 msgr2=0x7f247806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:44.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.223+0000 7f2491fff700 1 --2- 172.21.15.111:0/3309952813 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2478069de0 0x7f247806c290 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f247c009130 tx=0x7f247c008030 comp rx=0 tx=0).stop 2023-03-29T10:58:44.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.223+0000 7f2491fff700 1 -- 172.21.15.111:0/3309952813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f248c1601e0 msgr2=0x7f248c0cae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:44.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.223+0000 7f2491fff700 1 --2- 172.21.15.111:0/3309952813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f248c1601e0 0x7f248c0cae40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f248000c750 tx=0x7f248000cb10 comp rx=0 tx=0).stop 2023-03-29T10:58:44.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.223+0000 7f2491fff700 1 -- 172.21.15.111:0/3309952813 shutdown_connections 2023-03-29T10:58:44.223 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.223+0000 7f2491fff700 1 --2- 172.21.15.111:0/3309952813 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f2478069de0 0x7f247806c290 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.223+0000 7f2491fff700 1 --2- 172.21.15.111:0/3309952813 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f248c160b70 0x7f248c150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.223+0000 7f2491fff700 1 --2- 172.21.15.111:0/3309952813 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f248c0b87a0 0x7f248c1580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.223+0000 7f2491fff700 1 --2- 172.21.15.111:0/3309952813 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f248c1601e0 0x7f248c0cae40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.223+0000 7f2491fff700 1 -- 172.21.15.111:0/3309952813 >> 172.21.15.111:0/3309952813 conn(0x7f248c071b10 msgr2=0x7f248c071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:44.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.224+0000 7f2491fff700 1 -- 172.21.15.111:0/3309952813 shutdown_connections 2023-03-29T10:58:44.224 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.224+0000 7f2491fff700 1 -- 172.21.15.111:0/3309952813 wait complete. 2023-03-29T10:58:44.240 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-03-29T10:58:44.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.424+0000 7fae5f5ff700 1 Processor -- start 2023-03-29T10:58:44.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.425+0000 7fae5f5ff700 1 -- start start 2023-03-29T10:58:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.426+0000 7fae5f5ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fae581589d0 0x7fae580cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.426+0000 7fae5f5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae580cd690 0x7fae5806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.426+0000 7fae5f5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae580ca890 con 0x7fae58158da0 2023-03-29T10:58:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.426+0000 7fae5f5ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae580ca5b0 con 0x7fae581589d0 2023-03-29T10:58:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.426+0000 7fae5f5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae580cc670 con 0x7fae580cd690 2023-03-29T10:58:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.426+0000 7fae5d39b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fae581589d0 0x7fae580cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:44.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.427+0000 7fae5d39b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fae581589d0 0x7fae580cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59138/0 (socket says 172.21.15.111:59138) 2023-03-29T10:58:44.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.427+0000 7fae5d39b700 1 -- 172.21.15.111:0/2528935822 learned_addr learned my addr 172.21.15.111:0/2528935822 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:44.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.427+0000 7fae5cb9a700 1 --2- 172.21.15.111:0/2528935822 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae580cd690 0x7fae5806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:44.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.427+0000 7fae5d39b700 1 -- 172.21.15.111:0/2528935822 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae580cd690 msgr2=0x7fae5806c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:44.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.427+0000 7fae5d39b700 1 --2- 172.21.15.111:0/2528935822 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae580cd690 0x7fae5806c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.427+0000 7fae5d39b700 1 -- 172.21.15.111:0/2528935822 >> v1:172.21.15.119:6789/0 conn(0x7fae58158da0 legacy=0x7fae5806a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:44.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.427+0000 7fae5d39b700 1 -- 172.21.15.111:0/2528935822 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae580cc950 con 0x7fae581589d0 2023-03-29T10:58:44.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.427+0000 7fae5d39b700 1 --2- 172.21.15.111:0/2528935822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae581589d0 0x7fae580cd150 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fae5400bb90 tx=0x7fae5400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc604b7efaaf929 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:44.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.427+0000 7fae4ffff700 1 -- 172.21.15.111:0/2528935822 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5400dd30 con 0x7fae581589d0 2023-03-29T10:58:44.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.428+0000 7fae4ffff700 1 -- 172.21.15.111:0/2528935822 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5400de90 con 0x7fae581589d0 2023-03-29T10:58:44.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.428+0000 7fae4ffff700 1 -- 172.21.15.111:0/2528935822 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae54010690 con 0x7fae581589d0 2023-03-29T10:58:44.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.428+0000 7fae5f5ff700 1 -- 172.21.15.111:0/2528935822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae581589d0 msgr2=0x7fae580cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:44.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.428+0000 7fae5f5ff700 1 --2- 172.21.15.111:0/2528935822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae581589d0 0x7fae580cd150 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fae5400bb90 tx=0x7fae5400bea0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.428+0000 7fae5f5ff700 1 -- 172.21.15.111:0/2528935822 shutdown_connections 2023-03-29T10:58:44.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.428+0000 7fae5f5ff700 1 --2- 172.21.15.111:0/2528935822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae581589d0 0x7fae580cd150 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.428+0000 7fae5f5ff700 1 --2- 172.21.15.111:0/2528935822 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae580cd690 0x7fae5806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.428+0000 7fae5f5ff700 1 -- 172.21.15.111:0/2528935822 >> 172.21.15.111:0/2528935822 conn(0x7fae580cf720 msgr2=0x7fae580cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:44.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.428+0000 7fae5f5ff700 1 -- 172.21.15.111:0/2528935822 shutdown_connections 2023-03-29T10:58:44.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.428+0000 7fae5f5ff700 1 -- 172.21.15.111:0/2528935822 wait complete. 2023-03-29T10:58:44.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.430+0000 7fae5f5ff700 1 Processor -- start 2023-03-29T10:58:44.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.430+0000 7fae5f5ff700 1 -- start start 2023-03-29T10:58:44.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.431+0000 7fae5f5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae580cd690 0x7fae5814b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:44.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.431+0000 7fae5f5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae581589d0 0x7fae580b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:44.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.431+0000 7fae5f5ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae58158da0 0x7fae580acc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:44.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.431+0000 7fae5f5ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae580ada10 con 0x7fae58158da0 2023-03-29T10:58:44.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.431+0000 7fae5f5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae580ad730 con 0x7fae581589d0 2023-03-29T10:58:44.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.431+0000 7fae5f5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae580b19d0 con 0x7fae580cd690 2023-03-29T10:58:44.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.431+0000 7fae5db9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae58158da0 0x7fae580acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:44.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.431+0000 7fae5cb9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae581589d0 0x7fae580b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:44.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.431+0000 7fae5db9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae58158da0 0x7fae580acc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59140/0 (socket says 172.21.15.111:59140) 2023-03-29T10:58:44.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.431+0000 7fae5db9c700 1 -- 172.21.15.111:0/700198293 learned_addr learned my addr 172.21.15.111:0/700198293 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:44.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.431+0000 7fae5d39b700 1 --2- 172.21.15.111:0/700198293 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae580cd690 0x7fae5814b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:44.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.431+0000 7fae5db9c700 1 -- 172.21.15.111:0/700198293 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae580cd690 msgr2=0x7fae5814b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:44.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.431+0000 7fae5db9c700 1 --2- 172.21.15.111:0/700198293 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae580cd690 0x7fae5814b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.431+0000 7fae5db9c700 1 -- 172.21.15.111:0/700198293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae581589d0 msgr2=0x7fae580b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:44.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.431+0000 7fae5db9c700 1 --2- 172.21.15.111:0/700198293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae581589d0 0x7fae580b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.431+0000 7fae5db9c700 1 -- 172.21.15.111:0/700198293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5400b7f0 con 0x7fae58158da0 2023-03-29T10:58:44.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.432+0000 7fae5db9c700 1 --2- 172.21.15.111:0/700198293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae58158da0 0x7fae580acc70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fae5000cbc0 tx=0x7fae5000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:44.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.432+0000 7fae4dffb700 1 -- 172.21.15.111:0/700198293 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae50003c00 con 0x7fae58158da0 2023-03-29T10:58:44.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.432+0000 7fae4dffb700 1 -- 172.21.15.111:0/700198293 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae50003d60 con 0x7fae58158da0 2023-03-29T10:58:44.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.432+0000 7fae4dffb700 1 -- 172.21.15.111:0/700198293 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5000aa20 con 0x7fae58158da0 2023-03-29T10:58:44.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.432+0000 7fae5f5ff700 1 -- 172.21.15.111:0/700198293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5806aaf0 con 0x7fae58158da0 2023-03-29T10:58:44.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.432+0000 7fae5f5ff700 1 -- 172.21.15.111:0/700198293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae580ccc30 con 0x7fae58158da0 2023-03-29T10:58:44.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.433+0000 7fae5f5ff700 1 -- 172.21.15.111:0/700198293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae5806ac80 con 0x7fae58158da0 2023-03-29T10:58:44.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.434+0000 7fae4dffb700 1 -- 172.21.15.111:0/700198293 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fae5000ab80 con 0x7fae58158da0 2023-03-29T10:58:44.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.435+0000 7fae4dffb700 1 --2- 172.21.15.111:0/700198293 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fae3c069de0 0x7fae3c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:44.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.435+0000 7fae4dffb700 1 -- 172.21.15.111:0/700198293 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fae500881d0 con 0x7fae58158da0 2023-03-29T10:58:44.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.435+0000 7fae5d39b700 1 --2- 172.21.15.111:0/700198293 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fae3c069de0 0x7fae3c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:44.435 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.436+0000 7fae5d39b700 1 --2- 172.21.15.111:0/700198293 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fae3c069de0 0x7fae3c06c290 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fae5400bee0 tx=0x7fae5401c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:44.438 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.439+0000 7fae4dffb700 1 -- 172.21.15.111:0/700198293 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fae50053800 con 0x7fae58158da0 2023-03-29T10:58:44.647 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.648+0000 7fae5f5ff700 1 -- 172.21.15.111:0/700198293 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fae580cdc30 con 0x7fae58158da0 2023-03-29T10:58:44.647 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.648+0000 7fae4dffb700 1 -- 172.21.15.111:0/700198293 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7fae50056e20 con 0x7fae58158da0 2023-03-29T10:58:44.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.650+0000 7fae5f5ff700 1 -- 172.21.15.111:0/700198293 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fae3c069de0 msgr2=0x7fae3c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:44.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.650+0000 7fae5f5ff700 1 --2- 172.21.15.111:0/700198293 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fae3c069de0 0x7fae3c06c290 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fae5400bee0 tx=0x7fae5401c040 comp rx=0 tx=0).stop 2023-03-29T10:58:44.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.650+0000 7fae5f5ff700 1 -- 172.21.15.111:0/700198293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae58158da0 msgr2=0x7fae580acc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:44.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.650+0000 7fae5f5ff700 1 --2- 172.21.15.111:0/700198293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae58158da0 0x7fae580acc70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fae5000cbc0 tx=0x7fae5000ced0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.651+0000 7fae5f5ff700 1 -- 172.21.15.111:0/700198293 shutdown_connections 2023-03-29T10:58:44.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.651+0000 7fae5f5ff700 1 --2- 172.21.15.111:0/700198293 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fae3c069de0 0x7fae3c06c290 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.651+0000 7fae5f5ff700 1 --2- 172.21.15.111:0/700198293 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fae58158da0 0x7fae580acc70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.651+0000 7fae5f5ff700 1 --2- 172.21.15.111:0/700198293 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fae580cd690 0x7fae5814b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.651+0000 7fae5f5ff700 1 --2- 172.21.15.111:0/700198293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae581589d0 0x7fae580b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.651+0000 7fae5f5ff700 1 -- 172.21.15.111:0/700198293 >> 172.21.15.111:0/700198293 conn(0x7fae580cf720 msgr2=0x7fae580c9840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:44.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.651+0000 7fae5f5ff700 1 -- 172.21.15.111:0/700198293 shutdown_connections 2023-03-29T10:58:44.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.651+0000 7fae5f5ff700 1 -- 172.21.15.111:0/700198293 wait complete. 2023-03-29T10:58:44.651 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-03-29T10:58:44.667 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:58:44.667 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-29T10:58:44.707 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:58:44.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.930+0000 7f02b15ff700 1 Processor -- start 2023-03-29T10:58:44.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.930+0000 7f02b15ff700 1 -- start start 2023-03-29T10:58:44.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.931+0000 7f02b15ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f02ac0cf970 0x7f02ac0cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:44.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.931+0000 7f02b15ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02ac0cdc10 con 0x7f02ac0cfd40 2023-03-29T10:58:44.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.931+0000 7f02b15ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02ac0b6c60 con 0x7f02ac0b70f0 2023-03-29T10:58:44.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.931+0000 7f02b15ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ac0d08d0 con 0x7f02ac0cf970 2023-03-29T10:58:44.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.931+0000 7f02aaffd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f02ac0cf970 0x7f02ac0cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:44.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.931+0000 7f02a3fff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f02ac0cfd40 0x7f02ac0d01c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54608/0 (socket says 172.21.15.111:54608) 2023-03-29T10:58:44.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.931+0000 7f02a3fff700 1 -- 172.21.15.111:0/3403410290 learned_addr learned my addr 172.21.15.111:0/3403410290 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:44.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.931+0000 7f02aaffd700 1 -- 172.21.15.111:0/3403410290 >> v1:172.21.15.119:6789/0 conn(0x7f02ac0b70f0 legacy=0x7f02ac0b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:44.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.932+0000 7f02aaffd700 1 -- 172.21.15.111:0/3403410290 >> v1:172.21.15.111:6789/0 conn(0x7f02ac0cfd40 legacy=0x7f02ac0d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:44.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.932+0000 7f02aaffd700 1 -- 172.21.15.111:0/3403410290 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ac0cd650 con 0x7f02ac0cf970 2023-03-29T10:58:44.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.932+0000 7f02aa7fc700 1 -- 172.21.15.111:0/3403410290 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428998339 0 0) 0x7f02ac0cdc10 con 0x7f02ac0cfd40 2023-03-29T10:58:44.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.932+0000 7f02aaffd700 1 --2- 172.21.15.111:0/3403410290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02ac0cf970 0x7f02ac0cf110 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f029c009840 tx=0x7f029c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e71fe500217ca30 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:44.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.932+0000 7f02aa7fc700 1 -- 172.21.15.111:0/3403410290 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c011830 con 0x7f02ac0cf970 2023-03-29T10:58:44.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.932+0000 7f02aa7fc700 1 -- 172.21.15.111:0/3403410290 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029c011990 con 0x7f02ac0cf970 2023-03-29T10:58:44.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.932+0000 7f02aa7fc700 1 -- 172.21.15.111:0/3403410290 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c011d20 con 0x7f02ac0cf970 2023-03-29T10:58:44.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.932+0000 7f02b15ff700 1 -- 172.21.15.111:0/3403410290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02ac0cf970 msgr2=0x7f02ac0cf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:44.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.933+0000 7f02b15ff700 1 --2- 172.21.15.111:0/3403410290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02ac0cf970 0x7f02ac0cf110 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f029c009840 tx=0x7f029c009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:44.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.933+0000 7f02b15ff700 1 -- 172.21.15.111:0/3403410290 shutdown_connections 2023-03-29T10:58:44.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.933+0000 7f02b15ff700 1 --2- 172.21.15.111:0/3403410290 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02ac0cf970 0x7f02ac0cf110 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.933+0000 7f02b15ff700 1 -- 172.21.15.111:0/3403410290 >> 172.21.15.111:0/3403410290 conn(0x7f02ac071b10 msgr2=0x7f02ac071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:44.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.933+0000 7f02b15ff700 1 -- 172.21.15.111:0/3403410290 shutdown_connections 2023-03-29T10:58:44.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.933+0000 7f02b15ff700 1 -- 172.21.15.111:0/3403410290 wait complete. 2023-03-29T10:58:44.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.934+0000 7f02b15ff700 1 Processor -- start 2023-03-29T10:58:44.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.934+0000 7f02b15ff700 1 -- start start 2023-03-29T10:58:44.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.935+0000 7f02b15ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02ac0b70f0 0x7f02ac0cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:44.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.935+0000 7f02b15ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ac0cf970 0x7f02ac159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:44.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.935+0000 7f02b15ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02ac0cfd40 0x7f02ac0d1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:44.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.935+0000 7f02b15ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac0cb390 con 0x7f02ac0b70f0 2023-03-29T10:58:44.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.935+0000 7f02b15ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac0ca0d0 con 0x7f02ac0cf970 2023-03-29T10:58:44.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.935+0000 7f02b15ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac0ca370 con 0x7f02ac0cfd40 2023-03-29T10:58:44.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.936+0000 7f02aaffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02ac0b70f0 0x7f02ac0cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:44.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.936+0000 7f02aaffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02ac0b70f0 0x7f02ac0cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59148/0 (socket says 172.21.15.111:59148) 2023-03-29T10:58:44.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.936+0000 7f02aaffd700 1 -- 172.21.15.111:0/341164211 learned_addr learned my addr 172.21.15.111:0/341164211 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:44.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.936+0000 7f02a3fff700 1 --2- 172.21.15.111:0/341164211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ac0cf970 0x7f02ac159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:44.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.936+0000 7f02ab7fe700 1 --2- 172.21.15.111:0/341164211 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02ac0cfd40 0x7f02ac0d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:44.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.936+0000 7f02aaffd700 1 -- 172.21.15.111:0/341164211 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02ac0cfd40 msgr2=0x7f02ac0d1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:44.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.936+0000 7f02aaffd700 1 --2- 172.21.15.111:0/341164211 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02ac0cfd40 0x7f02ac0d1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.936+0000 7f02aaffd700 1 -- 172.21.15.111:0/341164211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ac0cf970 msgr2=0x7f02ac159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:44.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.936+0000 7f02aaffd700 1 --2- 172.21.15.111:0/341164211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ac0cf970 0x7f02ac159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:44.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.936+0000 7f02aaffd700 1 -- 172.21.15.111:0/341164211 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029c0094a0 con 0x7f02ac0b70f0 2023-03-29T10:58:44.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.936+0000 7f02a3fff700 1 --2- 172.21.15.111:0/341164211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ac0cf970 0x7f02ac159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:44.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.936+0000 7f02aaffd700 1 --2- 172.21.15.111:0/341164211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02ac0b70f0 0x7f02ac0cae50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f029c014040 tx=0x7f029c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:44.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.936+0000 7f02a37fe700 1 -- 172.21.15.111:0/341164211 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c01a900 con 0x7f02ac0b70f0 2023-03-29T10:58:44.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.936+0000 7f02a37fe700 1 -- 172.21.15.111:0/341164211 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029c01aa60 con 0x7f02ac0b70f0 2023-03-29T10:58:44.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.936+0000 7f02a37fe700 1 -- 172.21.15.111:0/341164211 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c01add0 con 0x7f02ac0b70f0 2023-03-29T10:58:44.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.936+0000 7f02b15ff700 1 -- 172.21.15.111:0/341164211 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ac0b68f0 con 0x7f02ac0b70f0 2023-03-29T10:58:44.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.937+0000 7f02b15ff700 1 -- 172.21.15.111:0/341164211 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ac0cd930 con 0x7f02ac0b70f0 2023-03-29T10:58:44.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.938+0000 7f02a17fa700 1 -- 172.21.15.111:0/341164211 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ac0cf650 con 0x7f02ac0b70f0 2023-03-29T10:58:44.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.939+0000 7f02a37fe700 1 -- 172.21.15.111:0/341164211 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f029c022070 con 0x7f02ac0b70f0 2023-03-29T10:58:44.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.939+0000 7f02a37fe700 1 --2- 172.21.15.111:0/341164211 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f028c069de0 0x7f028c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:44.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.939+0000 7f02a37fe700 1 -- 172.21.15.111:0/341164211 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f029c0891f0 con 0x7f02ac0b70f0 2023-03-29T10:58:44.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.940+0000 7f02a3fff700 1 --2- 172.21.15.111:0/341164211 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f028c069de0 0x7f028c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:44.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.940+0000 7f02a3fff700 1 --2- 172.21.15.111:0/341164211 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f028c069de0 0x7f028c06c290 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0c9480 tx=0x7f0294007e50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:44.943 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:44.944+0000 7f02a37fe700 1 -- 172.21.15.111:0/341164211 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f029c054810 con 0x7f02ac0b70f0 2023-03-29T10:58:45.134 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.134+0000 7f02a17fa700 1 -- 172.21.15.111:0/341164211 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02ac074430 con 0x7f02ac0b70f0 2023-03-29T10:58:45.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.136+0000 7f02a37fe700 1 -- 172.21.15.111:0/341164211 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19892 (secure 0 0 0) 0x7f029c057e30 con 0x7f02ac0b70f0 2023-03-29T10:58:45.136 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:45.136 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":30,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:58:09.476334+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.26","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:58:45.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.138+0000 7f02a17fa700 1 -- 172.21.15.111:0/341164211 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f028c069de0 msgr2=0x7f028c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:45.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.138+0000 7f02a17fa700 1 --2- 172.21.15.111:0/341164211 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f028c069de0 0x7f028c06c290 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0c9480 tx=0x7f0294007e50 comp rx=0 tx=0).stop 2023-03-29T10:58:45.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.138+0000 7f02a17fa700 1 -- 172.21.15.111:0/341164211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02ac0b70f0 msgr2=0x7f02ac0cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:45.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.138+0000 7f02a17fa700 1 --2- 172.21.15.111:0/341164211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02ac0b70f0 0x7f02ac0cae50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f029c014040 tx=0x7f029c004750 comp rx=0 tx=0).stop 2023-03-29T10:58:45.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.138+0000 7f02a17fa700 1 -- 172.21.15.111:0/341164211 shutdown_connections 2023-03-29T10:58:45.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.138+0000 7f02a17fa700 1 --2- 172.21.15.111:0/341164211 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f028c069de0 0x7f028c06c290 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.138 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.138+0000 7f02a17fa700 1 --2- 172.21.15.111:0/341164211 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f02ac0b70f0 0x7f02ac0cae50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.138+0000 7f02a17fa700 1 --2- 172.21.15.111:0/341164211 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f02ac0cfd40 0x7f02ac0d1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.138+0000 7f02a17fa700 1 --2- 172.21.15.111:0/341164211 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02ac0cf970 0x7f02ac159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.138+0000 7f02a17fa700 1 -- 172.21.15.111:0/341164211 >> 172.21.15.111:0/341164211 conn(0x7f02ac071b10 msgr2=0x7f02ac15b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:45.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.139+0000 7f02a17fa700 1 -- 172.21.15.111:0/341164211 shutdown_connections 2023-03-29T10:58:45.139 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.139+0000 7f02a17fa700 1 -- 172.21.15.111:0/341164211 wait complete. 2023-03-29T10:58:45.156 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-29T10:57:16.183979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-29T10:57:21.239809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-29T10:57:26.192124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-29T10:57:26.796777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-29T10:57:29.874795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-29T10:58:45.157 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-29T10:58:45.342 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.343+0000 7f663b7ff700 1 Processor -- start 2023-03-29T10:58:45.343 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.343+0000 7f663b7ff700 1 -- start start 2023-03-29T10:58:45.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.344+0000 7f663b7ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f663415c590 0x7f6634160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:45.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.344+0000 7f663b7ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66340cd520 con 0x7f66340cff50 2023-03-29T10:58:45.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.344+0000 7f663b7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66340d2640 con 0x7f66340cf010 2023-03-29T10:58:45.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.344+0000 7f663b7ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66341610d0 con 0x7f663415c590 2023-03-29T10:58:45.344 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.344+0000 7f6639d9c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f66340cff50 0x7f66340d2ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54620/0 (socket says 172.21.15.111:54620) 2023-03-29T10:58:45.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.344+0000 7f6639d9c700 1 -- 172.21.15.111:0/1358094704 learned_addr learned my addr 172.21.15.111:0/1358094704 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:45.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.345+0000 7f663959b700 1 --2- 172.21.15.111:0/1358094704 >> v2:172.21.15.119:3300/0 conn(0x7f663415c590 0x7f6634160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:45.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.345+0000 7f662bfff700 1 -- 172.21.15.111:0/1358094704 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842890774 0 0) 0x7f66340cd520 con 0x7f66340cff50 2023-03-29T10:58:45.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.345+0000 7f662bfff700 1 -- 172.21.15.111:0/1358094704 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66200035b0 con 0x7f66340cff50 2023-03-29T10:58:45.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.345+0000 7f662bfff700 1 -- 172.21.15.111:0/1358094704 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411130331 0 0) 0x7f66340d2640 con 0x7f66340cf010 2023-03-29T10:58:45.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.345+0000 7f662bfff700 1 -- 172.21.15.111:0/1358094704 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66340cd520 con 0x7f66340cf010 2023-03-29T10:58:45.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.345+0000 7f663959b700 1 -- 172.21.15.111:0/1358094704 >> v1:172.21.15.119:6789/0 conn(0x7f66340cf010 legacy=0x7f66340cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:45.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.345+0000 7f663959b700 1 -- 172.21.15.111:0/1358094704 >> v1:172.21.15.111:6789/0 conn(0x7f66340cff50 legacy=0x7f66340d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:45.345 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.345+0000 7f663959b700 1 -- 172.21.15.111:0/1358094704 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66340b23c0 con 0x7f663415c590 2023-03-29T10:58:45.346 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.345+0000 7f663959b700 1 --2- 172.21.15.111:0/1358094704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f663415c590 0x7f6634160b90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6630009840 tx=0x7f6630009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=470516d422296b6 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:45.346 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.346+0000 7f662bfff700 1 -- 172.21.15.111:0/1358094704 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6630011830 con 0x7f663415c590 2023-03-29T10:58:45.346 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.346+0000 7f662bfff700 1 -- 172.21.15.111:0/1358094704 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6630011990 con 0x7f663415c590 2023-03-29T10:58:45.346 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.346+0000 7f662bfff700 1 -- 172.21.15.111:0/1358094704 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6630011d20 con 0x7f663415c590 2023-03-29T10:58:45.346 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.346+0000 7f663b7ff700 1 -- 172.21.15.111:0/1358094704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f663415c590 msgr2=0x7f6634160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:45.346 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.346+0000 7f663b7ff700 1 --2- 172.21.15.111:0/1358094704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f663415c590 0x7f6634160b90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6630009840 tx=0x7f6630009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:45.346 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.346+0000 7f663b7ff700 1 -- 172.21.15.111:0/1358094704 shutdown_connections 2023-03-29T10:58:45.346 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.346+0000 7f663b7ff700 1 --2- 172.21.15.111:0/1358094704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f663415c590 0x7f6634160b90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.346+0000 7f663b7ff700 1 -- 172.21.15.111:0/1358094704 >> 172.21.15.111:0/1358094704 conn(0x7f6634071b10 msgr2=0x7f6634071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:45.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.346+0000 7f663b7ff700 1 -- 172.21.15.111:0/1358094704 shutdown_connections 2023-03-29T10:58:45.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.346+0000 7f663b7ff700 1 -- 172.21.15.111:0/1358094704 wait complete. 2023-03-29T10:58:45.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.348+0000 7f663b7ff700 1 Processor -- start 2023-03-29T10:58:45.347 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.348+0000 7f663b7ff700 1 -- start start 2023-03-29T10:58:45.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.349+0000 7f663b7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66340cf010 0x7f6634155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:45.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.349+0000 7f663b7ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f66340cff50 0x7f6634154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:45.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.349+0000 7f663b7ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f663415c590 0x7f66341e5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:45.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.349+0000 7f663b7ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66341583e0 con 0x7f663415c590 2023-03-29T10:58:45.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.349+0000 7f663b7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66340b5e80 con 0x7f66340cf010 2023-03-29T10:58:45.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.349+0000 7f663b7ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66340b6160 con 0x7f66340cff50 2023-03-29T10:58:45.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.349+0000 7f6639d9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f663415c590 0x7f66341e5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:45.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.349+0000 7f6639d9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f663415c590 0x7f66341e5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59152/0 (socket says 172.21.15.111:59152) 2023-03-29T10:58:45.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.349+0000 7f6639d9c700 1 -- 172.21.15.111:0/1785063705 learned_addr learned my addr 172.21.15.111:0/1785063705 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:45.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.349+0000 7f663959b700 1 --2- 172.21.15.111:0/1785063705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66340cf010 0x7f6634155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:45.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.349+0000 7f6638d9a700 1 --2- 172.21.15.111:0/1785063705 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f66340cff50 0x7f6634154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:45.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.350+0000 7f6639d9c700 1 -- 172.21.15.111:0/1785063705 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f66340cff50 msgr2=0x7f6634154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:45.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.350+0000 7f6639d9c700 1 --2- 172.21.15.111:0/1785063705 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f66340cff50 0x7f6634154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.350+0000 7f6639d9c700 1 -- 172.21.15.111:0/1785063705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66340cf010 msgr2=0x7f6634155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:45.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.350+0000 7f6639d9c700 1 --2- 172.21.15.111:0/1785063705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66340cf010 0x7f6634155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.350+0000 7f6639d9c700 1 -- 172.21.15.111:0/1785063705 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66300094a0 con 0x7f663415c590 2023-03-29T10:58:45.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.350+0000 7f6639d9c700 1 --2- 172.21.15.111:0/1785063705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f663415c590 0x7f66341e5f80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f662c00cba0 tx=0x7f662c00ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:45.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.350+0000 7f6629ffb700 1 -- 172.21.15.111:0/1785063705 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c00aa40 con 0x7f663415c590 2023-03-29T10:58:45.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.350+0000 7f6629ffb700 1 -- 172.21.15.111:0/1785063705 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662c00aba0 con 0x7f663415c590 2023-03-29T10:58:45.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.350+0000 7f663b7ff700 1 -- 172.21.15.111:0/1785063705 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66340cd770 con 0x7f663415c590 2023-03-29T10:58:45.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.350+0000 7f6629ffb700 1 -- 172.21.15.111:0/1785063705 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662c012400 con 0x7f663415c590 2023-03-29T10:58:45.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.350+0000 7f663b7ff700 1 -- 172.21.15.111:0/1785063705 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66340cf560 con 0x7f663415c590 2023-03-29T10:58:45.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.351+0000 7f663b7ff700 1 -- 172.21.15.111:0/1785063705 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663415c960 con 0x7f663415c590 2023-03-29T10:58:45.351 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.352+0000 7f6629ffb700 1 -- 172.21.15.111:0/1785063705 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f662c01b3b0 con 0x7f663415c590 2023-03-29T10:58:45.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.353+0000 7f6629ffb700 1 --2- 172.21.15.111:0/1785063705 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6618069de0 0x7f661806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:45.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.353+0000 7f6629ffb700 1 -- 172.21.15.111:0/1785063705 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f662c087f30 con 0x7f663415c590 2023-03-29T10:58:45.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.353+0000 7f663959b700 1 --2- 172.21.15.111:0/1785063705 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6618069de0 0x7f661806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:45.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.354+0000 7f663959b700 1 --2- 172.21.15.111:0/1785063705 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6618069de0 0x7f661806c290 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6630009fd0 tx=0x7f663001c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:45.356 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.357+0000 7f6629ffb700 1 -- 172.21.15.111:0/1785063705 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f662c0534e0 con 0x7f663415c590 2023-03-29T10:58:45.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.546+0000 7f663b7ff700 1 -- 172.21.15.111:0/1785063705 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f66340b51f0 con 0x7f663415c590 2023-03-29T10:58:45.546 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.547+0000 7f6629ffb700 1 -- 172.21.15.111:0/1785063705 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f662c0170a0 con 0x7f663415c590 2023-03-29T10:58:45.547 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-03-29T10:58:45.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.549+0000 7f663b7ff700 1 -- 172.21.15.111:0/1785063705 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6618069de0 msgr2=0x7f661806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:45.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.549+0000 7f663b7ff700 1 --2- 172.21.15.111:0/1785063705 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6618069de0 0x7f661806c290 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6630009fd0 tx=0x7f663001c040 comp rx=0 tx=0).stop 2023-03-29T10:58:45.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.549+0000 7f663b7ff700 1 -- 172.21.15.111:0/1785063705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f663415c590 msgr2=0x7f66341e5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:45.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.549+0000 7f663b7ff700 1 --2- 172.21.15.111:0/1785063705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f663415c590 0x7f66341e5f80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f662c00cba0 tx=0x7f662c00ceb0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.549+0000 7f663b7ff700 1 -- 172.21.15.111:0/1785063705 shutdown_connections 2023-03-29T10:58:45.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.549+0000 7f663b7ff700 1 --2- 172.21.15.111:0/1785063705 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6618069de0 0x7f661806c290 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.549+0000 7f663b7ff700 1 --2- 172.21.15.111:0/1785063705 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f663415c590 0x7f66341e5f80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.549+0000 7f663b7ff700 1 --2- 172.21.15.111:0/1785063705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66340cf010 0x7f6634155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.549 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.549+0000 7f663b7ff700 1 --2- 172.21.15.111:0/1785063705 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f66340cff50 0x7f6634154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.549+0000 7f663b7ff700 1 -- 172.21.15.111:0/1785063705 >> 172.21.15.111:0/1785063705 conn(0x7f6634071b10 msgr2=0x7f66340d2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:45.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.549+0000 7f663b7ff700 1 -- 172.21.15.111:0/1785063705 shutdown_connections 2023-03-29T10:58:45.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.550+0000 7f663b7ff700 1 -- 172.21.15.111:0/1785063705 wait complete. 2023-03-29T10:58:45.566 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-29T10:58:45.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.752+0000 7f60bf7ff700 1 Processor -- start 2023-03-29T10:58:45.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.753+0000 7f60bf7ff700 1 -- start start 2023-03-29T10:58:45.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.754+0000 7f60bf7ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f60b8158da0 0x7f60b806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:45.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.754+0000 7f60bf7ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f60b806cc00 0x7f60b806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:45.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.754+0000 7f60bf7ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60b80ca890 con 0x7f60b81589d0 2023-03-29T10:58:45.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.754+0000 7f60bf7ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60b80cd860 con 0x7f60b8158da0 2023-03-29T10:58:45.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.754+0000 7f60bf7ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60b80ca5b0 con 0x7f60b806cc00 2023-03-29T10:58:45.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.754+0000 7f60bd59b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f60b81589d0 0x7f60b80cd150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54628/0 (socket says 172.21.15.111:54628) 2023-03-29T10:58:45.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.754+0000 7f60bd59b700 1 -- 172.21.15.111:0/2964379816 learned_addr learned my addr 172.21.15.111:0/2964379816 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:45.754 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.754+0000 7f60bcd9a700 1 --2- 172.21.15.111:0/2964379816 >> v2:172.21.15.111:3300/0 conn(0x7f60b8158da0 0x7f60b806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:45.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.754+0000 7f60bdd9c700 1 --2- 172.21.15.111:0/2964379816 >> v2:172.21.15.119:3300/0 conn(0x7f60b806cc00 0x7f60b806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:45.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.754+0000 7f60affff700 1 -- 172.21.15.111:0/2964379816 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110018023 0 0) 0x7f60b80ca890 con 0x7f60b81589d0 2023-03-29T10:58:45.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.754+0000 7f60affff700 1 -- 172.21.15.111:0/2964379816 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60a40035b0 con 0x7f60b81589d0 2023-03-29T10:58:45.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.754+0000 7f60bcd9a700 1 -- 172.21.15.111:0/2964379816 >> v2:172.21.15.119:3300/0 conn(0x7f60b806cc00 msgr2=0x7f60b806a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:45.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.754+0000 7f60bcd9a700 1 --2- 172.21.15.111:0/2964379816 >> v2:172.21.15.119:3300/0 conn(0x7f60b806cc00 0x7f60b806a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.754+0000 7f60bcd9a700 1 -- 172.21.15.111:0/2964379816 >> v1:172.21.15.111:6789/0 conn(0x7f60b81589d0 legacy=0x7f60b80cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:45.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.754+0000 7f60bcd9a700 1 -- 172.21.15.111:0/2964379816 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b80cc670 con 0x7f60b8158da0 2023-03-29T10:58:45.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.755+0000 7f60bcd9a700 1 --2- 172.21.15.111:0/2964379816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60b8158da0 0x7f60b806c6c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f60b806b370 tx=0x7f60a8009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54567910a0f69f9 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:45.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.755+0000 7f60affff700 1 -- 172.21.15.111:0/2964379816 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8005e30 con 0x7f60b8158da0 2023-03-29T10:58:45.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.755+0000 7f60affff700 1 -- 172.21.15.111:0/2964379816 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a8006030 con 0x7f60b8158da0 2023-03-29T10:58:45.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.755+0000 7f60affff700 1 -- 172.21.15.111:0/2964379816 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a80063c0 con 0x7f60b8158da0 2023-03-29T10:58:45.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.755+0000 7f60bf7ff700 1 -- 172.21.15.111:0/2964379816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60b8158da0 msgr2=0x7f60b806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:45.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.755+0000 7f60bf7ff700 1 --2- 172.21.15.111:0/2964379816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60b8158da0 0x7f60b806c6c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f60b806b370 tx=0x7f60a8009b00 comp rx=0 tx=0).stop 2023-03-29T10:58:45.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.756+0000 7f60bf7ff700 1 -- 172.21.15.111:0/2964379816 shutdown_connections 2023-03-29T10:58:45.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.756+0000 7f60bf7ff700 1 --2- 172.21.15.111:0/2964379816 >> v2:172.21.15.119:3300/0 conn(0x7f60b806cc00 0x7f60b806a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.756+0000 7f60bf7ff700 1 --2- 172.21.15.111:0/2964379816 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60b8158da0 0x7f60b806c6c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.756+0000 7f60bf7ff700 1 -- 172.21.15.111:0/2964379816 >> 172.21.15.111:0/2964379816 conn(0x7f60b80cf720 msgr2=0x7f60b80cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:45.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.756+0000 7f60bf7ff700 1 -- 172.21.15.111:0/2964379816 shutdown_connections 2023-03-29T10:58:45.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.756+0000 7f60bf7ff700 1 -- 172.21.15.111:0/2964379816 wait complete. 2023-03-29T10:58:45.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.757+0000 7f60bf7ff700 1 Processor -- start 2023-03-29T10:58:45.757 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.757+0000 7f60bf7ff700 1 -- start start 2023-03-29T10:58:45.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.758+0000 7f60bf7ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60b806cc00 0x7f60b8150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:45.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.758+0000 7f60bf7ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60b81589d0 0x7f60b80afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.758+0000 7f60bf7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60b8158da0 0x7f60b8153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.758+0000 7f60bf7ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b80c2fd0 con 0x7f60b81589d0 2023-03-29T10:58:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.758+0000 7f60bf7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b80c53a0 con 0x7f60b8158da0 2023-03-29T10:58:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.758+0000 7f60bf7ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b8150580 con 0x7f60b806cc00 2023-03-29T10:58:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.758+0000 7f60bcd9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60b81589d0 0x7f60b80afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.759+0000 7f60bcd9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60b81589d0 0x7f60b80afd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59182/0 (socket says 172.21.15.111:59182) 2023-03-29T10:58:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.759+0000 7f60bcd9a700 1 -- 172.21.15.111:0/1953929771 learned_addr learned my addr 172.21.15.111:0/1953929771 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.759+0000 7f60bd59b700 1 --2- 172.21.15.111:0/1953929771 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60b806cc00 0x7f60b8150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.759+0000 7f60bdd9c700 1 --2- 172.21.15.111:0/1953929771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60b8158da0 0x7f60b8153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.759+0000 7f60bcd9a700 1 -- 172.21.15.111:0/1953929771 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60b806cc00 msgr2=0x7f60b8150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.759+0000 7f60bcd9a700 1 --2- 172.21.15.111:0/1953929771 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60b806cc00 0x7f60b8150040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.759+0000 7f60bcd9a700 1 -- 172.21.15.111:0/1953929771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60b8158da0 msgr2=0x7f60b8153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.759+0000 7f60bcd9a700 1 --2- 172.21.15.111:0/1953929771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60b8158da0 0x7f60b8153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.759+0000 7f60bcd9a700 1 -- 172.21.15.111:0/1953929771 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a8009430 con 0x7f60b81589d0 2023-03-29T10:58:45.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.759+0000 7f60bcd9a700 1 --2- 172.21.15.111:0/1953929771 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60b81589d0 0x7f60b80afd80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f60b80708b0 tx=0x7f60a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:45.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.759+0000 7f60adffb700 1 -- 172.21.15.111:0/1953929771 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a801e070 con 0x7f60b81589d0 2023-03-29T10:58:45.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.759+0000 7f60adffb700 1 -- 172.21.15.111:0/1953929771 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a8019660 con 0x7f60b81589d0 2023-03-29T10:58:45.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.759+0000 7f60bf7ff700 1 -- 172.21.15.111:0/1953929771 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b80ccc30 con 0x7f60b81589d0 2023-03-29T10:58:45.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.759+0000 7f60adffb700 1 -- 172.21.15.111:0/1953929771 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8019a10 con 0x7f60b81589d0 2023-03-29T10:58:45.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.759+0000 7f60bf7ff700 1 -- 172.21.15.111:0/1953929771 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b80cc950 con 0x7f60b81589d0 2023-03-29T10:58:45.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.760+0000 7f60bf7ff700 1 -- 172.21.15.111:0/1953929771 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b806a920 con 0x7f60b81589d0 2023-03-29T10:58:45.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.761+0000 7f60adffb700 1 -- 172.21.15.111:0/1953929771 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f60a8019b70 con 0x7f60b81589d0 2023-03-29T10:58:45.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.762+0000 7f60adffb700 1 --2- 172.21.15.111:0/1953929771 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f609c069de0 0x7f609c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:45.761 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.762+0000 7f60adffb700 1 -- 172.21.15.111:0/1953929771 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f60a8089520 con 0x7f60b81589d0 2023-03-29T10:58:45.762 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.762+0000 7f60bd59b700 1 --2- 172.21.15.111:0/1953929771 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f609c069de0 0x7f609c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:45.762 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.763+0000 7f60bd59b700 1 --2- 172.21.15.111:0/1953929771 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f609c069de0 0x7f609c06c290 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f60b40011e0 tx=0x7f60b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:45.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.766+0000 7f60adffb700 1 -- 172.21.15.111:0/1953929771 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f60a8054ad0 con 0x7f60b81589d0 2023-03-29T10:58:45.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.959+0000 7f60bf7ff700 1 -- 172.21.15.111:0/1953929771 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f60b806acf0 con 0x7f60b81589d0 2023-03-29T10:58:45.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.959+0000 7f60adffb700 1 -- 172.21.15.111:0/1953929771 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f60a80580f0 con 0x7f60b81589d0 2023-03-29T10:58:45.959 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 8 2023-03-29T10:58:45.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.961+0000 7f60bf7ff700 1 -- 172.21.15.111:0/1953929771 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f609c069de0 msgr2=0x7f609c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:45.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.961+0000 7f60bf7ff700 1 --2- 172.21.15.111:0/1953929771 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f609c069de0 0x7f609c06c290 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f60b40011e0 tx=0x7f60b4009040 comp rx=0 tx=0).stop 2023-03-29T10:58:45.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.961+0000 7f60bf7ff700 1 -- 172.21.15.111:0/1953929771 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60b81589d0 msgr2=0x7f60b80afd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:45.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.961+0000 7f60bf7ff700 1 --2- 172.21.15.111:0/1953929771 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60b81589d0 0x7f60b80afd80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f60b80708b0 tx=0x7f60a8004750 comp rx=0 tx=0).stop 2023-03-29T10:58:45.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.962+0000 7f60bf7ff700 1 -- 172.21.15.111:0/1953929771 shutdown_connections 2023-03-29T10:58:45.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.962+0000 7f60bf7ff700 1 --2- 172.21.15.111:0/1953929771 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f609c069de0 0x7f609c06c290 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.962+0000 7f60bf7ff700 1 --2- 172.21.15.111:0/1953929771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60b8158da0 0x7f60b8153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.962+0000 7f60bf7ff700 1 --2- 172.21.15.111:0/1953929771 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60b806cc00 0x7f60b8150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.962+0000 7f60bf7ff700 1 --2- 172.21.15.111:0/1953929771 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60b81589d0 0x7f60b80afd80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:45.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.962+0000 7f60bf7ff700 1 -- 172.21.15.111:0/1953929771 >> 172.21.15.111:0/1953929771 conn(0x7f60b80cf720 msgr2=0x7f60b8154cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:45.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.962+0000 7f60bf7ff700 1 -- 172.21.15.111:0/1953929771 shutdown_connections 2023-03-29T10:58:45.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:45.962+0000 7f60bf7ff700 1 -- 172.21.15.111:0/1953929771 wait complete. 2023-03-29T10:58:45.978 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-29T10:58:46.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.165+0000 7f6091fff700 1 Processor -- start 2023-03-29T10:58:46.165 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.166+0000 7f6091fff700 1 -- start start 2023-03-29T10:58:46.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.167+0000 7f6091fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f608c0cd760 0x7f608c0cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:46.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.167+0000 7f6091fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f608c0adf70 0x7f608c0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:46.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.167+0000 7f6091fff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608c0cc9e0 con 0x7f608c15cf10 2023-03-29T10:58:46.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.167+0000 7f6091fff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f608c15de20 con 0x7f608c0adf70 2023-03-29T10:58:46.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.167+0000 7f6091fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0ce070 con 0x7f608c0cd760 2023-03-29T10:58:46.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.167+0000 7f608bfff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f608c15cf10 0x7f608c06c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54642/0 (socket says 172.21.15.111:54642) 2023-03-29T10:58:46.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.167+0000 7f608bfff700 1 -- 172.21.15.111:0/2741285447 learned_addr learned my addr 172.21.15.111:0/2741285447 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:46.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.167+0000 7f608affd700 1 --2- 172.21.15.111:0/2741285447 >> v2:172.21.15.111:3300/0 conn(0x7f608c0adf70 0x7f608c0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:46.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.167+0000 7f608b7fe700 1 --2- 172.21.15.111:0/2741285447 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f608c0cd760 0x7f608c0cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:46.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.167+0000 7f608affd700 1 -- 172.21.15.111:0/2741285447 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f608c0cd760 msgr2=0x7f608c0cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:46.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.167+0000 7f608affd700 1 --2- 172.21.15.111:0/2741285447 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f608c0cd760 0x7f608c0cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.167+0000 7f608affd700 1 -- 172.21.15.111:0/2741285447 >> v1:172.21.15.111:6789/0 conn(0x7f608c15cf10 legacy=0x7f608c06c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:46.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.167+0000 7f608affd700 1 -- 172.21.15.111:0/2741285447 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c15d860 con 0x7f608c0adf70 2023-03-29T10:58:46.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.167+0000 7f608a7fc700 1 -- 172.21.15.111:0/2741285447 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2310506603 0 0) 0x7f608c0cc9e0 con 0x7f608c15cf10 2023-03-29T10:58:46.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.168+0000 7f608affd700 1 --2- 172.21.15.111:0/2741285447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f608c0adf70 0x7f608c0cce70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6080009820 tx=0x7f6080009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0d3623ca898290 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:46.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.168+0000 7f608a7fc700 1 -- 172.21.15.111:0/2741285447 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6080005e30 con 0x7f608c0adf70 2023-03-29T10:58:46.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.168+0000 7f608a7fc700 1 -- 172.21.15.111:0/2741285447 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6080006030 con 0x7f608c0adf70 2023-03-29T10:58:46.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.168+0000 7f608a7fc700 1 -- 172.21.15.111:0/2741285447 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60800063c0 con 0x7f608c0adf70 2023-03-29T10:58:46.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.168+0000 7f6091fff700 1 -- 172.21.15.111:0/2741285447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f608c0adf70 msgr2=0x7f608c0cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:46.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.168+0000 7f6091fff700 1 --2- 172.21.15.111:0/2741285447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f608c0adf70 0x7f608c0cce70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6080009820 tx=0x7f6080009b30 comp rx=0 tx=0).stop 2023-03-29T10:58:46.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.168+0000 7f6091fff700 1 -- 172.21.15.111:0/2741285447 shutdown_connections 2023-03-29T10:58:46.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.169+0000 7f6091fff700 1 --2- 172.21.15.111:0/2741285447 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f608c0cd760 0x7f608c0cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.169+0000 7f6091fff700 1 --2- 172.21.15.111:0/2741285447 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f608c0adf70 0x7f608c0cce70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.169+0000 7f6091fff700 1 -- 172.21.15.111:0/2741285447 >> 172.21.15.111:0/2741285447 conn(0x7f608c0cf720 msgr2=0x7f608c0cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:46.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.169+0000 7f6091fff700 1 -- 172.21.15.111:0/2741285447 shutdown_connections 2023-03-29T10:58:46.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.169+0000 7f6091fff700 1 -- 172.21.15.111:0/2741285447 wait complete. 2023-03-29T10:58:46.169 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.170+0000 7f6091fff700 1 Processor -- start 2023-03-29T10:58:46.170 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.170+0000 7f6091fff700 1 -- start start 2023-03-29T10:58:46.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.171+0000 7f6091fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f608c0adf70 0x7f608c14f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:46.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.171+0000 7f6091fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f608c0cd760 0x7f608c14faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:46.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.171+0000 7f6091fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f608c15cf10 0x7f608c0afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:46.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.171+0000 7f6091fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0c54e0 con 0x7f608c15cf10 2023-03-29T10:58:46.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.171+0000 7f6091fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0ae580 con 0x7f608c0cd760 2023-03-29T10:58:46.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.171+0000 7f6091fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608c0c4f60 con 0x7f608c0adf70 2023-03-29T10:58:46.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.172+0000 7f608bfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f608c15cf10 0x7f608c0afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:46.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.172+0000 7f608bfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f608c15cf10 0x7f608c0afaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59198/0 (socket says 172.21.15.111:59198) 2023-03-29T10:58:46.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.172+0000 7f608bfff700 1 -- 172.21.15.111:0/160869824 learned_addr learned my addr 172.21.15.111:0/160869824 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:46.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.172+0000 7f608affd700 1 --2- 172.21.15.111:0/160869824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f608c0cd760 0x7f608c14faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:46.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.172+0000 7f608b7fe700 1 --2- 172.21.15.111:0/160869824 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f608c0adf70 0x7f608c14f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:46.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.172+0000 7f608bfff700 1 -- 172.21.15.111:0/160869824 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f608c0adf70 msgr2=0x7f608c14f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:46.172 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.172+0000 7f608bfff700 1 --2- 172.21.15.111:0/160869824 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f608c0adf70 0x7f608c14f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.172+0000 7f608bfff700 1 -- 172.21.15.111:0/160869824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f608c0cd760 msgr2=0x7f608c14faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:46.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.172+0000 7f608bfff700 1 --2- 172.21.15.111:0/160869824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f608c0cd760 0x7f608c14faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.172+0000 7f608bfff700 1 -- 172.21.15.111:0/160869824 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6080009430 con 0x7f608c15cf10 2023-03-29T10:58:46.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.172+0000 7f608bfff700 1 --2- 172.21.15.111:0/160869824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f608c15cf10 0x7f608c0afaf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f608400cb60 tx=0x7f608400ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:46.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.172+0000 7f6073fff700 1 -- 172.21.15.111:0/160869824 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608400a9e0 con 0x7f608c15cf10 2023-03-29T10:58:46.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.172+0000 7f6073fff700 1 -- 172.21.15.111:0/160869824 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608400ab40 con 0x7f608c15cf10 2023-03-29T10:58:46.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.172+0000 7f6091fff700 1 -- 172.21.15.111:0/160869824 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608c0cc670 con 0x7f608c15cf10 2023-03-29T10:58:46.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.172+0000 7f6073fff700 1 -- 172.21.15.111:0/160869824 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60840070d0 con 0x7f608c15cf10 2023-03-29T10:58:46.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.173+0000 7f6091fff700 1 -- 172.21.15.111:0/160869824 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608c15db40 con 0x7f608c15cf10 2023-03-29T10:58:46.173 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.174+0000 7f6091fff700 1 -- 172.21.15.111:0/160869824 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608c071600 con 0x7f608c15cf10 2023-03-29T10:58:46.174 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.174+0000 7f6073fff700 1 -- 172.21.15.111:0/160869824 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f6084007230 con 0x7f608c15cf10 2023-03-29T10:58:46.175 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.176+0000 7f6073fff700 1 --2- 172.21.15.111:0/160869824 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6078069de0 0x7f607806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:46.175 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.176+0000 7f6073fff700 1 -- 172.21.15.111:0/160869824 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6084087fa0 con 0x7f608c15cf10 2023-03-29T10:58:46.175 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.176+0000 7f608b7fe700 1 --2- 172.21.15.111:0/160869824 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6078069de0 0x7f607806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:46.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.177+0000 7f608b7fe700 1 --2- 172.21.15.111:0/160869824 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6078069de0 0x7f607806c290 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f608c154510 tx=0x7f607c009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:46.179 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.180+0000 7f6073fff700 1 -- 172.21.15.111:0/160869824 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f6084053520 con 0x7f608c15cf10 2023-03-29T10:58:46.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.374+0000 7f6091fff700 1 -- 172.21.15.111:0/160869824 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f608c071b50 con 0x7f608c15cf10 2023-03-29T10:58:46.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.375+0000 7f6073fff700 1 -- 172.21.15.111:0/160869824 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f60840180a0 con 0x7f608c15cf10 2023-03-29T10:58:46.374 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:46.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.377+0000 7f6091fff700 1 -- 172.21.15.111:0/160869824 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6078069de0 msgr2=0x7f607806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:46.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.377+0000 7f6091fff700 1 --2- 172.21.15.111:0/160869824 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6078069de0 0x7f607806c290 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f608c154510 tx=0x7f607c009210 comp rx=0 tx=0).stop 2023-03-29T10:58:46.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.377+0000 7f6091fff700 1 -- 172.21.15.111:0/160869824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f608c15cf10 msgr2=0x7f608c0afaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:46.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.377+0000 7f6091fff700 1 --2- 172.21.15.111:0/160869824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f608c15cf10 0x7f608c0afaf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f608400cb60 tx=0x7f608400ce70 comp rx=0 tx=0).stop 2023-03-29T10:58:46.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.377+0000 7f6091fff700 1 -- 172.21.15.111:0/160869824 shutdown_connections 2023-03-29T10:58:46.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.377+0000 7f6091fff700 1 --2- 172.21.15.111:0/160869824 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6078069de0 0x7f607806c290 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.377+0000 7f6091fff700 1 --2- 172.21.15.111:0/160869824 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f608c15cf10 0x7f608c0afaf0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.377+0000 7f6091fff700 1 --2- 172.21.15.111:0/160869824 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f608c0adf70 0x7f608c14f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.377+0000 7f6091fff700 1 --2- 172.21.15.111:0/160869824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f608c0cd760 0x7f608c14faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.377+0000 7f6091fff700 1 -- 172.21.15.111:0/160869824 >> 172.21.15.111:0/160869824 conn(0x7f608c0cf720 msgr2=0x7f608c15aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:46.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.377+0000 7f6091fff700 1 -- 172.21.15.111:0/160869824 shutdown_connections 2023-03-29T10:58:46.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.377+0000 7f6091fff700 1 -- 172.21.15.111:0/160869824 wait complete. 2023-03-29T10:58:46.394 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-29T10:58:46.580 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.581+0000 7fe4c53ff700 1 Processor -- start 2023-03-29T10:58:46.581 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.581+0000 7fe4c53ff700 1 -- start start 2023-03-29T10:58:46.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.582+0000 7fe4c53ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe4c015a5a0 0x7fe4c006a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:46.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.582+0000 7fe4c53ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c006a920 0x7fe4c0070af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:46.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.582+0000 7fe4c53ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe4c0071030 0x7fe4c015dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:46.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.582+0000 7fe4c53ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c015a970 con 0x7fe4c0071030 2023-03-29T10:58:46.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.582+0000 7fe4c53ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c006ad90 con 0x7fe4c015a5a0 2023-03-29T10:58:46.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.582+0000 7fe4c53ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c00cb280 con 0x7fe4c006a920 2023-03-29T10:58:46.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.582+0000 7fe4bf7fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe4c0071030 0x7fe4c015dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:46.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.582+0000 7fe4bf7fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe4c0071030 0x7fe4c015dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59200/0 (socket says 172.21.15.111:59200) 2023-03-29T10:58:46.582 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.582+0000 7fe4bf7fe700 1 -- 172.21.15.111:0/998899016 learned_addr learned my addr 172.21.15.111:0/998899016 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:46.583 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.583+0000 7fe4beffd700 1 --2- 172.21.15.111:0/998899016 >> v2:172.21.15.119:3300/0 conn(0x7fe4c015a5a0 0x7fe4c006a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:46.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.583+0000 7fe4be7fc700 1 --2- 172.21.15.111:0/998899016 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c006a920 0x7fe4c0070af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:46.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.583+0000 7fe4bf7fe700 1 -- 172.21.15.111:0/998899016 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c006a920 msgr2=0x7fe4c0070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:46.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.583+0000 7fe4bf7fe700 1 --2- 172.21.15.111:0/998899016 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c006a920 0x7fe4c0070af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.583+0000 7fe4bf7fe700 1 -- 172.21.15.111:0/998899016 >> v2:172.21.15.119:3300/0 conn(0x7fe4c015a5a0 msgr2=0x7fe4c006a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:46.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.583+0000 7fe4bf7fe700 1 --2- 172.21.15.111:0/998899016 >> v2:172.21.15.119:3300/0 conn(0x7fe4c015a5a0 0x7fe4c006a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.583+0000 7fe4bf7fe700 1 -- 172.21.15.111:0/998899016 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c00714b0 con 0x7fe4c0071030 2023-03-29T10:58:46.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.583+0000 7fe4be7fc700 1 --2- 172.21.15.111:0/998899016 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c006a920 0x7fe4c0070af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:46.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.583+0000 7fe4bf7fe700 1 --2- 172.21.15.111:0/998899016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c0071030 0x7fe4c015dff0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0067990 tx=0x7fe4b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc87a56be91647e server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:46.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.583+0000 7fe4bdffb700 1 -- 172.21.15.111:0/998899016 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b800e070 con 0x7fe4c0071030 2023-03-29T10:58:46.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.583+0000 7fe4bdffb700 1 -- 172.21.15.111:0/998899016 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b800b980 con 0x7fe4c0071030 2023-03-29T10:58:46.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.583+0000 7fe4bdffb700 1 -- 172.21.15.111:0/998899016 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b800bd30 con 0x7fe4c0071030 2023-03-29T10:58:46.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.584+0000 7fe4c53ff700 1 -- 172.21.15.111:0/998899016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c0071030 msgr2=0x7fe4c015dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:46.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.584+0000 7fe4c53ff700 1 --2- 172.21.15.111:0/998899016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c0071030 0x7fe4c015dff0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0067990 tx=0x7fe4b800b330 comp rx=0 tx=0).stop 2023-03-29T10:58:46.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.584+0000 7fe4c53ff700 1 -- 172.21.15.111:0/998899016 shutdown_connections 2023-03-29T10:58:46.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.584+0000 7fe4c53ff700 1 --2- 172.21.15.111:0/998899016 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c0071030 0x7fe4c015dff0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.584+0000 7fe4c53ff700 1 --2- 172.21.15.111:0/998899016 >> v2:172.21.15.119:3300/0 conn(0x7fe4c015a5a0 0x7fe4c006a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.584 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.584+0000 7fe4c53ff700 1 --2- 172.21.15.111:0/998899016 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c006a920 0x7fe4c0070af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.584+0000 7fe4c53ff700 1 -- 172.21.15.111:0/998899016 >> 172.21.15.111:0/998899016 conn(0x7fe4c00cf720 msgr2=0x7fe4c00cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:46.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.584+0000 7fe4c53ff700 1 -- 172.21.15.111:0/998899016 shutdown_connections 2023-03-29T10:58:46.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.584+0000 7fe4c53ff700 1 -- 172.21.15.111:0/998899016 wait complete. 2023-03-29T10:58:46.585 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.586+0000 7fe4c53ff700 1 Processor -- start 2023-03-29T10:58:46.586 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.586+0000 7fe4c53ff700 1 -- start start 2023-03-29T10:58:46.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.588+0000 7fe4c53ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c006a920 0x7fe4c00cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:46.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.588+0000 7fe4c53ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c0071030 0x7fe4c015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:46.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.588+0000 7fe4c53ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c015a5a0 0x7fe4c0155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:46.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.588+0000 7fe4c53ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c015ccc0 con 0x7fe4c0071030 2023-03-29T10:58:46.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.588+0000 7fe4c53ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c00ce130 con 0x7fe4c015a5a0 2023-03-29T10:58:46.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.588+0000 7fe4c53ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c015d0e0 con 0x7fe4c006a920 2023-03-29T10:58:46.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.588+0000 7fe4be7fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c0071030 0x7fe4c015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:46.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.588+0000 7fe4be7fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c0071030 0x7fe4c015b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59210/0 (socket says 172.21.15.111:59210) 2023-03-29T10:58:46.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.588+0000 7fe4be7fc700 1 -- 172.21.15.111:0/1609137207 learned_addr learned my addr 172.21.15.111:0/1609137207 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:46.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.588+0000 7fe4beffd700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c006a920 0x7fe4c00cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:46.588 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.588+0000 7fe4bf7fe700 1 --2- 172.21.15.111:0/1609137207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c015a5a0 0x7fe4c0155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:46.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.588+0000 7fe4be7fc700 1 -- 172.21.15.111:0/1609137207 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c006a920 msgr2=0x7fe4c00cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:46.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.588+0000 7fe4be7fc700 1 --2- 172.21.15.111:0/1609137207 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c006a920 0x7fe4c00cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.588+0000 7fe4be7fc700 1 -- 172.21.15.111:0/1609137207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c015a5a0 msgr2=0x7fe4c0155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:46.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.588+0000 7fe4be7fc700 1 --2- 172.21.15.111:0/1609137207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c015a5a0 0x7fe4c0155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.588+0000 7fe4be7fc700 1 -- 172.21.15.111:0/1609137207 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b8009b30 con 0x7fe4c0071030 2023-03-29T10:58:46.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.589+0000 7fe4be7fc700 1 --2- 172.21.15.111:0/1609137207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c0071030 0x7fe4c015b7d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4006fd0 tx=0x7fe4b400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:46.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.589+0000 7fe4a77fe700 1 -- 172.21.15.111:0/1609137207 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4005c20 con 0x7fe4c0071030 2023-03-29T10:58:46.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.589+0000 7fe4a77fe700 1 -- 172.21.15.111:0/1609137207 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b4005d80 con 0x7fe4c0071030 2023-03-29T10:58:46.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.589+0000 7fe4a77fe700 1 -- 172.21.15.111:0/1609137207 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4011840 con 0x7fe4c0071030 2023-03-29T10:58:46.589 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.589+0000 7fe4c53ff700 1 -- 172.21.15.111:0/1609137207 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c00719e0 con 0x7fe4c0071030 2023-03-29T10:58:46.590 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.589+0000 7fe4c53ff700 1 -- 172.21.15.111:0/1609137207 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c0071700 con 0x7fe4c0071030 2023-03-29T10:58:46.590 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.590+0000 7fe4c53ff700 1 -- 172.21.15.111:0/1609137207 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c006c6c0 con 0x7fe4c0071030 2023-03-29T10:58:46.590 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.591+0000 7fe4a77fe700 1 -- 172.21.15.111:0/1609137207 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fe4b40119a0 con 0x7fe4c0071030 2023-03-29T10:58:46.591 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.592+0000 7fe4a77fe700 1 --2- 172.21.15.111:0/1609137207 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe4ac069de0 0x7fe4ac06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:46.591 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.592+0000 7fe4a77fe700 1 -- 172.21.15.111:0/1609137207 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe4b40884d0 con 0x7fe4c0071030 2023-03-29T10:58:46.592 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.592+0000 7fe4beffd700 1 --2- 172.21.15.111:0/1609137207 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe4ac069de0 0x7fe4ac06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:46.592 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.593+0000 7fe4beffd700 1 --2- 172.21.15.111:0/1609137207 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe4ac069de0 0x7fe4ac06c290 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00ad4d0 tx=0x7fe4b0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:46.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.596+0000 7fe4a77fe700 1 -- 172.21.15.111:0/1609137207 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fe4b4053a40 con 0x7fe4c0071030 2023-03-29T10:58:46.788 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.789+0000 7fe4c53ff700 1 -- 172.21.15.111:0/1609137207 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe4c015ce00 con 0x7fe4c0071030 2023-03-29T10:58:46.789 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.789+0000 7fe4a77fe700 1 -- 172.21.15.111:0/1609137207 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe4c015ce00 con 0x7fe4c0071030 2023-03-29T10:58:46.789 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-03-29T10:58:46.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.792+0000 7fe4c53ff700 1 -- 172.21.15.111:0/1609137207 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe4ac069de0 msgr2=0x7fe4ac06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:46.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.792+0000 7fe4c53ff700 1 --2- 172.21.15.111:0/1609137207 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe4ac069de0 0x7fe4ac06c290 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00ad4d0 tx=0x7fe4b0008040 comp rx=0 tx=0).stop 2023-03-29T10:58:46.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.792+0000 7fe4c53ff700 1 -- 172.21.15.111:0/1609137207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c0071030 msgr2=0x7fe4c015b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:46.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.792+0000 7fe4c53ff700 1 --2- 172.21.15.111:0/1609137207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c0071030 0x7fe4c015b7d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4006fd0 tx=0x7fe4b400ca30 comp rx=0 tx=0).stop 2023-03-29T10:58:46.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.792+0000 7fe4c53ff700 1 -- 172.21.15.111:0/1609137207 shutdown_connections 2023-03-29T10:58:46.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.792+0000 7fe4c53ff700 1 --2- 172.21.15.111:0/1609137207 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe4ac069de0 0x7fe4ac06c290 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.792+0000 7fe4c53ff700 1 --2- 172.21.15.111:0/1609137207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4c015a5a0 0x7fe4c0155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.792+0000 7fe4c53ff700 1 --2- 172.21.15.111:0/1609137207 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe4c006a920 0x7fe4c00cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.793 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.792+0000 7fe4c53ff700 1 --2- 172.21.15.111:0/1609137207 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe4c0071030 0x7fe4c015b7d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.793 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.792+0000 7fe4c53ff700 1 -- 172.21.15.111:0/1609137207 >> 172.21.15.111:0/1609137207 conn(0x7fe4c00cf720 msgr2=0x7fe4c00c8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:46.793 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.792+0000 7fe4c53ff700 1 -- 172.21.15.111:0/1609137207 shutdown_connections 2023-03-29T10:58:46.793 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.792+0000 7fe4c53ff700 1 -- 172.21.15.111:0/1609137207 wait complete. 2023-03-29T10:58:46.809 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-29T10:58:46.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.995+0000 7f22de5ff700 1 Processor -- start 2023-03-29T10:58:46.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.995+0000 7f22de5ff700 1 -- start start 2023-03-29T10:58:46.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.997+0000 7f22de5ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f22d80d2110 0x7f22d8167550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:46.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.997+0000 7f22de5ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d8161000 con 0x7f22d80b87a0 2023-03-29T10:58:46.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.997+0000 7f22de5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d80d0020 con 0x7f22d80cfbf0 2023-03-29T10:58:46.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.997+0000 7f22de5ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d80d02d0 con 0x7f22d80d2110 2023-03-29T10:58:46.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.997+0000 7f22d7fff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f22d80b87a0 0x7f22d81601e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54646/0 (socket says 172.21.15.111:54646) 2023-03-29T10:58:46.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.997+0000 7f22d7fff700 1 -- 172.21.15.111:0/25915463 learned_addr learned my addr 172.21.15.111:0/25915463 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:46.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.997+0000 7f22dcb9c700 1 --2- 172.21.15.111:0/25915463 >> v2:172.21.15.111:3300/0 conn(0x7f22d80d2110 0x7f22d8167550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:46.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.997+0000 7f22d6ffd700 1 -- 172.21.15.111:0/25915463 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2291574421 0 0) 0x7f22d8161000 con 0x7f22d80b87a0 2023-03-29T10:58:46.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.997+0000 7f22d6ffd700 1 -- 172.21.15.111:0/25915463 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22c00035b0 con 0x7f22d80b87a0 2023-03-29T10:58:46.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.997+0000 7f22dcb9c700 1 -- 172.21.15.111:0/25915463 >> v1:172.21.15.119:6789/0 conn(0x7f22d80cfbf0 legacy=0x7f22d81608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:46.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.997+0000 7f22dcb9c700 1 -- 172.21.15.111:0/25915463 >> v1:172.21.15.111:6789/0 conn(0x7f22d80b87a0 legacy=0x7f22d81601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:46.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.997+0000 7f22dcb9c700 1 -- 172.21.15.111:0/25915463 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d80cdc60 con 0x7f22d80d2110 2023-03-29T10:58:46.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.998+0000 7f22dcb9c700 1 --2- 172.21.15.111:0/25915463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80d2110 0x7f22d8167550 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009d90 tx=0x7f22d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbfa0a51c9a3718 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:46.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.998+0000 7f22d6ffd700 1 -- 172.21.15.111:0/25915463 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d000e070 con 0x7f22d80d2110 2023-03-29T10:58:46.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.998+0000 7f22d6ffd700 1 -- 172.21.15.111:0/25915463 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d000bb20 con 0x7f22d80d2110 2023-03-29T10:58:46.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.998+0000 7f22d6ffd700 1 -- 172.21.15.111:0/25915463 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0012430 con 0x7f22d80d2110 2023-03-29T10:58:46.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.998+0000 7f22de5ff700 1 -- 172.21.15.111:0/25915463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80d2110 msgr2=0x7f22d8167550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:46.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.998+0000 7f22de5ff700 1 --2- 172.21.15.111:0/25915463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80d2110 0x7f22d8167550 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009d90 tx=0x7f22d000b330 comp rx=0 tx=0).stop 2023-03-29T10:58:46.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.998+0000 7f22de5ff700 1 -- 172.21.15.111:0/25915463 shutdown_connections 2023-03-29T10:58:46.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.998+0000 7f22de5ff700 1 --2- 172.21.15.111:0/25915463 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80d2110 0x7f22d8167550 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:46.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.998+0000 7f22de5ff700 1 -- 172.21.15.111:0/25915463 >> 172.21.15.111:0/25915463 conn(0x7f22d8071b10 msgr2=0x7f22d8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:46.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.999+0000 7f22de5ff700 1 -- 172.21.15.111:0/25915463 shutdown_connections 2023-03-29T10:58:46.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:46.999+0000 7f22de5ff700 1 -- 172.21.15.111:0/25915463 wait complete. 2023-03-29T10:58:46.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.000+0000 7f22de5ff700 1 Processor -- start 2023-03-29T10:58:47.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.000+0000 7f22de5ff700 1 -- start start 2023-03-29T10:58:47.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.001+0000 7f22de5ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80b87a0 0x7f22d80d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:47.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.001+0000 7f22de5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22d80cfbf0 0x7f22d80c69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:47.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.001+0000 7f22de5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d80d2110 0x7f22d81543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:47.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.001+0000 7f22de5ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80c7c40 con 0x7f22d80b87a0 2023-03-29T10:58:47.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.001+0000 7f22de5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80c83e0 con 0x7f22d80d2110 2023-03-29T10:58:47.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.001+0000 7f22de5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80c73a0 con 0x7f22d80cfbf0 2023-03-29T10:58:47.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.001+0000 7f22d7fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80b87a0 0x7f22d80d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:47.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.001+0000 7f22d7fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80b87a0 0x7f22d80d2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59230/0 (socket says 172.21.15.111:59230) 2023-03-29T10:58:47.001 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.001+0000 7f22d7fff700 1 -- 172.21.15.111:0/2178285321 learned_addr learned my addr 172.21.15.111:0/2178285321 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:47.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.002+0000 7f22dcb9c700 1 --2- 172.21.15.111:0/2178285321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d80d2110 0x7f22d81543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:47.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.002+0000 7f22d77fe700 1 --2- 172.21.15.111:0/2178285321 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22d80cfbf0 0x7f22d80c69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:47.002 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.002+0000 7f22d7fff700 1 -- 172.21.15.111:0/2178285321 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22d80cfbf0 msgr2=0x7f22d80c69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:47.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.002+0000 7f22d7fff700 1 --2- 172.21.15.111:0/2178285321 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22d80cfbf0 0x7f22d80c69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:47.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.002+0000 7f22d7fff700 1 -- 172.21.15.111:0/2178285321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d80d2110 msgr2=0x7f22d81543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:47.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.002+0000 7f22d7fff700 1 --2- 172.21.15.111:0/2178285321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d80d2110 0x7f22d81543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:47.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.002+0000 7f22d7fff700 1 -- 172.21.15.111:0/2178285321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d0009a70 con 0x7f22d80b87a0 2023-03-29T10:58:47.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.002+0000 7f22d77fe700 1 --2- 172.21.15.111:0/2178285321 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22d80cfbf0 0x7f22d80c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:47.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.002+0000 7f22d7fff700 1 --2- 172.21.15.111:0/2178285321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80b87a0 0x7f22d80d2a60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f22c8001eb0 tx=0x7f22c800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:47.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.002+0000 7f22d4ff9700 1 -- 172.21.15.111:0/2178285321 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8005d70 con 0x7f22d80b87a0 2023-03-29T10:58:47.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.002+0000 7f22d4ff9700 1 -- 172.21.15.111:0/2178285321 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c80070d0 con 0x7f22d80b87a0 2023-03-29T10:58:47.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.002+0000 7f22d4ff9700 1 -- 172.21.15.111:0/2178285321 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c8007460 con 0x7f22d80b87a0 2023-03-29T10:58:47.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.002+0000 7f22de5ff700 1 -- 172.21.15.111:0/2178285321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d80cd980 con 0x7f22d80b87a0 2023-03-29T10:58:47.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.002+0000 7f22de5ff700 1 -- 172.21.15.111:0/2178285321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d80cd6a0 con 0x7f22d80b87a0 2023-03-29T10:58:47.004 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.004+0000 7f22de5ff700 1 -- 172.21.15.111:0/2178285321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d80c8710 con 0x7f22d80b87a0 2023-03-29T10:58:47.004 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.004+0000 7f22d4ff9700 1 -- 172.21.15.111:0/2178285321 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f22c80075c0 con 0x7f22d80b87a0 2023-03-29T10:58:47.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.006+0000 7f22d4ff9700 1 --2- 172.21.15.111:0/2178285321 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f22c4069de0 0x7f22c406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:47.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.006+0000 7f22d4ff9700 1 -- 172.21.15.111:0/2178285321 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f22c80877f0 con 0x7f22d80b87a0 2023-03-29T10:58:47.005 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.006+0000 7f22d77fe700 1 --2- 172.21.15.111:0/2178285321 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f22c4069de0 0x7f22c406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:47.006 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.006+0000 7f22d77fe700 1 --2- 172.21.15.111:0/2178285321 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f22c4069de0 0x7f22c406c290 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f22cc000f60 tx=0x7f22cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:47.009 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.010+0000 7f22d4ff9700 1 -- 172.21.15.111:0/2178285321 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f22c8052e00 con 0x7f22d80b87a0 2023-03-29T10:58:47.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.203+0000 7f22de5ff700 1 -- 172.21.15.111:0/2178285321 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f22d80b52f0 con 0x7f22d80b87a0 2023-03-29T10:58:47.203 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.204+0000 7f22d4ff9700 1 -- 172.21.15.111:0/2178285321 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f22c8056420 con 0x7f22d80b87a0 2023-03-29T10:58:47.203 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:47.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.206+0000 7f22de5ff700 1 -- 172.21.15.111:0/2178285321 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f22c4069de0 msgr2=0x7f22c406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:47.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.206+0000 7f22de5ff700 1 --2- 172.21.15.111:0/2178285321 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f22c4069de0 0x7f22c406c290 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f22cc000f60 tx=0x7f22cc009040 comp rx=0 tx=0).stop 2023-03-29T10:58:47.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.206+0000 7f22de5ff700 1 -- 172.21.15.111:0/2178285321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80b87a0 msgr2=0x7f22d80d2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:47.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.206+0000 7f22de5ff700 1 --2- 172.21.15.111:0/2178285321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80b87a0 0x7f22d80d2a60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f22c8001eb0 tx=0x7f22c800c860 comp rx=0 tx=0).stop 2023-03-29T10:58:47.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.206+0000 7f22de5ff700 1 -- 172.21.15.111:0/2178285321 shutdown_connections 2023-03-29T10:58:47.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.206+0000 7f22de5ff700 1 --2- 172.21.15.111:0/2178285321 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f22c4069de0 0x7f22c406c290 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:47.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.206+0000 7f22de5ff700 1 --2- 172.21.15.111:0/2178285321 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80b87a0 0x7f22d80d2a60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:47.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.206+0000 7f22de5ff700 1 --2- 172.21.15.111:0/2178285321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d80d2110 0x7f22d81543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:47.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.206+0000 7f22de5ff700 1 --2- 172.21.15.111:0/2178285321 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22d80cfbf0 0x7f22d80c69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:47.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.206+0000 7f22de5ff700 1 -- 172.21.15.111:0/2178285321 >> 172.21.15.111:0/2178285321 conn(0x7f22d8071b10 msgr2=0x7f22d80cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:47.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.206+0000 7f22de5ff700 1 -- 172.21.15.111:0/2178285321 shutdown_connections 2023-03-29T10:58:47.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.206+0000 7f22de5ff700 1 -- 172.21.15.111:0/2178285321 wait complete. 2023-03-29T10:58:47.223 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-03-29T10:58:47.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.407+0000 7fb3acfff700 1 Processor -- start 2023-03-29T10:58:47.407 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.407+0000 7fb3acfff700 1 -- start start 2023-03-29T10:58:47.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.408+0000 7fb3acfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb3a80b87a0 0x7fb3a80cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:47.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.408+0000 7fb3acfff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3a80cdc60 con 0x7fb3a8160d40 2023-03-29T10:58:47.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.409+0000 7fb3acfff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3a80cd6a0 con 0x7fb3a81601e0 2023-03-29T10:58:47.408 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.409+0000 7fb3acfff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3a80d02d0 con 0x7fb3a80b87a0 2023-03-29T10:58:47.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.409+0000 7fb3a659c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb3a80b87a0 0x7fb3a80cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:47.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.409+0000 7fb3a6d9d700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb3a8160d40 0x7fb3a80d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54650/0 (socket says 172.21.15.111:54650) 2023-03-29T10:58:47.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.409+0000 7fb3a6d9d700 1 -- 172.21.15.111:0/4181504526 learned_addr learned my addr 172.21.15.111:0/4181504526 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:47.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.409+0000 7fb3a5d9b700 1 -- 172.21.15.111:0/4181504526 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2868332363 0 0) 0x7fb3a80cdc60 con 0x7fb3a8160d40 2023-03-29T10:58:47.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.409+0000 7fb3a5d9b700 1 -- 172.21.15.111:0/4181504526 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3940035b0 con 0x7fb3a8160d40 2023-03-29T10:58:47.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.409+0000 7fb3a659c700 1 -- 172.21.15.111:0/4181504526 >> v1:172.21.15.119:6789/0 conn(0x7fb3a81601e0 legacy=0x7fb3a8160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:47.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.409+0000 7fb3a5d9b700 1 -- 172.21.15.111:0/4181504526 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245513136 0 0) 0x7fb3a80cd6a0 con 0x7fb3a81601e0 2023-03-29T10:58:47.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.409+0000 7fb3a659c700 1 -- 172.21.15.111:0/4181504526 >> v1:172.21.15.111:6789/0 conn(0x7fb3a8160d40 legacy=0x7fb3a80d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:47.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.409+0000 7fb3a659c700 1 -- 172.21.15.111:0/4181504526 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a80cd980 con 0x7fb3a80b87a0 2023-03-29T10:58:47.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.410+0000 7fb3a659c700 1 --2- 172.21.15.111:0/4181504526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3a80b87a0 0x7fb3a80cfbf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb398009840 tx=0x7fb398009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4540af721ea568f server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:47.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.410+0000 7fb3a5d9b700 1 -- 172.21.15.111:0/4181504526 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb398011830 con 0x7fb3a80b87a0 2023-03-29T10:58:47.410 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.410+0000 7fb3a5d9b700 1 -- 172.21.15.111:0/4181504526 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb398011990 con 0x7fb3a80b87a0 2023-03-29T10:58:47.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.410+0000 7fb3a5d9b700 1 -- 172.21.15.111:0/4181504526 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb398011d20 con 0x7fb3a80b87a0 2023-03-29T10:58:47.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.410+0000 7fb3acfff700 1 -- 172.21.15.111:0/4181504526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3a80b87a0 msgr2=0x7fb3a80cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:47.411 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.410+0000 7fb3acfff700 1 --2- 172.21.15.111:0/4181504526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3a80b87a0 0x7fb3a80cfbf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb398009840 tx=0x7fb398009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:47.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.410+0000 7fb3acfff700 1 -- 172.21.15.111:0/4181504526 shutdown_connections 2023-03-29T10:58:47.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.410+0000 7fb3acfff700 1 --2- 172.21.15.111:0/4181504526 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3a80b87a0 0x7fb3a80cfbf0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:47.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.410+0000 7fb3acfff700 1 -- 172.21.15.111:0/4181504526 >> 172.21.15.111:0/4181504526 conn(0x7fb3a8071b10 msgr2=0x7fb3a8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:47.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.410+0000 7fb3acfff700 1 -- 172.21.15.111:0/4181504526 shutdown_connections 2023-03-29T10:58:47.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.410+0000 7fb3acfff700 1 -- 172.21.15.111:0/4181504526 wait complete. 2023-03-29T10:58:47.412 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.412+0000 7fb3acfff700 1 Processor -- start 2023-03-29T10:58:47.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.412+0000 7fb3acfff700 1 -- start start 2023-03-29T10:58:47.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.413+0000 7fb3acfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3a80b87a0 0x7fb3a815dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:47.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.413+0000 7fb3acfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3a81601e0 0x7fb3a814ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:47.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.413+0000 7fb3acfff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb3a8160d40 0x7fb3a814e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:47.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.413+0000 7fb3acfff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a80c7030 con 0x7fb3a80b87a0 2023-03-29T10:58:47.414 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.413+0000 7fb3acfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a80cbad0 con 0x7fb3a81601e0 2023-03-29T10:58:47.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.413+0000 7fb3acfff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a80caf50 con 0x7fb3a8160d40 2023-03-29T10:58:47.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.413+0000 7fb3a659c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3a80b87a0 0x7fb3a815dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:47.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.413+0000 7fb3a659c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3a80b87a0 0x7fb3a815dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59250/0 (socket says 172.21.15.111:59250) 2023-03-29T10:58:47.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.413+0000 7fb3a659c700 1 -- 172.21.15.111:0/1156494131 learned_addr learned my addr 172.21.15.111:0/1156494131 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:47.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.414+0000 7fb39ffff700 1 --2- 172.21.15.111:0/1156494131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3a81601e0 0x7fb3a814ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:47.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.414+0000 7fb3a6d9d700 1 --2- 172.21.15.111:0/1156494131 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb3a8160d40 0x7fb3a814e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:47.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.414+0000 7fb3a659c700 1 -- 172.21.15.111:0/1156494131 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb3a8160d40 msgr2=0x7fb3a814e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:47.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.414+0000 7fb3a659c700 1 --2- 172.21.15.111:0/1156494131 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb3a8160d40 0x7fb3a814e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:47.415 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.414+0000 7fb3a659c700 1 -- 172.21.15.111:0/1156494131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3a81601e0 msgr2=0x7fb3a814ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:47.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.414+0000 7fb3a659c700 1 --2- 172.21.15.111:0/1156494131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3a81601e0 0x7fb3a814ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:47.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.414+0000 7fb3a659c700 1 -- 172.21.15.111:0/1156494131 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3980094a0 con 0x7fb3a80b87a0 2023-03-29T10:58:47.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.414+0000 7fb39ffff700 1 --2- 172.21.15.111:0/1156494131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3a81601e0 0x7fb3a814ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:47.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.414+0000 7fb3a6d9d700 1 --2- 172.21.15.111:0/1156494131 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb3a8160d40 0x7fb3a814e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:47.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.414+0000 7fb3a659c700 1 --2- 172.21.15.111:0/1156494131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3a80b87a0 0x7fb3a815dc60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb3980090b0 tx=0x7fb398004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:47.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.414+0000 7fb39effd700 1 -- 172.21.15.111:0/1156494131 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39801a910 con 0x7fb3a80b87a0 2023-03-29T10:58:47.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.414+0000 7fb39effd700 1 -- 172.21.15.111:0/1156494131 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb39801aa70 con 0x7fb3a80b87a0 2023-03-29T10:58:47.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.414+0000 7fb39effd700 1 -- 172.21.15.111:0/1156494131 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb39801ade0 con 0x7fb3a80b87a0 2023-03-29T10:58:47.416 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.414+0000 7fb3acfff700 1 -- 172.21.15.111:0/1156494131 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a80d19a0 con 0x7fb3a80b87a0 2023-03-29T10:58:47.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.415+0000 7fb3acfff700 1 -- 172.21.15.111:0/1156494131 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a80d1810 con 0x7fb3a80b87a0 2023-03-29T10:58:47.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.416+0000 7fb3acfff700 1 -- 172.21.15.111:0/1156494131 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a80d1b30 con 0x7fb3a80b87a0 2023-03-29T10:58:47.417 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.416+0000 7fb39effd700 1 -- 172.21.15.111:0/1156494131 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fb398022070 con 0x7fb3a80b87a0 2023-03-29T10:58:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.417+0000 7fb39effd700 1 --2- 172.21.15.111:0/1156494131 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb388069de0 0x7fb38806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.417+0000 7fb39effd700 1 -- 172.21.15.111:0/1156494131 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb398089120 con 0x7fb3a80b87a0 2023-03-29T10:58:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.417+0000 7fb39ffff700 1 --2- 172.21.15.111:0/1156494131 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb388069de0 0x7fb38806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:47.418 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.418+0000 7fb39ffff700 1 --2- 172.21.15.111:0/1156494131 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb388069de0 0x7fb38806c290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb3a815d400 tx=0x7fb390009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:47.421 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.421+0000 7fb39effd700 1 -- 172.21.15.111:0/1156494131 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fb3980546b0 con 0x7fb3a80b87a0 2023-03-29T10:58:47.630 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.631+0000 7fb3acfff700 1 -- 172.21.15.111:0/1156494131 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb3a80b7cb0 con 0x7fb3a80b87a0 2023-03-29T10:58:47.632 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.633+0000 7fb39effd700 1 -- 172.21.15.111:0/1156494131 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb398015080 con 0x7fb3a80b87a0 2023-03-29T10:58:47.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.635+0000 7fb3acfff700 1 -- 172.21.15.111:0/1156494131 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb388069de0 msgr2=0x7fb38806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:47.634 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.635+0000 7fb3acfff700 1 --2- 172.21.15.111:0/1156494131 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb388069de0 0x7fb38806c290 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb3a815d400 tx=0x7fb390009040 comp rx=0 tx=0).stop 2023-03-29T10:58:47.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.635+0000 7fb3acfff700 1 -- 172.21.15.111:0/1156494131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3a80b87a0 msgr2=0x7fb3a815dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:47.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.635+0000 7fb3acfff700 1 --2- 172.21.15.111:0/1156494131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3a80b87a0 0x7fb3a815dc60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb3980090b0 tx=0x7fb398004750 comp rx=0 tx=0).stop 2023-03-29T10:58:47.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.635+0000 7fb3acfff700 1 -- 172.21.15.111:0/1156494131 shutdown_connections 2023-03-29T10:58:47.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.635+0000 7fb3acfff700 1 --2- 172.21.15.111:0/1156494131 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb388069de0 0x7fb38806c290 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:47.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.635+0000 7fb3acfff700 1 --2- 172.21.15.111:0/1156494131 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb3a80b87a0 0x7fb3a815dc60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:47.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.635+0000 7fb3acfff700 1 --2- 172.21.15.111:0/1156494131 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb3a8160d40 0x7fb3a814e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:47.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.635+0000 7fb3acfff700 1 --2- 172.21.15.111:0/1156494131 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3a81601e0 0x7fb3a814ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:47.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.635+0000 7fb3acfff700 1 -- 172.21.15.111:0/1156494131 >> 172.21.15.111:0/1156494131 conn(0x7fb3a8071b10 msgr2=0x7fb3a8071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:47.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.635+0000 7fb3acfff700 1 -- 172.21.15.111:0/1156494131 shutdown_connections 2023-03-29T10:58:47.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:47.635+0000 7fb3acfff700 1 -- 172.21.15.111:0/1156494131 wait complete. 2023-03-29T10:58:47.635 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-03-29T10:58:47.652 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi111.front.sepia.ceph.com... 2023-03-29T10:58:47.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:47.652 DEBUG:teuthology.orchestra.run.smithi111:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-03-29T10:58:47.715 INFO:teuthology.orchestra.run:waiting for 300 2023-03-29T10:58:47.715 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-03-29T10:58:47.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:47.715 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-03-29T10:58:47.756 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:58:47.757 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-29T10:58:47.803 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:58:47.803 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-29T10:58:47.888 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:58:48.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.069+0000 7f3e1739d700 1 Processor -- start 2023-03-29T10:58:48.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.069+0000 7f3e1739d700 1 -- start start 2023-03-29T10:58:48.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.071+0000 7f3e1739d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3e101653f0 0x7f3e101677d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:48.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.071+0000 7f3e1739d700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e1015c9f0 con 0x7f3e100d3890 2023-03-29T10:58:48.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.071+0000 7f3e1739d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e100cd930 con 0x7f3e1015c620 2023-03-29T10:58:48.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.071+0000 7f3e1739d700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e100d01d0 con 0x7f3e101653f0 2023-03-29T10:58:48.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.071+0000 7f3e15b9a700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f3e100d3890 0x7f3e101607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:54656/0 (socket says 172.21.15.111:54656) 2023-03-29T10:58:48.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.071+0000 7f3e15b9a700 1 -- 172.21.15.111:0/3524185944 learned_addr learned my addr 172.21.15.111:0/3524185944 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:48.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.071+0000 7f3e16b9c700 1 --2- 172.21.15.111:0/3524185944 >> v2:172.21.15.111:3300/0 conn(0x7f3e101653f0 0x7f3e101677d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:48.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.071+0000 7f3e15399700 1 -- 172.21.15.111:0/3524185944 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656008288 0 0) 0x7f3e00001110 con 0x7f3e100d3890 2023-03-29T10:58:48.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.071+0000 7f3e15399700 1 -- 172.21.15.111:0/3524185944 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dfc0035b0 con 0x7f3e100d3890 2023-03-29T10:58:48.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.071+0000 7f3e16b9c700 1 -- 172.21.15.111:0/3524185944 >> v1:172.21.15.119:6789/0 conn(0x7f3e1015c620 legacy=0x7f3e10160090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:48.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.071+0000 7f3e15399700 1 -- 172.21.15.111:0/3524185944 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 260790713 0 0) 0x7f3e100cd930 con 0x7f3e1015c620 2023-03-29T10:58:48.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.071+0000 7f3e16b9c700 1 -- 172.21.15.111:0/3524185944 >> v1:172.21.15.111:6789/0 conn(0x7f3e100d3890 legacy=0x7f3e101607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:48.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.071+0000 7f3e16b9c700 1 -- 172.21.15.111:0/3524185944 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e10160eb0 con 0x7f3e101653f0 2023-03-29T10:58:48.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.072+0000 7f3e16b9c700 1 --2- 172.21.15.111:0/3524185944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e101653f0 0x7f3e101677d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3e08009d90 tx=0x7f3e0800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42de7d89ae8ef16f server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:48.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.072+0000 7f3e15399700 1 -- 172.21.15.111:0/3524185944 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0800e070 con 0x7f3e101653f0 2023-03-29T10:58:48.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.072+0000 7f3e15399700 1 -- 172.21.15.111:0/3524185944 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0800bb20 con 0x7f3e101653f0 2023-03-29T10:58:48.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.072+0000 7f3e15399700 1 -- 172.21.15.111:0/3524185944 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0800bed0 con 0x7f3e101653f0 2023-03-29T10:58:48.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.072+0000 7f3e1739d700 1 -- 172.21.15.111:0/3524185944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e101653f0 msgr2=0x7f3e101677d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:48.072 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.072+0000 7f3e1739d700 1 --2- 172.21.15.111:0/3524185944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e101653f0 0x7f3e101677d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3e08009d90 tx=0x7f3e0800b330 comp rx=0 tx=0).stop 2023-03-29T10:58:48.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.072+0000 7f3e1739d700 1 -- 172.21.15.111:0/3524185944 shutdown_connections 2023-03-29T10:58:48.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.073+0000 7f3e1739d700 1 --2- 172.21.15.111:0/3524185944 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e101653f0 0x7f3e101677d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.073+0000 7f3e1739d700 1 -- 172.21.15.111:0/3524185944 >> 172.21.15.111:0/3524185944 conn(0x7f3e10071b10 msgr2=0x7f3e10071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:48.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.073+0000 7f3e1739d700 1 -- 172.21.15.111:0/3524185944 shutdown_connections 2023-03-29T10:58:48.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.073+0000 7f3e1739d700 1 -- 172.21.15.111:0/3524185944 wait complete. 2023-03-29T10:58:48.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.074+0000 7f3e1739d700 1 Processor -- start 2023-03-29T10:58:48.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.074+0000 7f3e1739d700 1 -- start start 2023-03-29T10:58:48.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.075+0000 7f3e1739d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e100d3890 0x7f3e100ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:48.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.075+0000 7f3e1739d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e1015c620 0x7f3e10154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:48.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.075+0000 7f3e1739d700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3e101653f0 0x7f3e1015d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:48.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.075+0000 7f3e1739d700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e10154c80 con 0x7f3e1015c620 2023-03-29T10:58:48.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.075+0000 7f3e1739d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1015ddf0 con 0x7f3e100d3890 2023-03-29T10:58:48.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.075+0000 7f3e1739d700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e100c6900 con 0x7f3e101653f0 2023-03-29T10:58:48.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.075+0000 7f3e15b9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e1015c620 0x7f3e10154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:48.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.075+0000 7f3e15b9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e1015c620 0x7f3e10154740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:59266/0 (socket says 172.21.15.111:59266) 2023-03-29T10:58:48.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.075+0000 7f3e15b9a700 1 -- 172.21.15.111:0/290498931 learned_addr learned my addr 172.21.15.111:0/290498931 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:48.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.075+0000 7f3e1639b700 1 --2- 172.21.15.111:0/290498931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e100d3890 0x7f3e100ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:48.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.076+0000 7f3e16b9c700 1 --2- 172.21.15.111:0/290498931 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3e101653f0 0x7f3e1015d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:48.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.076+0000 7f3e15b9a700 1 -- 172.21.15.111:0/290498931 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3e101653f0 msgr2=0x7f3e1015d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:48.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.076+0000 7f3e15b9a700 1 --2- 172.21.15.111:0/290498931 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3e101653f0 0x7f3e1015d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.076+0000 7f3e15b9a700 1 -- 172.21.15.111:0/290498931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e100d3890 msgr2=0x7f3e100ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:48.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.076+0000 7f3e15b9a700 1 --2- 172.21.15.111:0/290498931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e100d3890 0x7f3e100ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.076+0000 7f3e15b9a700 1 -- 172.21.15.111:0/290498931 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e08009a70 con 0x7f3e1015c620 2023-03-29T10:58:48.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.076+0000 7f3e15b9a700 1 --2- 172.21.15.111:0/290498931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e1015c620 0x7f3e10154740 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000c4d0 tx=0x7f3e0000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:48.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.076+0000 7f3e06ffd700 1 -- 172.21.15.111:0/290498931 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e00005d70 con 0x7f3e1015c620 2023-03-29T10:58:48.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.076+0000 7f3e06ffd700 1 -- 172.21.15.111:0/290498931 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e000070d0 con 0x7f3e1015c620 2023-03-29T10:58:48.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.076+0000 7f3e1739d700 1 -- 172.21.15.111:0/290498931 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e100d17c0 con 0x7f3e1015c620 2023-03-29T10:58:48.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.076+0000 7f3e06ffd700 1 -- 172.21.15.111:0/290498931 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e00007480 con 0x7f3e1015c620 2023-03-29T10:58:48.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.076+0000 7f3e1739d700 1 -- 172.21.15.111:0/290498931 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e100d14e0 con 0x7f3e1015c620 2023-03-29T10:58:48.077 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.077+0000 7f3e1739d700 1 -- 172.21.15.111:0/290498931 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e100cd5c0 con 0x7f3e1015c620 2023-03-29T10:58:48.078 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.078+0000 7f3e06ffd700 1 -- 172.21.15.111:0/290498931 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f3e000075e0 con 0x7f3e1015c620 2023-03-29T10:58:48.079 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.079+0000 7f3e06ffd700 1 --2- 172.21.15.111:0/290498931 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3df4069de0 0x7f3df406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:48.079 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.079+0000 7f3e06ffd700 1 -- 172.21.15.111:0/290498931 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3e00087620 con 0x7f3e1015c620 2023-03-29T10:58:48.079 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.079+0000 7f3e1639b700 1 --2- 172.21.15.111:0/290498931 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3df4069de0 0x7f3df406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:48.079 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.080+0000 7f3e1639b700 1 --2- 172.21.15.111:0/290498931 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3df4069de0 0x7f3df406c290 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c007fd0 tx=0x7f3e0c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:48.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.083+0000 7f3e06ffd700 1 -- 172.21.15.111:0/290498931 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f3e00052b00 con 0x7f3e1015c620 2023-03-29T10:58:48.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.280+0000 7f3e1739d700 1 -- 172.21.15.111:0/290498931 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e1015c9f0 con 0x7f3e1015c620 2023-03-29T10:58:48.281 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.282+0000 7f3e06ffd700 1 -- 172.21.15.111:0/290498931 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19892 (secure 0 0 0) 0x7f3e1015c9f0 con 0x7f3e1015c620 2023-03-29T10:58:48.282 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:48.282 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":30,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:58:09.476334+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.26","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:58:48.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.284+0000 7f3e1739d700 1 -- 172.21.15.111:0/290498931 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3df4069de0 msgr2=0x7f3df406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.284+0000 7f3e1739d700 1 --2- 172.21.15.111:0/290498931 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3df4069de0 0x7f3df406c290 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c007fd0 tx=0x7f3e0c009040 comp rx=0 tx=0).stop 2023-03-29T10:58:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.284+0000 7f3e1739d700 1 -- 172.21.15.111:0/290498931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e1015c620 msgr2=0x7f3e10154740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.284+0000 7f3e1739d700 1 --2- 172.21.15.111:0/290498931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e1015c620 0x7f3e10154740 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3e0000c4d0 tx=0x7f3e0000c7e0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.284+0000 7f3e1739d700 1 -- 172.21.15.111:0/290498931 shutdown_connections 2023-03-29T10:58:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.284+0000 7f3e1739d700 1 --2- 172.21.15.111:0/290498931 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3df4069de0 0x7f3df406c290 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.284+0000 7f3e1739d700 1 --2- 172.21.15.111:0/290498931 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3e101653f0 0x7f3e1015d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.284+0000 7f3e1739d700 1 --2- 172.21.15.111:0/290498931 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e100d3890 0x7f3e100ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.284+0000 7f3e1739d700 1 --2- 172.21.15.111:0/290498931 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3e1015c620 0x7f3e10154740 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.284+0000 7f3e1739d700 1 -- 172.21.15.111:0/290498931 >> 172.21.15.111:0/290498931 conn(0x7f3e10071b10 msgr2=0x7f3e1015b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:48.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.285+0000 7f3e1739d700 1 -- 172.21.15.111:0/290498931 shutdown_connections 2023-03-29T10:58:48.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.285+0000 7f3e1739d700 1 -- 172.21.15.111:0/290498931 wait complete. 2023-03-29T10:58:48.302 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-29T10:57:16.183979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-29T10:57:21.239809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-29T10:57:26.192124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-29T10:57:26.796777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-29T10:57:29.874795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-29T10:58:48.302 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-29T10:58:48.488 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.488+0000 7fe735bff700 1 Processor -- start 2023-03-29T10:58:48.488 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.488+0000 7fe735bff700 1 -- start start 2023-03-29T10:58:48.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.489+0000 7fe735bff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe7301589d0 0x7fe7300cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:48.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.489+0000 7fe735bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe730158da0 0x7fe7300cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:48.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.490+0000 7fe735bff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7300ca890 con 0x7fe7300cd690 2023-03-29T10:58:48.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.490+0000 7fe735bff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7300ca5b0 con 0x7fe7301589d0 2023-03-29T10:58:48.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.490+0000 7fe735bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7300cc670 con 0x7fe730158da0 2023-03-29T10:58:48.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.490+0000 7fe72f7fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fe7301589d0 0x7fe7300cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:48.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.490+0000 7fe72ffff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fe7300cd690 0x7fe73006c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47362/0 (socket says 172.21.15.111:47362) 2023-03-29T10:58:48.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.490+0000 7fe72ffff700 1 -- 172.21.15.111:0/555728390 learned_addr learned my addr 172.21.15.111:0/555728390 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:48.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.490+0000 7fe727fff700 1 --2- 172.21.15.111:0/555728390 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe730158da0 0x7fe7300cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:48.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.490+0000 7fe72effd700 1 -- 172.21.15.111:0/555728390 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 368037665 0 0) 0x7fe7300ca890 con 0x7fe7300cd690 2023-03-29T10:58:48.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.490+0000 7fe72effd700 1 -- 172.21.15.111:0/555728390 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe71c0035b0 con 0x7fe7300cd690 2023-03-29T10:58:48.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.490+0000 7fe72f7fe700 1 -- 172.21.15.111:0/555728390 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe730158da0 msgr2=0x7fe7300cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:48.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.490+0000 7fe72f7fe700 1 --2- 172.21.15.111:0/555728390 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe730158da0 0x7fe7300cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.490+0000 7fe72f7fe700 1 -- 172.21.15.111:0/555728390 >> v1:172.21.15.111:6789/0 conn(0x7fe7300cd690 legacy=0x7fe73006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:48.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.490+0000 7fe72f7fe700 1 -- 172.21.15.111:0/555728390 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7300cc950 con 0x7fe7301589d0 2023-03-29T10:58:48.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.490+0000 7fe727fff700 1 --2- 172.21.15.111:0/555728390 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe730158da0 0x7fe7300cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:48.491 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.491+0000 7fe72f7fe700 1 --2- 172.21.15.111:0/555728390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7301589d0 0x7fe7300cbc20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe720009840 tx=0x7fe720009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563ec9450f25dc02 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:48.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.491+0000 7fe72effd700 1 -- 172.21.15.111:0/555728390 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe720011830 con 0x7fe7301589d0 2023-03-29T10:58:48.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.491+0000 7fe72effd700 1 -- 172.21.15.111:0/555728390 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe720011990 con 0x7fe7301589d0 2023-03-29T10:58:48.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.491+0000 7fe72effd700 1 -- 172.21.15.111:0/555728390 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe720011d20 con 0x7fe7301589d0 2023-03-29T10:58:48.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.491+0000 7fe735bff700 1 -- 172.21.15.111:0/555728390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7301589d0 msgr2=0x7fe7300cbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:48.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.491+0000 7fe735bff700 1 --2- 172.21.15.111:0/555728390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7301589d0 0x7fe7300cbc20 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe720009840 tx=0x7fe720009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:48.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.491+0000 7fe735bff700 1 -- 172.21.15.111:0/555728390 shutdown_connections 2023-03-29T10:58:48.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.491+0000 7fe735bff700 1 --2- 172.21.15.111:0/555728390 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7301589d0 0x7fe7300cbc20 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.491+0000 7fe735bff700 1 --2- 172.21.15.111:0/555728390 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe730158da0 0x7fe7300cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.491+0000 7fe735bff700 1 -- 172.21.15.111:0/555728390 >> 172.21.15.111:0/555728390 conn(0x7fe7300cf720 msgr2=0x7fe7300cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:48.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.491+0000 7fe735bff700 1 -- 172.21.15.111:0/555728390 shutdown_connections 2023-03-29T10:58:48.492 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.491+0000 7fe735bff700 1 -- 172.21.15.111:0/555728390 wait complete. 2023-03-29T10:58:48.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.493+0000 7fe735bff700 1 Processor -- start 2023-03-29T10:58:48.493 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.493+0000 7fe735bff700 1 -- start start 2023-03-29T10:58:48.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.494+0000 7fe735bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7300cd690 0x7fe7300c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:48.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.494+0000 7fe735bff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7301589d0 0x7fe7300b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:48.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.494+0000 7fe735bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe730158da0 0x7fe7300b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:48.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.494+0000 7fe735bff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe730159bf0 con 0x7fe7301589d0 2023-03-29T10:58:48.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.494+0000 7fe735bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7300c5350 con 0x7fe7300cd690 2023-03-29T10:58:48.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.494+0000 7fe735bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe730159250 con 0x7fe730158da0 2023-03-29T10:58:48.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.494+0000 7fe727fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7301589d0 0x7fe7300b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:48.495 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.494+0000 7fe727fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7301589d0 0x7fe7300b1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51616/0 (socket says 172.21.15.111:51616) 2023-03-29T10:58:48.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.494+0000 7fe727fff700 1 -- 172.21.15.111:0/2203915432 learned_addr learned my addr 172.21.15.111:0/2203915432 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:48.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.495+0000 7fe72f7fe700 1 --2- 172.21.15.111:0/2203915432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7300cd690 0x7fe7300c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:48.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.495+0000 7fe72ffff700 1 --2- 172.21.15.111:0/2203915432 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe730158da0 0x7fe7300b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:48.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.495+0000 7fe727fff700 1 -- 172.21.15.111:0/2203915432 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe730158da0 msgr2=0x7fe7300b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:48.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.495+0000 7fe727fff700 1 --2- 172.21.15.111:0/2203915432 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe730158da0 0x7fe7300b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.495+0000 7fe727fff700 1 -- 172.21.15.111:0/2203915432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7300cd690 msgr2=0x7fe7300c4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:48.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.495+0000 7fe727fff700 1 --2- 172.21.15.111:0/2203915432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7300cd690 0x7fe7300c4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.495+0000 7fe727fff700 1 -- 172.21.15.111:0/2203915432 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7200094a0 con 0x7fe7301589d0 2023-03-29T10:58:48.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.495+0000 7fe72f7fe700 1 --2- 172.21.15.111:0/2203915432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7300cd690 0x7fe7300c4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:48.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.495+0000 7fe727fff700 1 --2- 172.21.15.111:0/2203915432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7301589d0 0x7fe7300b1970 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe71800e810 tx=0x7fe71800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:48.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.495+0000 7fe72cff9700 1 -- 172.21.15.111:0/2203915432 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718005c20 con 0x7fe7301589d0 2023-03-29T10:58:48.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.495+0000 7fe72cff9700 1 -- 172.21.15.111:0/2203915432 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe718005d80 con 0x7fe7301589d0 2023-03-29T10:58:48.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.495+0000 7fe72cff9700 1 -- 172.21.15.111:0/2203915432 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718006270 con 0x7fe7301589d0 2023-03-29T10:58:48.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.495+0000 7fe735bff700 1 -- 172.21.15.111:0/2203915432 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7300cc160 con 0x7fe7301589d0 2023-03-29T10:58:48.496 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.495+0000 7fe735bff700 1 -- 172.21.15.111:0/2203915432 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7300ccc30 con 0x7fe7301589d0 2023-03-29T10:58:48.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.496+0000 7fe735bff700 1 -- 172.21.15.111:0/2203915432 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7300cc2f0 con 0x7fe7301589d0 2023-03-29T10:58:48.497 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.497+0000 7fe72cff9700 1 -- 172.21.15.111:0/2203915432 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fe7180063d0 con 0x7fe7301589d0 2023-03-29T10:58:48.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.498+0000 7fe72cff9700 1 --2- 172.21.15.111:0/2203915432 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe710069de0 0x7fe71006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:48.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.498+0000 7fe72cff9700 1 -- 172.21.15.111:0/2203915432 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe718087f60 con 0x7fe7301589d0 2023-03-29T10:58:48.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.499+0000 7fe72f7fe700 1 --2- 172.21.15.111:0/2203915432 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe710069de0 0x7fe71006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:48.498 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.499+0000 7fe72f7fe700 1 --2- 172.21.15.111:0/2203915432 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe710069de0 0x7fe71006c290 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe720000c00 tx=0x7fe720008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:48.502 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.502+0000 7fe72cff9700 1 -- 172.21.15.111:0/2203915432 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fe718053440 con 0x7fe7301589d0 2023-03-29T10:58:48.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.699+0000 7fe735bff700 1 -- 172.21.15.111:0/2203915432 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe7300cd0c0 con 0x7fe7301589d0 2023-03-29T10:58:48.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.700+0000 7fe72cff9700 1 -- 172.21.15.111:0/2203915432 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe718020020 con 0x7fe7301589d0 2023-03-29T10:58:48.699 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-03-29T10:58:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.702+0000 7fe735bff700 1 -- 172.21.15.111:0/2203915432 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe710069de0 msgr2=0x7fe71006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.702+0000 7fe735bff700 1 --2- 172.21.15.111:0/2203915432 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe710069de0 0x7fe71006c290 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe720000c00 tx=0x7fe720008030 comp rx=0 tx=0).stop 2023-03-29T10:58:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.702+0000 7fe735bff700 1 -- 172.21.15.111:0/2203915432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7301589d0 msgr2=0x7fe7300b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.702+0000 7fe735bff700 1 --2- 172.21.15.111:0/2203915432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7301589d0 0x7fe7300b1970 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe71800e810 tx=0x7fe71800ebd0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.702+0000 7fe735bff700 1 -- 172.21.15.111:0/2203915432 shutdown_connections 2023-03-29T10:58:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.702+0000 7fe735bff700 1 --2- 172.21.15.111:0/2203915432 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe710069de0 0x7fe71006c290 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.702+0000 7fe735bff700 1 --2- 172.21.15.111:0/2203915432 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fe730158da0 0x7fe7300b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.702+0000 7fe735bff700 1 --2- 172.21.15.111:0/2203915432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7300cd690 0x7fe7300c4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.702 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.702+0000 7fe735bff700 1 --2- 172.21.15.111:0/2203915432 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fe7301589d0 0x7fe7300b1970 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.702+0000 7fe735bff700 1 -- 172.21.15.111:0/2203915432 >> 172.21.15.111:0/2203915432 conn(0x7fe7300cf720 msgr2=0x7fe730153240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:48.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.702+0000 7fe735bff700 1 -- 172.21.15.111:0/2203915432 shutdown_connections 2023-03-29T10:58:48.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.702+0000 7fe735bff700 1 -- 172.21.15.111:0/2203915432 wait complete. 2023-03-29T10:58:48.719 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-29T10:58:48.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.907+0000 7f159b9ff700 1 Processor -- start 2023-03-29T10:58:48.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.907+0000 7f159b9ff700 1 -- start start 2023-03-29T10:58:48.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.908+0000 7f159b9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f15940b87a0 0x7f15940cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:48.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.908+0000 7f159b9ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15940cdc60 con 0x7f15941601e0 2023-03-29T10:58:48.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.909+0000 7f159b9ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15940cd6a0 con 0x7f1594160d40 2023-03-29T10:58:48.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.909+0000 7f159b9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15940d02d0 con 0x7f15940b87a0 2023-03-29T10:58:48.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.909+0000 7f1598f9a700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f15941601e0 0x7f1594160630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47370/0 (socket says 172.21.15.111:47370) 2023-03-29T10:58:48.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.909+0000 7f1598f9a700 1 -- 172.21.15.111:0/3293457523 learned_addr learned my addr 172.21.15.111:0/3293457523 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:48.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.909+0000 7f159979b700 1 --2- 172.21.15.111:0/3293457523 >> v2:172.21.15.119:3300/0 conn(0x7f15940b87a0 0x7f15940cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:48.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.909+0000 7f158bfff700 1 -- 172.21.15.111:0/3293457523 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844262326 0 0) 0x7f15940cdc60 con 0x7f15941601e0 2023-03-29T10:58:48.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.909+0000 7f158bfff700 1 -- 172.21.15.111:0/3293457523 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15800035b0 con 0x7f15941601e0 2023-03-29T10:58:48.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.909+0000 7f158bfff700 1 -- 172.21.15.111:0/3293457523 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3413768818 0 0) 0x7f15800035b0 con 0x7f15941601e0 2023-03-29T10:58:48.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.909+0000 7f158bfff700 1 -- 172.21.15.111:0/3293457523 >> v2:172.21.15.119:3300/0 conn(0x7f15940b87a0 msgr2=0x7f15940cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:48.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.909+0000 7f158bfff700 1 --2- 172.21.15.111:0/3293457523 >> v2:172.21.15.119:3300/0 conn(0x7f15940b87a0 0x7f15940cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.909+0000 7f158bfff700 1 -- 172.21.15.111:0/3293457523 >> v1:172.21.15.119:6789/0 conn(0x7f1594160d40 legacy=0x7f15940d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:48.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.909+0000 7f158bfff700 1 -- 172.21.15.111:0/3293457523 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15940cd980 con 0x7f15941601e0 2023-03-29T10:58:48.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.909+0000 7f158bfff700 1 -- 172.21.15.111:0/3293457523 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f1584001670 con 0x7f15941601e0 2023-03-29T10:58:48.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.910+0000 7f158bfff700 1 -- 172.21.15.111:0/3293457523 >> v1:172.21.15.111:6789/0 conn(0x7f15941601e0 legacy=0x7f1594160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:48.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.910+0000 7f158bfff700 1 --2- 172.21.15.111:0/3293457523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1580003a00 0x7f1580003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:48.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.910+0000 7f158bfff700 1 --2- 172.21.15.111:0/3293457523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1580004850 0x7f1580004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:48.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.910+0000 7f158bfff700 1 --2- 172.21.15.111:0/3293457523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1580005200 0x7f158000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:48.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.910+0000 7f158bfff700 1 -- 172.21.15.111:0/3293457523 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15940d02d0 con 0x7f1580005200 2023-03-29T10:58:48.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.910+0000 7f158bfff700 1 -- 172.21.15.111:0/3293457523 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158000c060 con 0x7f1580003a00 2023-03-29T10:58:48.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.910+0000 7f158bfff700 1 -- 172.21.15.111:0/3293457523 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f158000c220 con 0x7f1580004850 2023-03-29T10:58:48.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.910+0000 7f1599f9c700 1 --2- 172.21.15.111:0/3293457523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1580005200 0x7f158000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:48.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.910+0000 7f159979b700 1 --2- 172.21.15.111:0/3293457523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1580003a00 0x7f1580003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:48.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.910+0000 7f1598f9a700 1 --2- 172.21.15.111:0/3293457523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1580004850 0x7f1580004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:48.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.910+0000 7f1599f9c700 1 -- 172.21.15.111:0/3293457523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1580004850 msgr2=0x7f1580004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:48.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.910+0000 7f1599f9c700 1 --2- 172.21.15.111:0/3293457523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1580004850 0x7f1580004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.910+0000 7f1599f9c700 1 -- 172.21.15.111:0/3293457523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1580003a00 msgr2=0x7f1580003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:48.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.910+0000 7f1599f9c700 1 --2- 172.21.15.111:0/3293457523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1580003a00 0x7f1580003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.910+0000 7f1599f9c700 1 -- 172.21.15.111:0/3293457523 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15800035b0 con 0x7f1580005200 2023-03-29T10:58:48.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.910+0000 7f1599f9c700 1 -- 172.21.15.111:0/3293457523 reap_dead start 2023-03-29T10:58:48.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.910+0000 7f1599f9c700 1 --2- 172.21.15.111:0/3293457523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1580005200 0x7f158000ba10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c290 tx=0x7f1580005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c314fe2e224ac60 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:48.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.911+0000 7f158bfff700 1 -- 172.21.15.111:0/3293457523 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c016070 con 0x7f1580005200 2023-03-29T10:58:48.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.911+0000 7f158bfff700 1 -- 172.21.15.111:0/3293457523 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158c00cc30 con 0x7f1580005200 2023-03-29T10:58:48.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.911+0000 7f158bfff700 1 -- 172.21.15.111:0/3293457523 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c001e70 con 0x7f1580005200 2023-03-29T10:58:48.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.911+0000 7f159b9ff700 1 -- 172.21.15.111:0/3293457523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1580005200 msgr2=0x7f158000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:48.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.911+0000 7f159b9ff700 1 --2- 172.21.15.111:0/3293457523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1580005200 0x7f158000ba10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f158c00c290 tx=0x7f1580005880 comp rx=0 tx=0).stop 2023-03-29T10:58:48.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.911+0000 7f159b9ff700 1 -- 172.21.15.111:0/3293457523 shutdown_connections 2023-03-29T10:58:48.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.911+0000 7f159b9ff700 1 --2- 172.21.15.111:0/3293457523 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1580005200 0x7f158000ba10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.911+0000 7f159b9ff700 1 -- 172.21.15.111:0/3293457523 >> 172.21.15.111:0/3293457523 conn(0x7f1594071b10 msgr2=0x7f1594071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:48.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.911+0000 7f159b9ff700 1 -- 172.21.15.111:0/3293457523 shutdown_connections 2023-03-29T10:58:48.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.911+0000 7f159b9ff700 1 -- 172.21.15.111:0/3293457523 wait complete. 2023-03-29T10:58:48.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.913+0000 7f159b9ff700 1 Processor -- start 2023-03-29T10:58:48.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.913+0000 7f159b9ff700 1 -- start start 2023-03-29T10:58:48.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.914+0000 7f159b9ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1580005200 0x7f159415b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:48.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.914+0000 7f159b9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f159415e6f0 0x7f159415eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:48.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.914+0000 7f159b9ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15940c8270 0x7f15940c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:48.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.914+0000 7f159b9ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15940c7520 con 0x7f15940c8270 2023-03-29T10:58:48.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.914+0000 7f159b9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159415f0a0 con 0x7f159415e6f0 2023-03-29T10:58:48.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.914+0000 7f159b9ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f159415b8f0 con 0x7f1580005200 2023-03-29T10:58:48.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.914+0000 7f1599f9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15940c8270 0x7f15940c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:48.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.914+0000 7f1599f9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15940c8270 0x7f15940c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51630/0 (socket says 172.21.15.111:51630) 2023-03-29T10:58:48.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.914+0000 7f1599f9c700 1 -- 172.21.15.111:0/2627653580 learned_addr learned my addr 172.21.15.111:0/2627653580 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:48.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.915+0000 7f1598f9a700 1 --2- 172.21.15.111:0/2627653580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f159415e6f0 0x7f159415eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:48.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.915+0000 7f159979b700 1 --2- 172.21.15.111:0/2627653580 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1580005200 0x7f159415b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:48.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.915+0000 7f1599f9c700 1 -- 172.21.15.111:0/2627653580 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1580005200 msgr2=0x7f159415b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:48.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.915+0000 7f1599f9c700 1 --2- 172.21.15.111:0/2627653580 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1580005200 0x7f159415b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.915+0000 7f1599f9c700 1 -- 172.21.15.111:0/2627653580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f159415e6f0 msgr2=0x7f159415eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:48.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.915+0000 7f1599f9c700 1 --2- 172.21.15.111:0/2627653580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f159415e6f0 0x7f159415eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:48.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.915+0000 7f1599f9c700 1 -- 172.21.15.111:0/2627653580 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15940cfd80 con 0x7f15940c8270 2023-03-29T10:58:48.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.915+0000 7f1599f9c700 1 --2- 172.21.15.111:0/2627653580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15940c8270 0x7f15940c86e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1594160140 tx=0x7f158c0015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:48.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.915+0000 7f1589ffb700 1 -- 172.21.15.111:0/2627653580 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c016040 con 0x7f15940c8270 2023-03-29T10:58:48.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.915+0000 7f1589ffb700 1 -- 172.21.15.111:0/2627653580 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f158c003e10 con 0x7f15940c8270 2023-03-29T10:58:48.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.915+0000 7f1589ffb700 1 -- 172.21.15.111:0/2627653580 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158c0052b0 con 0x7f15940c8270 2023-03-29T10:58:48.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.915+0000 7f159b9ff700 1 -- 172.21.15.111:0/2627653580 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15940cfbf0 con 0x7f15940c8270 2023-03-29T10:58:48.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.915+0000 7f159b9ff700 1 -- 172.21.15.111:0/2627653580 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15940cff10 con 0x7f15940c8270 2023-03-29T10:58:48.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.916+0000 7f159b9ff700 1 -- 172.21.15.111:0/2627653580 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15940c7660 con 0x7f15940c8270 2023-03-29T10:58:48.917 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.917+0000 7f1589ffb700 1 -- 172.21.15.111:0/2627653580 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f158c005410 con 0x7f15940c8270 2023-03-29T10:58:48.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.918+0000 7f1589ffb700 1 --2- 172.21.15.111:0/2627653580 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1578069de0 0x7f157806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:48.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.918+0000 7f1589ffb700 1 -- 172.21.15.111:0/2627653580 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f158c089170 con 0x7f15940c8270 2023-03-29T10:58:48.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.919+0000 7f159979b700 1 --2- 172.21.15.111:0/2627653580 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1578069de0 0x7f157806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:48.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.919+0000 7f159979b700 1 --2- 172.21.15.111:0/2627653580 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1578069de0 0x7f157806c290 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1590009be0 tx=0x7f1590009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:48.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:48.923+0000 7f1589ffb700 1 -- 172.21.15.111:0/2627653580 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f15940c7660 con 0x7f15940c8270 2023-03-29T10:58:49.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.113+0000 7f159b9ff700 1 -- 172.21.15.111:0/2627653580 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f15940c7660 con 0x7f15940c8270 2023-03-29T10:58:49.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.114+0000 7f1589ffb700 1 -- 172.21.15.111:0/2627653580 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f15940c7660 con 0x7f15940c8270 2023-03-29T10:58:49.113 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 8 2023-03-29T10:58:49.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.116+0000 7f159b9ff700 1 -- 172.21.15.111:0/2627653580 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1578069de0 msgr2=0x7f157806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:49.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.116+0000 7f159b9ff700 1 --2- 172.21.15.111:0/2627653580 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1578069de0 0x7f157806c290 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1590009be0 tx=0x7f1590009210 comp rx=0 tx=0).stop 2023-03-29T10:58:49.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.116+0000 7f159b9ff700 1 -- 172.21.15.111:0/2627653580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15940c8270 msgr2=0x7f15940c86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:49.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.116+0000 7f159b9ff700 1 --2- 172.21.15.111:0/2627653580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15940c8270 0x7f15940c86e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1594160140 tx=0x7f158c0015b0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.116+0000 7f159b9ff700 1 -- 172.21.15.111:0/2627653580 shutdown_connections 2023-03-29T10:58:49.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.116+0000 7f159b9ff700 1 --2- 172.21.15.111:0/2627653580 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1578069de0 0x7f157806c290 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.116+0000 7f159b9ff700 1 --2- 172.21.15.111:0/2627653580 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f15940c8270 0x7f15940c86e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.116+0000 7f159b9ff700 1 --2- 172.21.15.111:0/2627653580 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1580005200 0x7f159415b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.116+0000 7f159b9ff700 1 --2- 172.21.15.111:0/2627653580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f159415e6f0 0x7f159415eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.116+0000 7f159b9ff700 1 -- 172.21.15.111:0/2627653580 >> 172.21.15.111:0/2627653580 conn(0x7f1594071b10 msgr2=0x7f15940b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:49.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.116+0000 7f159b9ff700 1 -- 172.21.15.111:0/2627653580 shutdown_connections 2023-03-29T10:58:49.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.116+0000 7f159b9ff700 1 -- 172.21.15.111:0/2627653580 wait complete. 2023-03-29T10:58:49.133 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-29T10:58:49.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.319+0000 7f9e48dff700 1 Processor -- start 2023-03-29T10:58:49.319 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.319+0000 7f9e48dff700 1 -- start start 2023-03-29T10:58:49.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.320+0000 7f9e48dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9e440cd760 0x7f9e440cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:49.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.320+0000 7f9e48dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9e440adf70 0x7f9e440cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:49.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.320+0000 7f9e48dff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9e4415cf10 0x7f9e4406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:49.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.320+0000 7f9e48dff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e440cc9e0 con 0x7f9e4415cf10 2023-03-29T10:58:49.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.320+0000 7f9e48dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e4415de20 con 0x7f9e440cd760 2023-03-29T10:58:49.320 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.320+0000 7f9e48dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440ce070 con 0x7f9e440adf70 2023-03-29T10:58:49.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.321+0000 7f9e42d9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9e4415cf10 0x7f9e4406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:49.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.321+0000 7f9e42d9d700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9e4415cf10 0x7f9e4406c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51632/0 (socket says 172.21.15.111:51632) 2023-03-29T10:58:49.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.321+0000 7f9e42d9d700 1 -- 172.21.15.111:0/3384840548 learned_addr learned my addr 172.21.15.111:0/3384840548 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:49.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.321+0000 7f9e4259c700 1 --2- 172.21.15.111:0/3384840548 >> v2:172.21.15.119:3300/0 conn(0x7f9e440cd760 0x7f9e440cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:49.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.321+0000 7f9e3bfff700 1 --2- 172.21.15.111:0/3384840548 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9e440adf70 0x7f9e440cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:49.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.321+0000 7f9e42d9d700 1 -- 172.21.15.111:0/3384840548 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9e440adf70 msgr2=0x7f9e440cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:49.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.321+0000 7f9e42d9d700 1 --2- 172.21.15.111:0/3384840548 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9e440adf70 0x7f9e440cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.321+0000 7f9e42d9d700 1 -- 172.21.15.111:0/3384840548 >> v2:172.21.15.119:3300/0 conn(0x7f9e440cd760 msgr2=0x7f9e440cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:49.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.321+0000 7f9e42d9d700 1 --2- 172.21.15.111:0/3384840548 >> v2:172.21.15.119:3300/0 conn(0x7f9e440cd760 0x7f9e440cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.322 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.321+0000 7f9e42d9d700 1 -- 172.21.15.111:0/3384840548 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4415d860 con 0x7f9e4415cf10 2023-03-29T10:58:49.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.321+0000 7f9e42d9d700 1 --2- 172.21.15.111:0/3384840548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e4415cf10 0x7f9e4406c6c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9e440cc230 tx=0x7f9e3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42e13d8c97cdfcd server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:49.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.322+0000 7f9e41d9b700 1 -- 172.21.15.111:0/3384840548 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c00e070 con 0x7f9e4415cf10 2023-03-29T10:58:49.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.322+0000 7f9e41d9b700 1 -- 172.21.15.111:0/3384840548 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3c00b9c0 con 0x7f9e4415cf10 2023-03-29T10:58:49.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.322+0000 7f9e41d9b700 1 -- 172.21.15.111:0/3384840548 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c00bd70 con 0x7f9e4415cf10 2023-03-29T10:58:49.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.322+0000 7f9e48dff700 1 -- 172.21.15.111:0/3384840548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e4415cf10 msgr2=0x7f9e4406c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:49.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.322+0000 7f9e48dff700 1 --2- 172.21.15.111:0/3384840548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e4415cf10 0x7f9e4406c6c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9e440cc230 tx=0x7f9e3c00b330 comp rx=0 tx=0).stop 2023-03-29T10:58:49.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.322+0000 7f9e48dff700 1 -- 172.21.15.111:0/3384840548 shutdown_connections 2023-03-29T10:58:49.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.322+0000 7f9e48dff700 1 --2- 172.21.15.111:0/3384840548 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e4415cf10 0x7f9e4406c6c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.322+0000 7f9e48dff700 1 --2- 172.21.15.111:0/3384840548 >> v2:172.21.15.119:3300/0 conn(0x7f9e440cd760 0x7f9e440cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.322+0000 7f9e48dff700 1 --2- 172.21.15.111:0/3384840548 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9e440adf70 0x7f9e440cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.322+0000 7f9e48dff700 1 -- 172.21.15.111:0/3384840548 >> 172.21.15.111:0/3384840548 conn(0x7f9e440cf720 msgr2=0x7f9e440cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:49.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.322+0000 7f9e48dff700 1 -- 172.21.15.111:0/3384840548 shutdown_connections 2023-03-29T10:58:49.323 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.322+0000 7f9e48dff700 1 -- 172.21.15.111:0/3384840548 wait complete. 2023-03-29T10:58:49.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.324+0000 7f9e48dff700 1 Processor -- start 2023-03-29T10:58:49.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.324+0000 7f9e48dff700 1 -- start start 2023-03-29T10:58:49.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.325+0000 7f9e48dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e440adf70 0x7f9e440b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:49.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.325+0000 7f9e48dff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e440cd760 0x7f9e44157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:49.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.325+0000 7f9e48dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9e4415cf10 0x7f9e44157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:49.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.325+0000 7f9e48dff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440b47c0 con 0x7f9e440cd760 2023-03-29T10:58:49.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.325+0000 7f9e48dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440b27b0 con 0x7f9e440adf70 2023-03-29T10:58:49.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.325+0000 7f9e48dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440b18c0 con 0x7f9e4415cf10 2023-03-29T10:58:49.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.325+0000 7f9e3bfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e440cd760 0x7f9e44157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:49.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.325+0000 7f9e3bfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e440cd760 0x7f9e44157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51644/0 (socket says 172.21.15.111:51644) 2023-03-29T10:58:49.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.325+0000 7f9e3bfff700 1 -- 172.21.15.111:0/2663634046 learned_addr learned my addr 172.21.15.111:0/2663634046 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:49.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.325+0000 7f9e42d9d700 1 --2- 172.21.15.111:0/2663634046 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9e4415cf10 0x7f9e44157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:49.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.325+0000 7f9e4259c700 1 --2- 172.21.15.111:0/2663634046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e440adf70 0x7f9e440b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:49.326 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.326+0000 7f9e3bfff700 1 -- 172.21.15.111:0/2663634046 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9e4415cf10 msgr2=0x7f9e44157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:49.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.326+0000 7f9e3bfff700 1 --2- 172.21.15.111:0/2663634046 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9e4415cf10 0x7f9e44157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.326+0000 7f9e3bfff700 1 -- 172.21.15.111:0/2663634046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e440adf70 msgr2=0x7f9e440b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:49.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.326+0000 7f9e3bfff700 1 --2- 172.21.15.111:0/2663634046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e440adf70 0x7f9e440b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.326+0000 7f9e3bfff700 1 -- 172.21.15.111:0/2663634046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c009b30 con 0x7f9e440cd760 2023-03-29T10:58:49.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.326+0000 7f9e4259c700 1 --2- 172.21.15.111:0/2663634046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e440adf70 0x7f9e440b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:49.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.326+0000 7f9e3bfff700 1 --2- 172.21.15.111:0/2663634046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e440cd760 0x7f9e44157a70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c00e730 tx=0x7f9e2c00ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:49.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.326+0000 7f9e3affd700 1 -- 172.21.15.111:0/2663634046 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c013610 con 0x7f9e440cd760 2023-03-29T10:58:49.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.326+0000 7f9e3affd700 1 -- 172.21.15.111:0/2663634046 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2c013770 con 0x7f9e440cd760 2023-03-29T10:58:49.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.326+0000 7f9e48dff700 1 -- 172.21.15.111:0/2663634046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e440cc670 con 0x7f9e440cd760 2023-03-29T10:58:49.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.326+0000 7f9e3affd700 1 -- 172.21.15.111:0/2663634046 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c013b20 con 0x7f9e440cd760 2023-03-29T10:58:49.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.326+0000 7f9e48dff700 1 -- 172.21.15.111:0/2663634046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4415db40 con 0x7f9e440cd760 2023-03-29T10:58:49.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.327+0000 7f9e48dff700 1 -- 172.21.15.111:0/2663634046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e44071600 con 0x7f9e440cd760 2023-03-29T10:58:49.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.328+0000 7f9e3affd700 1 -- 172.21.15.111:0/2663634046 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f9e2c013c80 con 0x7f9e440cd760 2023-03-29T10:58:49.328 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.329+0000 7f9e3affd700 1 --2- 172.21.15.111:0/2663634046 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9e24069de0 0x7f9e2406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:49.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.329+0000 7f9e3affd700 1 -- 172.21.15.111:0/2663634046 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9e2c088100 con 0x7f9e440cd760 2023-03-29T10:58:49.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.329+0000 7f9e4259c700 1 --2- 172.21.15.111:0/2663634046 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9e24069de0 0x7f9e2406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:49.329 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.330+0000 7f9e4259c700 1 --2- 172.21.15.111:0/2663634046 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9e24069de0 0x7f9e2406c290 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400bb20 tx=0x7f9e34008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:49.333 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.333+0000 7f9e3affd700 1 -- 172.21.15.111:0/2663634046 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f9e2c057020 con 0x7f9e440cd760 2023-03-29T10:58:49.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.525+0000 7f9e48dff700 1 -- 172.21.15.111:0/2663634046 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e440b2a90 con 0x7f9e440cd760 2023-03-29T10:58:49.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.525+0000 7f9e3affd700 1 -- 172.21.15.111:0/2663634046 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e2c00f040 con 0x7f9e440cd760 2023-03-29T10:58:49.525 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:49.527 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.527+0000 7f9e48dff700 1 -- 172.21.15.111:0/2663634046 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9e24069de0 msgr2=0x7f9e2406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:49.527 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.527+0000 7f9e48dff700 1 --2- 172.21.15.111:0/2663634046 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9e24069de0 0x7f9e2406c290 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400bb20 tx=0x7f9e34008040 comp rx=0 tx=0).stop 2023-03-29T10:58:49.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.528+0000 7f9e48dff700 1 -- 172.21.15.111:0/2663634046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e440cd760 msgr2=0x7f9e44157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:49.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.528+0000 7f9e48dff700 1 --2- 172.21.15.111:0/2663634046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e440cd760 0x7f9e44157a70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c00e730 tx=0x7f9e2c00ea40 comp rx=0 tx=0).stop 2023-03-29T10:58:49.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.528+0000 7f9e48dff700 1 -- 172.21.15.111:0/2663634046 shutdown_connections 2023-03-29T10:58:49.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.528+0000 7f9e48dff700 1 --2- 172.21.15.111:0/2663634046 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9e24069de0 0x7f9e2406c290 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.528+0000 7f9e48dff700 1 --2- 172.21.15.111:0/2663634046 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9e4415cf10 0x7f9e44157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.528+0000 7f9e48dff700 1 --2- 172.21.15.111:0/2663634046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e440adf70 0x7f9e440b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.528+0000 7f9e48dff700 1 --2- 172.21.15.111:0/2663634046 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9e440cd760 0x7f9e44157a70 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.528+0000 7f9e48dff700 1 -- 172.21.15.111:0/2663634046 >> 172.21.15.111:0/2663634046 conn(0x7f9e440cf720 msgr2=0x7f9e44156940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:49.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.528+0000 7f9e48dff700 1 -- 172.21.15.111:0/2663634046 shutdown_connections 2023-03-29T10:58:49.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.528+0000 7f9e48dff700 1 -- 172.21.15.111:0/2663634046 wait complete. 2023-03-29T10:58:49.545 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-29T10:58:49.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.731+0000 7ff4929ff700 1 Processor -- start 2023-03-29T10:58:49.730 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.731+0000 7ff4929ff700 1 -- start start 2023-03-29T10:58:49.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.732+0000 7ff4929ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff48c0b87a0 0x7ff48c0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:49.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.732+0000 7ff4929ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff48c1601e0 0x7ff48c160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:49.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.732+0000 7ff4929ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff48c160b70 0x7ff48c0d2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:49.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.732+0000 7ff4929ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48c0d02d0 con 0x7ff48c160b70 2023-03-29T10:58:49.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.732+0000 7ff4929ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48c0d0cb0 con 0x7ff48c1601e0 2023-03-29T10:58:49.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.732+0000 7ff4929ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0b2730 con 0x7ff48c0b87a0 2023-03-29T10:58:49.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.732+0000 7ff490f9c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff48c160b70 0x7ff48c0d2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:49.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.732+0000 7ff490f9c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff48c160b70 0x7ff48c0d2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51654/0 (socket says 172.21.15.111:51654) 2023-03-29T10:58:49.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.732+0000 7ff490f9c700 1 -- 172.21.15.111:0/2845866984 learned_addr learned my addr 172.21.15.111:0/2845866984 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:49.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.732+0000 7ff490f9c700 1 -- 172.21.15.111:0/2845866984 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff48c0b87a0 msgr2=0x7ff48c0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:49.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.732+0000 7ff490f9c700 1 --2- 172.21.15.111:0/2845866984 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff48c0b87a0 0x7ff48c0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.733+0000 7ff490f9c700 1 -- 172.21.15.111:0/2845866984 >> v2:172.21.15.119:3300/0 conn(0x7ff48c1601e0 msgr2=0x7ff48c160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:49.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.733+0000 7ff483fff700 1 --2- 172.21.15.111:0/2845866984 >> v2:172.21.15.119:3300/0 conn(0x7ff48c1601e0 0x7ff48c160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:49.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.733+0000 7ff490f9c700 1 --2- 172.21.15.111:0/2845866984 >> v2:172.21.15.119:3300/0 conn(0x7ff48c1601e0 0x7ff48c160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.733+0000 7ff490f9c700 1 -- 172.21.15.111:0/2845866984 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c160ff0 con 0x7ff48c160b70 2023-03-29T10:58:49.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.733+0000 7ff483fff700 1 --2- 172.21.15.111:0/2845866984 >> v2:172.21.15.119:3300/0 conn(0x7ff48c1601e0 0x7ff48c160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:49.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.733+0000 7ff490f9c700 1 --2- 172.21.15.111:0/2845866984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff48c160b70 0x7ff48c0d2de0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff48c08be90 tx=0x7ff48400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac7279246d4e929 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:49.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.733+0000 7ff48b7fe700 1 -- 172.21.15.111:0/2845866984 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48400e070 con 0x7ff48c160b70 2023-03-29T10:58:49.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.733+0000 7ff48b7fe700 1 -- 172.21.15.111:0/2845866984 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48400b9c0 con 0x7ff48c160b70 2023-03-29T10:58:49.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.733+0000 7ff48b7fe700 1 -- 172.21.15.111:0/2845866984 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48400bd70 con 0x7ff48c160b70 2023-03-29T10:58:49.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.733+0000 7ff4929ff700 1 -- 172.21.15.111:0/2845866984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff48c160b70 msgr2=0x7ff48c0d2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:49.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.733+0000 7ff4929ff700 1 --2- 172.21.15.111:0/2845866984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff48c160b70 0x7ff48c0d2de0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff48c08be90 tx=0x7ff48400b330 comp rx=0 tx=0).stop 2023-03-29T10:58:49.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.734+0000 7ff4929ff700 1 -- 172.21.15.111:0/2845866984 shutdown_connections 2023-03-29T10:58:49.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.734+0000 7ff4929ff700 1 --2- 172.21.15.111:0/2845866984 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff48c0b87a0 0x7ff48c0cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.734+0000 7ff4929ff700 1 --2- 172.21.15.111:0/2845866984 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff48c160b70 0x7ff48c0d2de0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.734+0000 7ff4929ff700 1 --2- 172.21.15.111:0/2845866984 >> v2:172.21.15.119:3300/0 conn(0x7ff48c1601e0 0x7ff48c160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.734+0000 7ff4929ff700 1 -- 172.21.15.111:0/2845866984 >> 172.21.15.111:0/2845866984 conn(0x7ff48c071b10 msgr2=0x7ff48c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:49.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.734+0000 7ff4929ff700 1 -- 172.21.15.111:0/2845866984 shutdown_connections 2023-03-29T10:58:49.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.734+0000 7ff4929ff700 1 -- 172.21.15.111:0/2845866984 wait complete. 2023-03-29T10:58:49.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.735+0000 7ff4929ff700 1 Processor -- start 2023-03-29T10:58:49.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.735+0000 7ff4929ff700 1 -- start start 2023-03-29T10:58:49.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.736+0000 7ff4929ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff48c0b87a0 0x7ff48c1f6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:49.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.736+0000 7ff4929ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff48c1601e0 0x7ff48c15b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:49.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.736+0000 7ff4929ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff48c160b70 0x7ff48c158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:49.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.736+0000 7ff4929ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0cae30 con 0x7ff48c160b70 2023-03-29T10:58:49.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.736+0000 7ff4929ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0c9f80 con 0x7ff48c0b87a0 2023-03-29T10:58:49.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.736+0000 7ff4929ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c1f6650 con 0x7ff48c1601e0 2023-03-29T10:58:49.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.737+0000 7ff490f9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff48c160b70 0x7ff48c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:49.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.737+0000 7ff490f9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff48c160b70 0x7ff48c158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51656/0 (socket says 172.21.15.111:51656) 2023-03-29T10:58:49.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.737+0000 7ff490f9c700 1 -- 172.21.15.111:0/3990705009 learned_addr learned my addr 172.21.15.111:0/3990705009 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:49.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.737+0000 7ff48bfff700 1 --2- 172.21.15.111:0/3990705009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff48c0b87a0 0x7ff48c1f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:49.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.737+0000 7ff483fff700 1 --2- 172.21.15.111:0/3990705009 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff48c1601e0 0x7ff48c15b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:49.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.737+0000 7ff490f9c700 1 -- 172.21.15.111:0/3990705009 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff48c1601e0 msgr2=0x7ff48c15b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:49.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.737+0000 7ff490f9c700 1 --2- 172.21.15.111:0/3990705009 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff48c1601e0 0x7ff48c15b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.737+0000 7ff490f9c700 1 -- 172.21.15.111:0/3990705009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff48c0b87a0 msgr2=0x7ff48c1f6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:49.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.737+0000 7ff490f9c700 1 --2- 172.21.15.111:0/3990705009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff48c0b87a0 0x7ff48c1f6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.737+0000 7ff490f9c700 1 -- 172.21.15.111:0/3990705009 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff484009b30 con 0x7ff48c160b70 2023-03-29T10:58:49.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.737+0000 7ff483fff700 1 --2- 172.21.15.111:0/3990705009 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff48c1601e0 0x7ff48c15b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:49.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.737+0000 7ff490f9c700 1 --2- 172.21.15.111:0/3990705009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff48c160b70 0x7ff48c158870 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff48400b370 tx=0x7ff484004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:49.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.738+0000 7ff4897fa700 1 -- 172.21.15.111:0/3990705009 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484004310 con 0x7ff48c160b70 2023-03-29T10:58:49.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.738+0000 7ff4897fa700 1 -- 172.21.15.111:0/3990705009 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff484004470 con 0x7ff48c160b70 2023-03-29T10:58:49.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.738+0000 7ff4897fa700 1 -- 172.21.15.111:0/3990705009 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484004820 con 0x7ff48c160b70 2023-03-29T10:58:49.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.738+0000 7ff4929ff700 1 -- 172.21.15.111:0/3990705009 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48c0cd6a0 con 0x7ff48c160b70 2023-03-29T10:58:49.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.738+0000 7ff4929ff700 1 -- 172.21.15.111:0/3990705009 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48c0cdc60 con 0x7ff48c160b70 2023-03-29T10:58:49.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.739+0000 7ff4827fc700 1 -- 172.21.15.111:0/3990705009 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48c0cd980 con 0x7ff48c160b70 2023-03-29T10:58:49.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.740+0000 7ff4897fa700 1 -- 172.21.15.111:0/3990705009 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7ff48401c440 con 0x7ff48c160b70 2023-03-29T10:58:49.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.741+0000 7ff4897fa700 1 --2- 172.21.15.111:0/3990705009 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff46c069de0 0x7ff46c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:49.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.741+0000 7ff4897fa700 1 -- 172.21.15.111:0/3990705009 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff484089990 con 0x7ff48c160b70 2023-03-29T10:58:49.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.741+0000 7ff48bfff700 1 --2- 172.21.15.111:0/3990705009 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff46c069de0 0x7ff46c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:49.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.742+0000 7ff48bfff700 1 --2- 172.21.15.111:0/3990705009 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff46c069de0 0x7ff46c06c290 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff47c006fd0 tx=0x7ff47c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:49.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.745+0000 7ff4897fa700 1 -- 172.21.15.111:0/3990705009 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7ff484054ea0 con 0x7ff48c160b70 2023-03-29T10:58:49.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.936+0000 7ff4827fc700 1 -- 172.21.15.111:0/3990705009 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff48c074430 con 0x7ff48c160b70 2023-03-29T10:58:49.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.937+0000 7ff4897fa700 1 -- 172.21.15.111:0/3990705009 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff484020020 con 0x7ff48c160b70 2023-03-29T10:58:49.936 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-03-29T10:58:49.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.939+0000 7ff4827fc700 1 -- 172.21.15.111:0/3990705009 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff46c069de0 msgr2=0x7ff46c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:49.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.939+0000 7ff4827fc700 1 --2- 172.21.15.111:0/3990705009 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff46c069de0 0x7ff46c06c290 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff47c006fd0 tx=0x7ff47c008040 comp rx=0 tx=0).stop 2023-03-29T10:58:49.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.939+0000 7ff4827fc700 1 -- 172.21.15.111:0/3990705009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff48c160b70 msgr2=0x7ff48c158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:49.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.939+0000 7ff4827fc700 1 --2- 172.21.15.111:0/3990705009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff48c160b70 0x7ff48c158870 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff48400b370 tx=0x7ff484004ad0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.939+0000 7ff4827fc700 1 -- 172.21.15.111:0/3990705009 shutdown_connections 2023-03-29T10:58:49.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.939+0000 7ff4827fc700 1 --2- 172.21.15.111:0/3990705009 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff46c069de0 0x7ff46c06c290 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.939+0000 7ff4827fc700 1 --2- 172.21.15.111:0/3990705009 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff48c160b70 0x7ff48c158870 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.939+0000 7ff4827fc700 1 --2- 172.21.15.111:0/3990705009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff48c0b87a0 0x7ff48c1f6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.939+0000 7ff4827fc700 1 --2- 172.21.15.111:0/3990705009 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff48c1601e0 0x7ff48c15b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:49.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.939+0000 7ff4827fc700 1 -- 172.21.15.111:0/3990705009 >> 172.21.15.111:0/3990705009 conn(0x7ff48c071b10 msgr2=0x7ff48c0b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:49.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.939+0000 7ff4827fc700 1 -- 172.21.15.111:0/3990705009 shutdown_connections 2023-03-29T10:58:49.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:49.939+0000 7ff4827fc700 1 -- 172.21.15.111:0/3990705009 wait complete. 2023-03-29T10:58:49.956 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-29T10:58:50.141 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.141+0000 7f60df9ff700 1 Processor -- start 2023-03-29T10:58:50.141 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.141+0000 7f60df9ff700 1 -- start start 2023-03-29T10:58:50.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.142+0000 7f60df9ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60d80cfbf0 0x7f60d81608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:50.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.142+0000 7f60df9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f60d80d1100 0x7f60d80d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:50.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.142+0000 7f60df9ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60d80d0020 con 0x7f60d80b87a0 2023-03-29T10:58:50.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.142+0000 7f60df9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60d80d02d0 con 0x7f60d80d1100 2023-03-29T10:58:50.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.142+0000 7f60df9ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d80d0cb0 con 0x7f60d80cfbf0 2023-03-29T10:58:50.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.143+0000 7f60dcf9a700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60d80cfbf0 0x7f60d81608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:50.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.143+0000 7f60dcf9a700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60d80cfbf0 0x7f60d81608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3301/0 says I am v2:172.21.15.111:37416/0 (socket says 172.21.15.111:37416) 2023-03-29T10:58:50.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.143+0000 7f60dcf9a700 1 -- 172.21.15.111:0/3361571719 learned_addr learned my addr 172.21.15.111:0/3361571719 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:50.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.143+0000 7f60ddf9c700 1 --2- 172.21.15.111:0/3361571719 >> v2:172.21.15.119:3300/0 conn(0x7f60d80d1100 0x7f60d80d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:50.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.143+0000 7f60dcf9a700 1 -- 172.21.15.111:0/3361571719 >> v2:172.21.15.119:3300/0 conn(0x7f60d80d1100 msgr2=0x7f60d80d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:50.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.143+0000 7f60dcf9a700 1 --2- 172.21.15.111:0/3361571719 >> v2:172.21.15.119:3300/0 conn(0x7f60d80d1100 0x7f60d80d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.143+0000 7f60dcf9a700 1 -- 172.21.15.111:0/3361571719 >> v1:172.21.15.119:6789/0 conn(0x7f60d80b87a0 legacy=0x7f60d81601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:50.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.143+0000 7f60dcf9a700 1 -- 172.21.15.111:0/3361571719 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d80cdc60 con 0x7f60d80cfbf0 2023-03-29T10:58:50.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.144+0000 7f60dcf9a700 1 --2- 172.21.15.111:0/3361571719 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60d80cfbf0 0x7f60d81608f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f60c80098f0 tx=0x7f60c8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=decfec09cde6cdee server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:50.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.144+0000 7f60cffff700 1 -- 172.21.15.111:0/3361571719 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c8011830 con 0x7f60d80cfbf0 2023-03-29T10:58:50.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.144+0000 7f60cffff700 1 -- 172.21.15.111:0/3361571719 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c8011990 con 0x7f60d80cfbf0 2023-03-29T10:58:50.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.144+0000 7f60cffff700 1 -- 172.21.15.111:0/3361571719 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60c8011d20 con 0x7f60d80cfbf0 2023-03-29T10:58:50.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.144+0000 7f60df9ff700 1 -- 172.21.15.111:0/3361571719 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60d80cfbf0 msgr2=0x7f60d81608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:50.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.144+0000 7f60df9ff700 1 --2- 172.21.15.111:0/3361571719 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60d80cfbf0 0x7f60d81608f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f60c80098f0 tx=0x7f60c8009c00 comp rx=0 tx=0).stop 2023-03-29T10:58:50.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.145+0000 7f60df9ff700 1 -- 172.21.15.111:0/3361571719 shutdown_connections 2023-03-29T10:58:50.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.145+0000 7f60df9ff700 1 --2- 172.21.15.111:0/3361571719 >> v2:172.21.15.119:3300/0 conn(0x7f60d80d1100 0x7f60d80d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.145+0000 7f60df9ff700 1 --2- 172.21.15.111:0/3361571719 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60d80cfbf0 0x7f60d81608f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.145+0000 7f60df9ff700 1 -- 172.21.15.111:0/3361571719 >> 172.21.15.111:0/3361571719 conn(0x7f60d8071b10 msgr2=0x7f60d8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:50.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.145+0000 7f60df9ff700 1 -- 172.21.15.111:0/3361571719 shutdown_connections 2023-03-29T10:58:50.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.145+0000 7f60df9ff700 1 -- 172.21.15.111:0/3361571719 wait complete. 2023-03-29T10:58:50.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.146+0000 7f60df9ff700 1 Processor -- start 2023-03-29T10:58:50.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.146+0000 7f60df9ff700 1 -- start start 2023-03-29T10:58:50.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.147+0000 7f60df9ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60d80b87a0 0x7f60d81541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:50.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.147+0000 7f60df9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60d80cfbf0 0x7f60d81e7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:50.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.147+0000 7f60df9ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60d80d1100 0x7f60d80d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:50.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.147+0000 7f60df9ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d8154d80 con 0x7f60d80d1100 2023-03-29T10:58:50.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.147+0000 7f60df9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d815d660 con 0x7f60d80cfbf0 2023-03-29T10:58:50.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.147+0000 7f60df9ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60d815d900 con 0x7f60d80b87a0 2023-03-29T10:58:50.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.148+0000 7f60ddf9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60d80d1100 0x7f60d80d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:50.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.148+0000 7f60ddf9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60d80d1100 0x7f60d80d2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51662/0 (socket says 172.21.15.111:51662) 2023-03-29T10:58:50.148 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.148+0000 7f60ddf9c700 1 -- 172.21.15.111:0/3232066310 learned_addr learned my addr 172.21.15.111:0/3232066310 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:50.148 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.148+0000 7f60dd79b700 1 --2- 172.21.15.111:0/3232066310 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60d80b87a0 0x7f60d81541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:50.148 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.148+0000 7f60dcf9a700 1 --2- 172.21.15.111:0/3232066310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60d80cfbf0 0x7f60d81e7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:50.148 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.148+0000 7f60ddf9c700 1 -- 172.21.15.111:0/3232066310 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60d80b87a0 msgr2=0x7f60d81541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:50.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.148+0000 7f60ddf9c700 1 --2- 172.21.15.111:0/3232066310 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60d80b87a0 0x7f60d81541a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.148+0000 7f60ddf9c700 1 -- 172.21.15.111:0/3232066310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60d80cfbf0 msgr2=0x7f60d81e7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:50.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.148+0000 7f60ddf9c700 1 --2- 172.21.15.111:0/3232066310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60d80cfbf0 0x7f60d81e7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.148+0000 7f60ddf9c700 1 -- 172.21.15.111:0/3232066310 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c8009550 con 0x7f60d80d1100 2023-03-29T10:58:50.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.148+0000 7f60ddf9c700 1 --2- 172.21.15.111:0/3232066310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60d80d1100 0x7f60d80d2ae0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f60d000efc0 tx=0x7f60d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:50.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.148+0000 7f60cdffb700 1 -- 172.21.15.111:0/3232066310 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d0009ea0 con 0x7f60d80d1100 2023-03-29T10:58:50.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.149+0000 7f60cdffb700 1 -- 172.21.15.111:0/3232066310 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d000f040 con 0x7f60d80d1100 2023-03-29T10:58:50.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.149+0000 7f60cdffb700 1 -- 172.21.15.111:0/3232066310 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d000ced0 con 0x7f60d80d1100 2023-03-29T10:58:50.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.149+0000 7f60df9ff700 1 -- 172.21.15.111:0/3232066310 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d80cd980 con 0x7f60d80d1100 2023-03-29T10:58:50.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.149+0000 7f60df9ff700 1 -- 172.21.15.111:0/3232066310 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d80cd6a0 con 0x7f60d80d1100 2023-03-29T10:58:50.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.150+0000 7f60df9ff700 1 -- 172.21.15.111:0/3232066310 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d8074430 con 0x7f60d80d1100 2023-03-29T10:58:50.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.151+0000 7f60cdffb700 1 -- 172.21.15.111:0/3232066310 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f60d0014480 con 0x7f60d80d1100 2023-03-29T10:58:50.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.152+0000 7f60cdffb700 1 --2- 172.21.15.111:0/3232066310 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f60bc069de0 0x7f60bc06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:50.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.152+0000 7f60cdffb700 1 -- 172.21.15.111:0/3232066310 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f60d0088d50 con 0x7f60d80d1100 2023-03-29T10:58:50.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.152+0000 7f60dd79b700 1 --2- 172.21.15.111:0/3232066310 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f60bc069de0 0x7f60bc06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:50.152 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.153+0000 7f60dd79b700 1 --2- 172.21.15.111:0/3232066310 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f60bc069de0 0x7f60bc06c290 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f60d4007fd0 tx=0x7f60d4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:50.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.156+0000 7f60cdffb700 1 -- 172.21.15.111:0/3232066310 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f60d008c030 con 0x7f60d80d1100 2023-03-29T10:58:50.348 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.349+0000 7f60df9ff700 1 -- 172.21.15.111:0/3232066310 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f60d80b52f0 con 0x7f60d80d1100 2023-03-29T10:58:50.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.349+0000 7f60cdffb700 1 -- 172.21.15.111:0/3232066310 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f60d0057d90 con 0x7f60d80d1100 2023-03-29T10:58:50.349 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.352+0000 7f60df9ff700 1 -- 172.21.15.111:0/3232066310 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f60bc069de0 msgr2=0x7f60bc06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.352+0000 7f60df9ff700 1 --2- 172.21.15.111:0/3232066310 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f60bc069de0 0x7f60bc06c290 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f60d4007fd0 tx=0x7f60d4009040 comp rx=0 tx=0).stop 2023-03-29T10:58:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.352+0000 7f60df9ff700 1 -- 172.21.15.111:0/3232066310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60d80d1100 msgr2=0x7f60d80d2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.352+0000 7f60df9ff700 1 --2- 172.21.15.111:0/3232066310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60d80d1100 0x7f60d80d2ae0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f60d000efc0 tx=0x7f60d000c3a0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.352+0000 7f60df9ff700 1 -- 172.21.15.111:0/3232066310 shutdown_connections 2023-03-29T10:58:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.352+0000 7f60df9ff700 1 --2- 172.21.15.111:0/3232066310 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f60bc069de0 0x7f60bc06c290 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.352+0000 7f60df9ff700 1 --2- 172.21.15.111:0/3232066310 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60d80d1100 0x7f60d80d2ae0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.352+0000 7f60df9ff700 1 --2- 172.21.15.111:0/3232066310 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60d80b87a0 0x7f60d81541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.352+0000 7f60df9ff700 1 --2- 172.21.15.111:0/3232066310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60d80cfbf0 0x7f60d81e7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.352+0000 7f60df9ff700 1 -- 172.21.15.111:0/3232066310 >> 172.21.15.111:0/3232066310 conn(0x7f60d8071b10 msgr2=0x7f60d80b3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.352+0000 7f60df9ff700 1 -- 172.21.15.111:0/3232066310 shutdown_connections 2023-03-29T10:58:50.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.352+0000 7f60df9ff700 1 -- 172.21.15.111:0/3232066310 wait complete. 2023-03-29T10:58:50.369 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-03-29T10:58:50.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.552+0000 7eff737ff700 1 Processor -- start 2023-03-29T10:58:50.552 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.553+0000 7eff737ff700 1 -- start start 2023-03-29T10:58:50.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.554+0000 7eff737ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7eff6c0cff50 0x7eff6c160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:50.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.554+0000 7eff737ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff6c0cd520 con 0x7eff6c0cf010 2023-03-29T10:58:50.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.554+0000 7eff737ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff6c0d2640 con 0x7eff6c15c590 2023-03-29T10:58:50.553 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.554+0000 7eff737ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff6c1610d0 con 0x7eff6c0cff50 2023-03-29T10:58:50.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.554+0000 7eff70d9a700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7eff6c0cf010 0x7eff6c0d2ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47372/0 (socket says 172.21.15.111:47372) 2023-03-29T10:58:50.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.554+0000 7eff70d9a700 1 -- 172.21.15.111:0/312489590 learned_addr learned my addr 172.21.15.111:0/312489590 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:50.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.554+0000 7eff71d9c700 1 --2- 172.21.15.111:0/312489590 >> v2:172.21.15.119:3300/0 conn(0x7eff6c0cff50 0x7eff6c160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:50.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.554+0000 7eff63fff700 1 -- 172.21.15.111:0/312489590 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 48159590 0 0) 0x7eff6c0cd520 con 0x7eff6c0cf010 2023-03-29T10:58:50.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.554+0000 7eff63fff700 1 -- 172.21.15.111:0/312489590 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff580035b0 con 0x7eff6c0cf010 2023-03-29T10:58:50.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.555+0000 7eff63fff700 1 -- 172.21.15.111:0/312489590 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3428260130 0 0) 0x7eff580035b0 con 0x7eff6c0cf010 2023-03-29T10:58:50.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.555+0000 7eff63fff700 1 -- 172.21.15.111:0/312489590 >> v2:172.21.15.119:3300/0 conn(0x7eff6c0cff50 msgr2=0x7eff6c160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:50.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.555+0000 7eff63fff700 1 --2- 172.21.15.111:0/312489590 >> v2:172.21.15.119:3300/0 conn(0x7eff6c0cff50 0x7eff6c160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.555+0000 7eff63fff700 1 -- 172.21.15.111:0/312489590 >> v1:172.21.15.119:6789/0 conn(0x7eff6c15c590 legacy=0x7eff6c0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:50.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.555+0000 7eff63fff700 1 -- 172.21.15.111:0/312489590 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6c0b23c0 con 0x7eff6c0cf010 2023-03-29T10:58:50.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.555+0000 7eff63fff700 1 -- 172.21.15.111:0/312489590 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7eff5c001670 con 0x7eff6c0cf010 2023-03-29T10:58:50.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.555+0000 7eff63fff700 1 -- 172.21.15.111:0/312489590 >> v1:172.21.15.111:6789/0 conn(0x7eff6c0cf010 legacy=0x7eff6c0d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:50.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.555+0000 7eff63fff700 1 --2- 172.21.15.111:0/312489590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff58003a00 0x7eff58003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:50.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.555+0000 7eff63fff700 1 --2- 172.21.15.111:0/312489590 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff58004850 0x7eff58004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:50.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.555+0000 7eff63fff700 1 --2- 172.21.15.111:0/312489590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff58005200 0x7eff5800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:50.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.555+0000 7eff63fff700 1 -- 172.21.15.111:0/312489590 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c1610d0 con 0x7eff58005200 2023-03-29T10:58:50.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.555+0000 7eff63fff700 1 -- 172.21.15.111:0/312489590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5800c060 con 0x7eff58003a00 2023-03-29T10:58:50.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.555+0000 7eff63fff700 1 -- 172.21.15.111:0/312489590 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5800c220 con 0x7eff58004850 2023-03-29T10:58:50.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.555+0000 7eff7159b700 1 --2- 172.21.15.111:0/312489590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff58003a00 0x7eff58003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:50.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.555+0000 7eff71d9c700 1 --2- 172.21.15.111:0/312489590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff58005200 0x7eff5800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:50.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.555+0000 7eff70d9a700 1 --2- 172.21.15.111:0/312489590 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff58004850 0x7eff58004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:50.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.556+0000 7eff71d9c700 1 -- 172.21.15.111:0/312489590 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff58004850 msgr2=0x7eff58004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:50.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.556+0000 7eff71d9c700 1 --2- 172.21.15.111:0/312489590 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff58004850 0x7eff58004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.556+0000 7eff71d9c700 1 -- 172.21.15.111:0/312489590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff58003a00 msgr2=0x7eff58003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:50.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.556+0000 7eff71d9c700 1 --2- 172.21.15.111:0/312489590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff58003a00 0x7eff58003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.556+0000 7eff71d9c700 1 -- 172.21.15.111:0/312489590 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff580035b0 con 0x7eff58005200 2023-03-29T10:58:50.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.556+0000 7eff71d9c700 1 -- 172.21.15.111:0/312489590 reap_dead start 2023-03-29T10:58:50.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.556+0000 7eff71d9c700 1 --2- 172.21.15.111:0/312489590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff58005200 0x7eff5800ba10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7eff64009fc0 tx=0x7eff58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11eaaabbc333e260 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:50.557 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.556+0000 7eff63fff700 1 -- 172.21.15.111:0/312489590 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff64003a20 con 0x7eff58005200 2023-03-29T10:58:50.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.556+0000 7eff63fff700 1 -- 172.21.15.111:0/312489590 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6400f040 con 0x7eff58005200 2023-03-29T10:58:50.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.556+0000 7eff63fff700 1 -- 172.21.15.111:0/312489590 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff64003de0 con 0x7eff58005200 2023-03-29T10:58:50.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.557+0000 7eff737ff700 1 -- 172.21.15.111:0/312489590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff58005200 msgr2=0x7eff5800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:50.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.557+0000 7eff737ff700 1 --2- 172.21.15.111:0/312489590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff58005200 0x7eff5800ba10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7eff64009fc0 tx=0x7eff58005880 comp rx=0 tx=0).stop 2023-03-29T10:58:50.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.557+0000 7eff737ff700 1 -- 172.21.15.111:0/312489590 shutdown_connections 2023-03-29T10:58:50.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.557+0000 7eff737ff700 1 --2- 172.21.15.111:0/312489590 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff58005200 0x7eff5800ba10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.557+0000 7eff737ff700 1 -- 172.21.15.111:0/312489590 >> 172.21.15.111:0/312489590 conn(0x7eff6c071b10 msgr2=0x7eff6c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:50.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.557+0000 7eff737ff700 1 -- 172.21.15.111:0/312489590 shutdown_connections 2023-03-29T10:58:50.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.557+0000 7eff737ff700 1 -- 172.21.15.111:0/312489590 wait complete. 2023-03-29T10:58:50.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.559+0000 7eff737ff700 1 Processor -- start 2023-03-29T10:58:50.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.559+0000 7eff737ff700 1 -- start start 2023-03-29T10:58:50.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.560+0000 7eff737ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff58005200 0x7eff6c0cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:50.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.560+0000 7eff737ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff6c0c7cf0 0x7eff6c0c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:50.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.560+0000 7eff737ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff6c155200 0x7eff6c155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:50.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.560+0000 7eff737ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c155bb0 con 0x7eff58005200 2023-03-29T10:58:50.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.560+0000 7eff737ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c0c8770 con 0x7eff6c0c7cf0 2023-03-29T10:58:50.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.560+0000 7eff737ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6c0c8a50 con 0x7eff6c155200 2023-03-29T10:58:50.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.560+0000 7eff7159b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff58005200 0x7eff6c0cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:50.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.560+0000 7eff7159b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff58005200 0x7eff6c0cb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51678/0 (socket says 172.21.15.111:51678) 2023-03-29T10:58:50.560 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.560+0000 7eff7159b700 1 -- 172.21.15.111:0/3342595296 learned_addr learned my addr 172.21.15.111:0/3342595296 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:50.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.560+0000 7eff70d9a700 1 --2- 172.21.15.111:0/3342595296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff6c0c7cf0 0x7eff6c0c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:50.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.560+0000 7eff7159b700 1 -- 172.21.15.111:0/3342595296 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff6c155200 msgr2=0x7eff6c155670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:50.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.560+0000 7eff7159b700 1 --2- 172.21.15.111:0/3342595296 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff6c155200 0x7eff6c155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.560+0000 7eff7159b700 1 -- 172.21.15.111:0/3342595296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff6c0c7cf0 msgr2=0x7eff6c0c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:50.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.560+0000 7eff7159b700 1 --2- 172.21.15.111:0/3342595296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff6c0c7cf0 0x7eff6c0c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.560+0000 7eff7159b700 1 -- 172.21.15.111:0/3342595296 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff6c0cd770 con 0x7eff58005200 2023-03-29T10:58:50.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.560+0000 7eff70d9a700 1 --2- 172.21.15.111:0/3342595296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff6c0c7cf0 0x7eff6c0c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:50.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.560+0000 7eff7159b700 1 --2- 172.21.15.111:0/3342595296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff58005200 0x7eff6c0cb920 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7eff6800f7c0 tx=0x7eff6800fb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:50.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.561+0000 7eff61ffb700 1 -- 172.21.15.111:0/3342595296 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff680052e0 con 0x7eff58005200 2023-03-29T10:58:50.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.561+0000 7eff61ffb700 1 -- 172.21.15.111:0/3342595296 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff68005460 con 0x7eff58005200 2023-03-29T10:58:50.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.561+0000 7eff61ffb700 1 -- 172.21.15.111:0/3342595296 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff680038c0 con 0x7eff58005200 2023-03-29T10:58:50.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.561+0000 7eff737ff700 1 -- 172.21.15.111:0/3342595296 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff6c0cf560 con 0x7eff58005200 2023-03-29T10:58:50.561 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.561+0000 7eff737ff700 1 -- 172.21.15.111:0/3342595296 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff6c15c960 con 0x7eff58005200 2023-03-29T10:58:50.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.562+0000 7eff737ff700 1 -- 172.21.15.111:0/3342595296 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff6c0ce910 con 0x7eff58005200 2023-03-29T10:58:50.562 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.563+0000 7eff61ffb700 1 -- 172.21.15.111:0/3342595296 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7eff68003a20 con 0x7eff58005200 2023-03-29T10:58:50.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.563+0000 7eff61ffb700 1 --2- 172.21.15.111:0/3342595296 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7eff50069de0 0x7eff5006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:50.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.564+0000 7eff61ffb700 1 -- 172.21.15.111:0/3342595296 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7eff68088250 con 0x7eff58005200 2023-03-29T10:58:50.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.564+0000 7eff70d9a700 1 --2- 172.21.15.111:0/3342595296 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7eff50069de0 0x7eff5006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:50.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.564+0000 7eff70d9a700 1 --2- 172.21.15.111:0/3342595296 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7eff50069de0 0x7eff5006c290 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7eff6c0c7bb0 tx=0x7eff5c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:50.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.568+0000 7eff61ffb700 1 -- 172.21.15.111:0/3342595296 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7eff68053740 con 0x7eff58005200 2023-03-29T10:58:50.778 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.779+0000 7eff737ff700 1 -- 172.21.15.111:0/3342595296 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7eff6c0ccdb0 con 0x7eff58005200 2023-03-29T10:58:50.779 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.779+0000 7eff61ffb700 1 -- 172.21.15.111:0/3342595296 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7eff68056d60 con 0x7eff58005200 2023-03-29T10:58:50.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.781+0000 7eff737ff700 1 -- 172.21.15.111:0/3342595296 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7eff50069de0 msgr2=0x7eff5006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:50.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.781+0000 7eff737ff700 1 --2- 172.21.15.111:0/3342595296 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7eff50069de0 0x7eff5006c290 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7eff6c0c7bb0 tx=0x7eff5c009040 comp rx=0 tx=0).stop 2023-03-29T10:58:50.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.781+0000 7eff737ff700 1 -- 172.21.15.111:0/3342595296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff58005200 msgr2=0x7eff6c0cb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:50.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.781+0000 7eff737ff700 1 --2- 172.21.15.111:0/3342595296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff58005200 0x7eff6c0cb920 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7eff6800f7c0 tx=0x7eff6800fb80 comp rx=0 tx=0).stop 2023-03-29T10:58:50.781 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.781+0000 7eff737ff700 1 -- 172.21.15.111:0/3342595296 shutdown_connections 2023-03-29T10:58:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.781+0000 7eff737ff700 1 --2- 172.21.15.111:0/3342595296 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7eff50069de0 0x7eff5006c290 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.781+0000 7eff737ff700 1 --2- 172.21.15.111:0/3342595296 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7eff58005200 0x7eff6c0cb920 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.781+0000 7eff737ff700 1 --2- 172.21.15.111:0/3342595296 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7eff6c155200 0x7eff6c155670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.781+0000 7eff737ff700 1 --2- 172.21.15.111:0/3342595296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff6c0c7cf0 0x7eff6c0c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.782+0000 7eff737ff700 1 -- 172.21.15.111:0/3342595296 >> 172.21.15.111:0/3342595296 conn(0x7eff6c071b10 msgr2=0x7eff6c0d13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.782+0000 7eff737ff700 1 -- 172.21.15.111:0/3342595296 shutdown_connections 2023-03-29T10:58:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:50.782+0000 7eff737ff700 1 -- 172.21.15.111:0/3342595296 wait complete. 2023-03-29T10:58:50.782 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-03-29T10:58:50.798 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T10:58:50.798 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-29T10:58:50.844 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:58:51.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.028+0000 7f92715ff700 1 Processor -- start 2023-03-29T10:58:51.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.028+0000 7f92715ff700 1 -- start start 2023-03-29T10:58:51.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.029+0000 7f92715ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f926c15c950 0x7f926c0cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:51.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.029+0000 7f92715ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f926c0d3a30 0x7f926c0b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:51.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.029+0000 7f92715ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926c0cdc80 con 0x7f926c0cf950 2023-03-29T10:58:51.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.029+0000 7f92715ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926c0cd6c0 con 0x7f926c15c950 2023-03-29T10:58:51.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.029+0000 7f92715ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0cd9a0 con 0x7f926c0d3a30 2023-03-29T10:58:51.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.029+0000 7f926a7fc700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f926c15c950 0x7f926c0cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:51.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.029+0000 7f926affd700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f926c0cf950 0x7f926c0cfd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47376/0 (socket says 172.21.15.111:47376) 2023-03-29T10:58:51.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.029+0000 7f926affd700 1 -- 172.21.15.111:0/288132038 learned_addr learned my addr 172.21.15.111:0/288132038 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:51.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.030+0000 7f926b7fe700 1 --2- 172.21.15.111:0/288132038 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f926c0d3a30 0x7f926c0b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:51.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.030+0000 7f9269ffb700 1 -- 172.21.15.111:0/288132038 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594441768 0 0) 0x7f926c0cdc80 con 0x7f926c0cf950 2023-03-29T10:58:51.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.030+0000 7f9269ffb700 1 -- 172.21.15.111:0/288132038 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92540035b0 con 0x7f926c0cf950 2023-03-29T10:58:51.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.030+0000 7f926a7fc700 1 -- 172.21.15.111:0/288132038 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f926c0d3a30 msgr2=0x7f926c0b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:51.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.030+0000 7f926a7fc700 1 --2- 172.21.15.111:0/288132038 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f926c0d3a30 0x7f926c0b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.030+0000 7f926a7fc700 1 -- 172.21.15.111:0/288132038 >> v1:172.21.15.111:6789/0 conn(0x7f926c0cf950 legacy=0x7f926c0cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:51.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.030+0000 7f926a7fc700 1 -- 172.21.15.111:0/288132038 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c0d16c0 con 0x7f926c15c950 2023-03-29T10:58:51.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.030+0000 7f926a7fc700 1 --2- 172.21.15.111:0/288132038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f926c15c950 0x7f926c0cf0f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9260009840 tx=0x7f9260009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af66e9545d287109 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:51.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.030+0000 7f9269ffb700 1 -- 172.21.15.111:0/288132038 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9260011830 con 0x7f926c15c950 2023-03-29T10:58:51.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.030+0000 7f9269ffb700 1 -- 172.21.15.111:0/288132038 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9260011990 con 0x7f926c15c950 2023-03-29T10:58:51.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.030+0000 7f9269ffb700 1 -- 172.21.15.111:0/288132038 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9260011d20 con 0x7f926c15c950 2023-03-29T10:58:51.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.031+0000 7f92715ff700 1 -- 172.21.15.111:0/288132038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f926c15c950 msgr2=0x7f926c0cf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:51.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.031+0000 7f92715ff700 1 --2- 172.21.15.111:0/288132038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f926c15c950 0x7f926c0cf0f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9260009840 tx=0x7f9260009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:51.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.031+0000 7f92715ff700 1 -- 172.21.15.111:0/288132038 shutdown_connections 2023-03-29T10:58:51.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.031+0000 7f92715ff700 1 --2- 172.21.15.111:0/288132038 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f926c0d3a30 0x7f926c0b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.031+0000 7f92715ff700 1 --2- 172.21.15.111:0/288132038 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f926c15c950 0x7f926c0cf0f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.031+0000 7f92715ff700 1 -- 172.21.15.111:0/288132038 >> 172.21.15.111:0/288132038 conn(0x7f926c071b10 msgr2=0x7f926c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:51.032 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.031+0000 7f92715ff700 1 -- 172.21.15.111:0/288132038 shutdown_connections 2023-03-29T10:58:51.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.031+0000 7f92715ff700 1 -- 172.21.15.111:0/288132038 wait complete. 2023-03-29T10:58:51.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.033+0000 7f92715ff700 1 Processor -- start 2023-03-29T10:58:51.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.033+0000 7f92715ff700 1 -- start start 2023-03-29T10:58:51.033 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.034+0000 7f92715ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f926c0cf950 0x7f926c157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:51.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.034+0000 7f92715ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f926c0d3a30 0x7f926c157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:51.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.034+0000 7f92715ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926c15c950 0x7f926c0d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:51.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.034+0000 7f92715ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0c7c10 con 0x7f926c0d3a30 2023-03-29T10:58:51.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.034+0000 7f92715ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c157e40 con 0x7f926c15c950 2023-03-29T10:58:51.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.034+0000 7f92715ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c15e270 con 0x7f926c0cf950 2023-03-29T10:58:51.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.034+0000 7f926a7fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f926c0d3a30 0x7f926c157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:51.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.034+0000 7f926a7fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f926c0d3a30 0x7f926c157900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51694/0 (socket says 172.21.15.111:51694) 2023-03-29T10:58:51.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.034+0000 7f926a7fc700 1 -- 172.21.15.111:0/1877050239 learned_addr learned my addr 172.21.15.111:0/1877050239 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:51.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.034+0000 7f926affd700 1 --2- 172.21.15.111:0/1877050239 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f926c0cf950 0x7f926c157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:51.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.035+0000 7f926b7fe700 1 --2- 172.21.15.111:0/1877050239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926c15c950 0x7f926c0d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:51.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.035+0000 7f926a7fc700 1 -- 172.21.15.111:0/1877050239 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f926c0cf950 msgr2=0x7f926c157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:51.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.035+0000 7f926a7fc700 1 --2- 172.21.15.111:0/1877050239 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f926c0cf950 0x7f926c157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.035+0000 7f926a7fc700 1 -- 172.21.15.111:0/1877050239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926c15c950 msgr2=0x7f926c0d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:51.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.035+0000 7f926a7fc700 1 --2- 172.21.15.111:0/1877050239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926c15c950 0x7f926c0d2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.035+0000 7f926a7fc700 1 -- 172.21.15.111:0/1877050239 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92600094a0 con 0x7f926c0d3a30 2023-03-29T10:58:51.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.035+0000 7f926affd700 1 --2- 172.21.15.111:0/1877050239 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f926c0cf950 0x7f926c157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:51.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.035+0000 7f926b7fe700 1 --2- 172.21.15.111:0/1877050239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926c15c950 0x7f926c0d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:51.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.035+0000 7f926a7fc700 1 --2- 172.21.15.111:0/1877050239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f926c0d3a30 0x7f926c157900 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9260009fd0 tx=0x7f9260004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:51.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.035+0000 7f92537fe700 1 -- 172.21.15.111:0/1877050239 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92600036a0 con 0x7f926c0d3a30 2023-03-29T10:58:51.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.035+0000 7f92537fe700 1 -- 172.21.15.111:0/1877050239 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926001e070 con 0x7f926c0d3a30 2023-03-29T10:58:51.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.035+0000 7f92715ff700 1 -- 172.21.15.111:0/1877050239 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926c0cf630 con 0x7f926c0d3a30 2023-03-29T10:58:51.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.035+0000 7f92537fe700 1 -- 172.21.15.111:0/1877050239 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9260003a60 con 0x7f926c0d3a30 2023-03-29T10:58:51.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.035+0000 7f92715ff700 1 -- 172.21.15.111:0/1877050239 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926c0d19a0 con 0x7f926c0d3a30 2023-03-29T10:58:51.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.036+0000 7f92715ff700 1 -- 172.21.15.111:0/1877050239 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926c0b62d0 con 0x7f926c0d3a30 2023-03-29T10:58:51.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.037+0000 7f92537fe700 1 -- 172.21.15.111:0/1877050239 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f9260003bc0 con 0x7f926c0d3a30 2023-03-29T10:58:51.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.038+0000 7f92537fe700 1 --2- 172.21.15.111:0/1877050239 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9258069de0 0x7f925806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:51.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.038+0000 7f92537fe700 1 -- 172.21.15.111:0/1877050239 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9260089310 con 0x7f926c0d3a30 2023-03-29T10:58:51.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.038+0000 7f926affd700 1 --2- 172.21.15.111:0/1877050239 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9258069de0 0x7f925806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:51.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.039+0000 7f926affd700 1 --2- 172.21.15.111:0/1877050239 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9258069de0 0x7f925806c290 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f925c0011e0 tx=0x7f925c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:51.041 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.042+0000 7f92537fe700 1 -- 172.21.15.111:0/1877050239 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f9260054880 con 0x7f926c0d3a30 2023-03-29T10:58:51.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.238+0000 7f92715ff700 1 -- 172.21.15.111:0/1877050239 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f926c074430 con 0x7f926c0d3a30 2023-03-29T10:58:51.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.239+0000 7f92537fe700 1 -- 172.21.15.111:0/1877050239 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19892 (secure 0 0 0) 0x7f9260057ea0 con 0x7f926c0d3a30 2023-03-29T10:58:51.239 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:51.240 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":30,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:58:09.476334+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.26","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:58:51.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.241+0000 7f92715ff700 1 -- 172.21.15.111:0/1877050239 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9258069de0 msgr2=0x7f925806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:51.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.242+0000 7f92715ff700 1 --2- 172.21.15.111:0/1877050239 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9258069de0 0x7f925806c290 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f925c0011e0 tx=0x7f925c009040 comp rx=0 tx=0).stop 2023-03-29T10:58:51.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.242+0000 7f92715ff700 1 -- 172.21.15.111:0/1877050239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f926c0d3a30 msgr2=0x7f926c157900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:51.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.242+0000 7f92715ff700 1 --2- 172.21.15.111:0/1877050239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f926c0d3a30 0x7f926c157900 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9260009fd0 tx=0x7f9260004750 comp rx=0 tx=0).stop 2023-03-29T10:58:51.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.242+0000 7f92715ff700 1 -- 172.21.15.111:0/1877050239 shutdown_connections 2023-03-29T10:58:51.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.242+0000 7f92715ff700 1 --2- 172.21.15.111:0/1877050239 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9258069de0 0x7f925806c290 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.242+0000 7f92715ff700 1 --2- 172.21.15.111:0/1877050239 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926c15c950 0x7f926c0d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.242+0000 7f92715ff700 1 --2- 172.21.15.111:0/1877050239 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f926c0cf950 0x7f926c157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.242+0000 7f92715ff700 1 --2- 172.21.15.111:0/1877050239 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f926c0d3a30 0x7f926c157900 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.242+0000 7f92715ff700 1 -- 172.21.15.111:0/1877050239 >> 172.21.15.111:0/1877050239 conn(0x7f926c071b10 msgr2=0x7f926c071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:51.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.242+0000 7f92715ff700 1 -- 172.21.15.111:0/1877050239 shutdown_connections 2023-03-29T10:58:51.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.242+0000 7f92715ff700 1 -- 172.21.15.111:0/1877050239 wait complete. 2023-03-29T10:58:51.259 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-29T10:57:16.183979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-29T10:57:21.239809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-29T10:57:26.192124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-29T10:57:26.796777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-29T10:57:29.874795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-29T10:58:51.259 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-29T10:58:51.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.444+0000 7f19f17ff700 1 Processor -- start 2023-03-29T10:58:51.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.444+0000 7f19f17ff700 1 -- start start 2023-03-29T10:58:51.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.445+0000 7f19f17ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19ec0d2110 0x7f19ec167550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:51.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.445+0000 7f19f17ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ec161000 con 0x7f19ec0cfbf0 2023-03-29T10:58:51.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.445+0000 7f19f17ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ec0d0020 con 0x7f19ec0b87a0 2023-03-29T10:58:51.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.445+0000 7f19f17ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0d02d0 con 0x7f19ec0d2110 2023-03-29T10:58:51.445 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.446+0000 7f19ea7fc700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f19ec0cfbf0 0x7f19ec1608f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47380/0 (socket says 172.21.15.111:47380) 2023-03-29T10:58:51.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.446+0000 7f19ea7fc700 1 -- 172.21.15.111:0/3548728523 learned_addr learned my addr 172.21.15.111:0/3548728523 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:51.446 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.446+0000 7f19eb7fe700 1 --2- 172.21.15.111:0/3548728523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19ec0d2110 0x7f19ec167550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:51.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.446+0000 7f19e9ffb700 1 -- 172.21.15.111:0/3548728523 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024394919 0 0) 0x7f19ec161000 con 0x7f19ec0cfbf0 2023-03-29T10:58:51.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.446+0000 7f19e9ffb700 1 -- 172.21.15.111:0/3548728523 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19d40035b0 con 0x7f19ec0cfbf0 2023-03-29T10:58:51.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.446+0000 7f19e9ffb700 1 -- 172.21.15.111:0/3548728523 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 254538346 0 0) 0x7f19ec0d0020 con 0x7f19ec0b87a0 2023-03-29T10:58:51.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.446+0000 7f19e9ffb700 1 -- 172.21.15.111:0/3548728523 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19ec161000 con 0x7f19ec0b87a0 2023-03-29T10:58:51.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.446+0000 7f19eb7fe700 1 -- 172.21.15.111:0/3548728523 >> v1:172.21.15.119:6789/0 conn(0x7f19ec0b87a0 legacy=0x7f19ec1601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:51.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.446+0000 7f19eb7fe700 1 -- 172.21.15.111:0/3548728523 >> v1:172.21.15.111:6789/0 conn(0x7f19ec0cfbf0 legacy=0x7f19ec1608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:51.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.446+0000 7f19eb7fe700 1 -- 172.21.15.111:0/3548728523 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec0cdc60 con 0x7f19ec0d2110 2023-03-29T10:58:51.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.447+0000 7f19eb7fe700 1 --2- 172.21.15.111:0/3548728523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19ec0d2110 0x7f19ec167550 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009dc0 tx=0x7f19e400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd48caeff6e648b9 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:51.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.447+0000 7f19e9ffb700 1 -- 172.21.15.111:0/3548728523 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e400e070 con 0x7f19ec0d2110 2023-03-29T10:58:51.448 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.447+0000 7f19e9ffb700 1 -- 172.21.15.111:0/3548728523 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e400bb20 con 0x7f19ec0d2110 2023-03-29T10:58:51.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.447+0000 7f19e9ffb700 1 -- 172.21.15.111:0/3548728523 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4012430 con 0x7f19ec0d2110 2023-03-29T10:58:51.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.447+0000 7f19f17ff700 1 -- 172.21.15.111:0/3548728523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19ec0d2110 msgr2=0x7f19ec167550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:51.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.447+0000 7f19f17ff700 1 --2- 172.21.15.111:0/3548728523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19ec0d2110 0x7f19ec167550 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009dc0 tx=0x7f19e400b330 comp rx=0 tx=0).stop 2023-03-29T10:58:51.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.447+0000 7f19f17ff700 1 -- 172.21.15.111:0/3548728523 shutdown_connections 2023-03-29T10:58:51.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.447+0000 7f19f17ff700 1 --2- 172.21.15.111:0/3548728523 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19ec0d2110 0x7f19ec167550 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.447+0000 7f19f17ff700 1 -- 172.21.15.111:0/3548728523 >> 172.21.15.111:0/3548728523 conn(0x7f19ec071b10 msgr2=0x7f19ec071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:51.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.447+0000 7f19f17ff700 1 -- 172.21.15.111:0/3548728523 shutdown_connections 2023-03-29T10:58:51.449 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.448+0000 7f19f17ff700 1 -- 172.21.15.111:0/3548728523 wait complete. 2023-03-29T10:58:51.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.449+0000 7f19f17ff700 1 Processor -- start 2023-03-29T10:58:51.451 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.449+0000 7f19f17ff700 1 -- start start 2023-03-29T10:58:51.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.450+0000 7f19f17ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec0b87a0 0x7f19ec14e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:51.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.450+0000 7f19f17ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19ec0cfbf0 0x7f19ec153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:51.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.450+0000 7f19f17ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19ec0d2110 0x7f19ec0b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:51.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.450+0000 7f19f17ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec14e610 con 0x7f19ec0d2110 2023-03-29T10:58:51.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.450+0000 7f19f17ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec1520b0 con 0x7f19ec0b87a0 2023-03-29T10:58:51.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.450+0000 7f19f17ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec152390 con 0x7f19ec0cfbf0 2023-03-29T10:58:51.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.450+0000 7f19eb7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19ec0d2110 0x7f19ec0b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:51.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.451+0000 7f19eb7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19ec0d2110 0x7f19ec0b3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51704/0 (socket says 172.21.15.111:51704) 2023-03-29T10:58:51.452 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.451+0000 7f19eb7fe700 1 -- 172.21.15.111:0/1583347030 learned_addr learned my addr 172.21.15.111:0/1583347030 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.451+0000 7f19eb7fe700 1 -- 172.21.15.111:0/1583347030 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19ec0cfbf0 msgr2=0x7f19ec153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.451+0000 7f19eb7fe700 1 --2- 172.21.15.111:0/1583347030 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19ec0cfbf0 0x7f19ec153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.451+0000 7f19eb7fe700 1 -- 172.21.15.111:0/1583347030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec0b87a0 msgr2=0x7f19ec14e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.451+0000 7f19eaffd700 1 --2- 172.21.15.111:0/1583347030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec0b87a0 0x7f19ec14e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.451+0000 7f19eb7fe700 1 --2- 172.21.15.111:0/1583347030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec0b87a0 0x7f19ec14e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.451+0000 7f19eb7fe700 1 -- 172.21.15.111:0/1583347030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e4009a70 con 0x7f19ec0d2110 2023-03-29T10:58:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.451+0000 7f19eaffd700 1 --2- 172.21.15.111:0/1583347030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec0b87a0 0x7f19ec14e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.451+0000 7f19eb7fe700 1 --2- 172.21.15.111:0/1583347030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19ec0d2110 0x7f19ec0b3a90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f19e400b370 tx=0x7f19e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.451+0000 7f19d37fe700 1 -- 172.21.15.111:0/1583347030 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e401b810 con 0x7f19ec0d2110 2023-03-29T10:58:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.451+0000 7f19d37fe700 1 -- 172.21.15.111:0/1583347030 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e4020070 con 0x7f19ec0d2110 2023-03-29T10:58:51.453 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.451+0000 7f19d37fe700 1 -- 172.21.15.111:0/1583347030 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e401bbf0 con 0x7f19ec0d2110 2023-03-29T10:58:51.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.451+0000 7f19f17ff700 1 -- 172.21.15.111:0/1583347030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec0cd980 con 0x7f19ec0d2110 2023-03-29T10:58:51.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.452+0000 7f19f17ff700 1 -- 172.21.15.111:0/1583347030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec0cd6a0 con 0x7f19ec0d2110 2023-03-29T10:58:51.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.453+0000 7f19f17ff700 1 -- 172.21.15.111:0/1583347030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec167a90 con 0x7f19ec0d2110 2023-03-29T10:58:51.454 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.453+0000 7f19d37fe700 1 -- 172.21.15.111:0/1583347030 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f19e401bd50 con 0x7f19ec0d2110 2023-03-29T10:58:51.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.454+0000 7f19d37fe700 1 --2- 172.21.15.111:0/1583347030 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f19d8069de0 0x7f19d806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:51.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.454+0000 7f19d37fe700 1 -- 172.21.15.111:0/1583347030 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f19e4022070 con 0x7f19ec0d2110 2023-03-29T10:58:51.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.454+0000 7f19eaffd700 1 --2- 172.21.15.111:0/1583347030 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f19d8069de0 0x7f19d806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:51.455 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.455+0000 7f19eaffd700 1 --2- 172.21.15.111:0/1583347030 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f19d8069de0 0x7f19d806c290 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0011e0 tx=0x7f19dc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:51.458 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.459+0000 7f19d37fe700 1 -- 172.21.15.111:0/1583347030 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f19e4055d30 con 0x7f19ec0d2110 2023-03-29T10:58:51.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.652+0000 7f19f17ff700 1 -- 172.21.15.111:0/1583347030 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f19ec0b52f0 con 0x7f19ec0d2110 2023-03-29T10:58:51.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.652+0000 7f19d37fe700 1 -- 172.21.15.111:0/1583347030 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f19e4059350 con 0x7f19ec0d2110 2023-03-29T10:58:51.652 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-03-29T10:58:51.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.654+0000 7f19f17ff700 1 -- 172.21.15.111:0/1583347030 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f19d8069de0 msgr2=0x7f19d806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:51.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.654+0000 7f19f17ff700 1 --2- 172.21.15.111:0/1583347030 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f19d8069de0 0x7f19d806c290 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0011e0 tx=0x7f19dc009040 comp rx=0 tx=0).stop 2023-03-29T10:58:51.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.654+0000 7f19f17ff700 1 -- 172.21.15.111:0/1583347030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19ec0d2110 msgr2=0x7f19ec0b3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:51.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.654+0000 7f19f17ff700 1 --2- 172.21.15.111:0/1583347030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19ec0d2110 0x7f19ec0b3a90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f19e400b370 tx=0x7f19e4004ad0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.655+0000 7f19f17ff700 1 -- 172.21.15.111:0/1583347030 shutdown_connections 2023-03-29T10:58:51.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.655+0000 7f19f17ff700 1 --2- 172.21.15.111:0/1583347030 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f19d8069de0 0x7f19d806c290 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.655+0000 7f19f17ff700 1 --2- 172.21.15.111:0/1583347030 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f19ec0d2110 0x7f19ec0b3a90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.655+0000 7f19f17ff700 1 --2- 172.21.15.111:0/1583347030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec0b87a0 0x7f19ec14e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.655+0000 7f19f17ff700 1 --2- 172.21.15.111:0/1583347030 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f19ec0cfbf0 0x7f19ec153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.655+0000 7f19f17ff700 1 -- 172.21.15.111:0/1583347030 >> 172.21.15.111:0/1583347030 conn(0x7f19ec071b10 msgr2=0x7f19ec0b7630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:51.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.655+0000 7f19f17ff700 1 -- 172.21.15.111:0/1583347030 shutdown_connections 2023-03-29T10:58:51.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.655+0000 7f19f17ff700 1 -- 172.21.15.111:0/1583347030 wait complete. 2023-03-29T10:58:51.672 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-29T10:58:51.801 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T10:58:51.805 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T10:58:51.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.860+0000 7f5cef7ff700 1 Processor -- start 2023-03-29T10:58:51.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.860+0000 7f5cef7ff700 1 -- start start 2023-03-29T10:58:51.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.861+0000 7f5cef7ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ce80b87a0 0x7f5ce80cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:51.860 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.861+0000 7f5cef7ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce80cdc60 con 0x7f5ce8160d40 2023-03-29T10:58:51.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.861+0000 7f5cef7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce80cd6a0 con 0x7f5ce81601e0 2023-03-29T10:58:51.861 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.861+0000 7f5cef7ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ce80d02d0 con 0x7f5ce80b87a0 2023-03-29T10:58:51.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.861+0000 7f5cedd9c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f5ce8160d40 0x7f5ce80d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47394/0 (socket says 172.21.15.111:47394) 2023-03-29T10:58:51.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.861+0000 7f5cedd9c700 1 -- 172.21.15.111:0/69227684 learned_addr learned my addr 172.21.15.111:0/69227684 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:51.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.861+0000 7f5ced59b700 1 --2- 172.21.15.111:0/69227684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce80b87a0 0x7f5ce80cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:51.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.862+0000 7f5cdffff700 1 -- 172.21.15.111:0/69227684 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3485055467 0 0) 0x7f5ce80cdc60 con 0x7f5ce8160d40 2023-03-29T10:58:51.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.862+0000 7f5cdffff700 1 -- 172.21.15.111:0/69227684 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cd40035b0 con 0x7f5ce8160d40 2023-03-29T10:58:51.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.862+0000 7f5cdffff700 1 -- 172.21.15.111:0/69227684 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228213981 0 0) 0x7f5ce80cd6a0 con 0x7f5ce81601e0 2023-03-29T10:58:51.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.862+0000 7f5cdffff700 1 -- 172.21.15.111:0/69227684 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ce80cdc60 con 0x7f5ce81601e0 2023-03-29T10:58:51.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.862+0000 7f5cdffff700 1 -- 172.21.15.111:0/69227684 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3277303471 0 0) 0x7f5cd40035b0 con 0x7f5ce8160d40 2023-03-29T10:58:51.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.862+0000 7f5cdffff700 1 -- 172.21.15.111:0/69227684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce80b87a0 msgr2=0x7f5ce80cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:51.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.862+0000 7f5cdffff700 1 --2- 172.21.15.111:0/69227684 >> v2:172.21.15.119:3300/0 conn(0x7f5ce80b87a0 0x7f5ce80cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.862+0000 7f5cdffff700 1 -- 172.21.15.111:0/69227684 >> v1:172.21.15.119:6789/0 conn(0x7f5ce81601e0 legacy=0x7f5ce8160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:51.862 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.862+0000 7f5cdffff700 1 -- 172.21.15.111:0/69227684 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce80cd980 con 0x7f5ce8160d40 2023-03-29T10:58:51.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.862+0000 7f5cdffff700 1 -- 172.21.15.111:0/69227684 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f5ce00019f0 con 0x7f5ce8160d40 2023-03-29T10:58:51.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.862+0000 7f5cdffff700 1 -- 172.21.15.111:0/69227684 >> v1:172.21.15.111:6789/0 conn(0x7f5ce8160d40 legacy=0x7f5ce80d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:51.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.862+0000 7f5cdffff700 1 --2- 172.21.15.111:0/69227684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cd4003b90 0x7f5cd4003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:51.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.862+0000 7f5cdffff700 1 --2- 172.21.15.111:0/69227684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cd4004850 0x7f5cd4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:51.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.862+0000 7f5cdffff700 1 --2- 172.21.15.111:0/69227684 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cd4005200 0x7f5cd400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:51.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.862+0000 7f5cdffff700 1 -- 172.21.15.111:0/69227684 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce80d02d0 con 0x7f5cd4004850 2023-03-29T10:58:51.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.862+0000 7f5cdffff700 1 -- 172.21.15.111:0/69227684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd400c060 con 0x7f5cd4003b90 2023-03-29T10:58:51.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.862+0000 7f5cdffff700 1 -- 172.21.15.111:0/69227684 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd400c220 con 0x7f5cd4005200 2023-03-29T10:58:51.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.862+0000 7f5cecd9a700 1 --2- 172.21.15.111:0/69227684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cd4004850 0x7f5cd4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:51.863 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.863+0000 7f5ced59b700 1 --2- 172.21.15.111:0/69227684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cd4003b90 0x7f5cd4003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:51.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.863+0000 7f5cedd9c700 1 --2- 172.21.15.111:0/69227684 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cd4005200 0x7f5cd400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:51.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.863+0000 7f5cecd9a700 1 -- 172.21.15.111:0/69227684 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cd4005200 msgr2=0x7f5cd400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:51.864 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.863+0000 7f5cecd9a700 1 --2- 172.21.15.111:0/69227684 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cd4005200 0x7f5cd400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.863+0000 7f5cecd9a700 1 -- 172.21.15.111:0/69227684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cd4003b90 msgr2=0x7f5cd4003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:51.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.863+0000 7f5cecd9a700 1 --2- 172.21.15.111:0/69227684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5cd4003b90 0x7f5cd4003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.863+0000 7f5cecd9a700 1 -- 172.21.15.111:0/69227684 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd40035b0 con 0x7f5cd4004850 2023-03-29T10:58:51.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.863+0000 7f5cedd9c700 1 -- 172.21.15.111:0/69227684 reap_dead start 2023-03-29T10:58:51.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.863+0000 7f5cecd9a700 1 --2- 172.21.15.111:0/69227684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cd4004850 0x7f5cd4004cc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8001940 tx=0x7f5cd4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c500f2d06e8a3c56 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:51.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.863+0000 7f5cdffff700 1 -- 172.21.15.111:0/69227684 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8001430 con 0x7f5cd4004850 2023-03-29T10:58:51.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.863+0000 7f5cdffff700 1 -- 172.21.15.111:0/69227684 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd8001d70 con 0x7f5cd4004850 2023-03-29T10:58:51.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.863+0000 7f5cdffff700 1 -- 172.21.15.111:0/69227684 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8005530 con 0x7f5cd4004850 2023-03-29T10:58:51.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.864+0000 7f5cef7ff700 1 -- 172.21.15.111:0/69227684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cd4004850 msgr2=0x7f5cd4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:51.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.864+0000 7f5cef7ff700 1 --2- 172.21.15.111:0/69227684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cd4004850 0x7f5cd4004cc0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8001940 tx=0x7f5cd4005880 comp rx=0 tx=0).stop 2023-03-29T10:58:51.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.864+0000 7f5cef7ff700 1 -- 172.21.15.111:0/69227684 shutdown_connections 2023-03-29T10:58:51.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.864+0000 7f5cef7ff700 1 --2- 172.21.15.111:0/69227684 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5cd4004850 0x7f5cd4004cc0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.864+0000 7f5cef7ff700 1 -- 172.21.15.111:0/69227684 >> 172.21.15.111:0/69227684 conn(0x7f5ce8071b10 msgr2=0x7f5ce8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:51.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.864+0000 7f5cef7ff700 1 -- 172.21.15.111:0/69227684 shutdown_connections 2023-03-29T10:58:51.865 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.864+0000 7f5cef7ff700 1 -- 172.21.15.111:0/69227684 wait complete. 2023-03-29T10:58:51.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.866+0000 7f5cef7ff700 1 Processor -- start 2023-03-29T10:58:51.866 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.867+0000 7f5cef7ff700 1 -- start start 2023-03-29T10:58:51.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.868+0000 7f5cef7ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cd4004850 0x7f5ce80d2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:51.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.868+0000 7f5cef7ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ce81e8c40 0x7f5ce81591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:51.867 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.868+0000 7f5cef7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ce815a320 0x7f5ce81596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:51.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.868+0000 7f5cef7ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce81e5fe0 con 0x7f5ce81e8c40 2023-03-29T10:58:51.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.868+0000 7f5cef7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce8159c30 con 0x7f5ce815a320 2023-03-29T10:58:51.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.868+0000 7f5cef7ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce815fac0 con 0x7f5cd4004850 2023-03-29T10:58:51.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.868+0000 7f5cecd9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ce81e8c40 0x7f5ce81591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:51.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.868+0000 7f5ced59b700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cd4004850 0x7f5ce80d2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:51.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.868+0000 7f5cecd9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ce81e8c40 0x7f5ce81591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51714/0 (socket says 172.21.15.111:51714) 2023-03-29T10:58:51.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.868+0000 7f5cecd9a700 1 -- 172.21.15.111:0/1445660331 learned_addr learned my addr 172.21.15.111:0/1445660331 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:51.868 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.868+0000 7f5cedd9c700 1 --2- 172.21.15.111:0/1445660331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ce815a320 0x7f5ce81596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.868+0000 7f5cecd9a700 1 -- 172.21.15.111:0/1445660331 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cd4004850 msgr2=0x7f5ce80d2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.868+0000 7f5cecd9a700 1 --2- 172.21.15.111:0/1445660331 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cd4004850 0x7f5ce80d2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.868+0000 7f5cecd9a700 1 -- 172.21.15.111:0/1445660331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ce815a320 msgr2=0x7f5ce81596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.868+0000 7f5cecd9a700 1 --2- 172.21.15.111:0/1445660331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ce815a320 0x7f5ce81596f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.868+0000 7f5cecd9a700 1 -- 172.21.15.111:0/1445660331 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce80cfd80 con 0x7f5ce81e8c40 2023-03-29T10:58:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.869+0000 7f5cecd9a700 1 --2- 172.21.15.111:0/1445660331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ce81e8c40 0x7f5ce81591b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400c030 tx=0x7f5cd80012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.869+0000 7f5cddffb700 1 -- 172.21.15.111:0/1445660331 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd80013d0 con 0x7f5ce81e8c40 2023-03-29T10:58:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.869+0000 7f5cddffb700 1 -- 172.21.15.111:0/1445660331 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd8006a70 con 0x7f5ce81e8c40 2023-03-29T10:58:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.869+0000 7f5cddffb700 1 -- 172.21.15.111:0/1445660331 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8006e20 con 0x7f5ce81e8c40 2023-03-29T10:58:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.869+0000 7f5cef7ff700 1 -- 172.21.15.111:0/1445660331 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce80cfbf0 con 0x7f5ce81e8c40 2023-03-29T10:58:51.869 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.869+0000 7f5cef7ff700 1 -- 172.21.15.111:0/1445660331 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce80cff10 con 0x7f5ce81e8c40 2023-03-29T10:58:51.870 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.870+0000 7f5cef7ff700 1 -- 172.21.15.111:0/1445660331 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce80b52f0 con 0x7f5ce81e8c40 2023-03-29T10:58:51.871 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.871+0000 7f5cddffb700 1 -- 172.21.15.111:0/1445660331 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f5cd8007040 con 0x7f5ce81e8c40 2023-03-29T10:58:51.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.872+0000 7f5cddffb700 1 --2- 172.21.15.111:0/1445660331 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5ccc069e30 0x7f5ccc06c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:51.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.872+0000 7f5cddffb700 1 -- 172.21.15.111:0/1445660331 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5cd8088b20 con 0x7f5ce81e8c40 2023-03-29T10:58:51.872 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.873+0000 7f5ced59b700 1 --2- 172.21.15.111:0/1445660331 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5ccc069e30 0x7f5ccc06c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:51.873 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.873+0000 7f5ced59b700 1 --2- 172.21.15.111:0/1445660331 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5ccc069e30 0x7f5ccc06c2e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8069690 tx=0x7f5ce4009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:51.877 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:51.878+0000 7f5cddffb700 1 -- 172.21.15.111:0/1445660331 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f5cd8054110 con 0x7f5ce81e8c40 2023-03-29T10:58:52.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.073+0000 7f5cef7ff700 1 -- 172.21.15.111:0/1445660331 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5ce8072260 con 0x7f5ce81e8c40 2023-03-29T10:58:52.073 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.074+0000 7f5cddffb700 1 -- 172.21.15.111:0/1445660331 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5cd802b020 con 0x7f5ce81e8c40 2023-03-29T10:58:52.073 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 8 2023-03-29T10:58:52.075 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.076+0000 7f5cef7ff700 1 -- 172.21.15.111:0/1445660331 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5ccc069e30 msgr2=0x7f5ccc06c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:52.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.076+0000 7f5cef7ff700 1 --2- 172.21.15.111:0/1445660331 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5ccc069e30 0x7f5ccc06c2e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8069690 tx=0x7f5ce4009210 comp rx=0 tx=0).stop 2023-03-29T10:58:52.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.076+0000 7f5cef7ff700 1 -- 172.21.15.111:0/1445660331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ce81e8c40 msgr2=0x7f5ce81591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:52.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.076+0000 7f5cef7ff700 1 --2- 172.21.15.111:0/1445660331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ce81e8c40 0x7f5ce81591b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5cd400c030 tx=0x7f5cd80012d0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.076+0000 7f5cef7ff700 1 -- 172.21.15.111:0/1445660331 shutdown_connections 2023-03-29T10:58:52.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.076+0000 7f5cef7ff700 1 --2- 172.21.15.111:0/1445660331 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5ccc069e30 0x7f5ccc06c2e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.076+0000 7f5cef7ff700 1 --2- 172.21.15.111:0/1445660331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ce815a320 0x7f5ce81596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.076+0000 7f5cef7ff700 1 --2- 172.21.15.111:0/1445660331 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f5cd4004850 0x7f5ce80d2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.076+0000 7f5cef7ff700 1 --2- 172.21.15.111:0/1445660331 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f5ce81e8c40 0x7f5ce81591b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.076+0000 7f5cef7ff700 1 -- 172.21.15.111:0/1445660331 >> 172.21.15.111:0/1445660331 conn(0x7f5ce8071b10 msgr2=0x7f5ce80d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:52.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.076+0000 7f5cef7ff700 1 -- 172.21.15.111:0/1445660331 shutdown_connections 2023-03-29T10:58:52.076 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.076+0000 7f5cef7ff700 1 -- 172.21.15.111:0/1445660331 wait complete. 2023-03-29T10:58:52.093 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-29T10:58:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.279+0000 7fd5bffff700 1 Processor -- start 2023-03-29T10:58:52.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.279+0000 7fd5bffff700 1 -- start start 2023-03-29T10:58:52.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.280+0000 7fd5bffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd5b80ccfa0 0x7fd5b81676c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:52.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.280+0000 7fd5bffff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b80b26f0 con 0x7fd5b806b4f0 2023-03-29T10:58:52.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.280+0000 7fd5bffff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b80cfb60 con 0x7fd5b80d0130 2023-03-29T10:58:52.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.280+0000 7fd5bffff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b80cfe40 con 0x7fd5b80ccfa0 2023-03-29T10:58:52.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.281+0000 7fd5bdd9b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fd5b806b4f0 0x7fd5b806b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47402/0 (socket says 172.21.15.111:47402) 2023-03-29T10:58:52.280 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.281+0000 7fd5bdd9b700 1 -- 172.21.15.111:0/2933390637 learned_addr learned my addr 172.21.15.111:0/2933390637 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:52.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.281+0000 7fd5bcd99700 1 -- 172.21.15.111:0/2933390637 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2606138683 0 0) 0x7fd5b80b26f0 con 0x7fd5b806b4f0 2023-03-29T10:58:52.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.281+0000 7fd5be59c700 1 --2- 172.21.15.111:0/2933390637 >> v2:172.21.15.119:3300/0 conn(0x7fd5b80ccfa0 0x7fd5b81676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:52.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.281+0000 7fd5bcd99700 1 -- 172.21.15.111:0/2933390637 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5a40035b0 con 0x7fd5b806b4f0 2023-03-29T10:58:52.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.281+0000 7fd5bcd99700 1 -- 172.21.15.111:0/2933390637 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1577844420 0 0) 0x7fd5a40035b0 con 0x7fd5b806b4f0 2023-03-29T10:58:52.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.281+0000 7fd5bcd99700 1 -- 172.21.15.111:0/2933390637 >> v2:172.21.15.119:3300/0 conn(0x7fd5b80ccfa0 msgr2=0x7fd5b81676c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:52.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.281+0000 7fd5bcd99700 1 --2- 172.21.15.111:0/2933390637 >> v2:172.21.15.119:3300/0 conn(0x7fd5b80ccfa0 0x7fd5b81676c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.281+0000 7fd5bcd99700 1 -- 172.21.15.111:0/2933390637 >> v1:172.21.15.119:6789/0 conn(0x7fd5b80d0130 legacy=0x7fd5b80cf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:52.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.281+0000 7fd5bcd99700 1 -- 172.21.15.111:0/2933390637 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b80cb4b0 con 0x7fd5b806b4f0 2023-03-29T10:58:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.281+0000 7fd5bcd99700 1 -- 172.21.15.111:0/2933390637 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7fd5b4001670 con 0x7fd5b806b4f0 2023-03-29T10:58:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.282+0000 7fd5bcd99700 1 -- 172.21.15.111:0/2933390637 >> v1:172.21.15.111:6789/0 conn(0x7fd5b806b4f0 legacy=0x7fd5b806b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.282+0000 7fd5bcd99700 1 --2- 172.21.15.111:0/2933390637 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5a4003a50 0x7fd5a4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.282+0000 7fd5bcd99700 1 --2- 172.21.15.111:0/2933390637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5a4004850 0x7fd5a4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.282+0000 7fd5bcd99700 1 --2- 172.21.15.111:0/2933390637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5a40052d0 0x7fd5a400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.282+0000 7fd5bcd99700 1 -- 172.21.15.111:0/2933390637 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b80cfe40 con 0x7fd5a40052d0 2023-03-29T10:58:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.282+0000 7fd5bcd99700 1 -- 172.21.15.111:0/2933390637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a400c200 con 0x7fd5a4004850 2023-03-29T10:58:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.282+0000 7fd5bcd99700 1 -- 172.21.15.111:0/2933390637 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a400c3c0 con 0x7fd5a4003a50 2023-03-29T10:58:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.282+0000 7fd5be59c700 1 --2- 172.21.15.111:0/2933390637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5a40052d0 0x7fd5a400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.282+0000 7fd5bdd9b700 1 --2- 172.21.15.111:0/2933390637 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5a4003a50 0x7fd5a4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.282+0000 7fd5bd59a700 1 --2- 172.21.15.111:0/2933390637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5a4004850 0x7fd5a4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.282+0000 7fd5be59c700 1 -- 172.21.15.111:0/2933390637 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5a4003a50 msgr2=0x7fd5a4003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.282+0000 7fd5be59c700 1 --2- 172.21.15.111:0/2933390637 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5a4003a50 0x7fd5a4003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.282+0000 7fd5be59c700 1 -- 172.21.15.111:0/2933390637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5a4004850 msgr2=0x7fd5a4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.282+0000 7fd5be59c700 1 --2- 172.21.15.111:0/2933390637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5a4004850 0x7fd5a4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.282+0000 7fd5be59c700 1 -- 172.21.15.111:0/2933390637 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a40035b0 con 0x7fd5a40052d0 2023-03-29T10:58:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.282+0000 7fd5bdd9b700 1 --2- 172.21.15.111:0/2933390637 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5a4003a50 0x7fd5a4003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:52.283 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.282+0000 7fd5bd59a700 1 --2- 172.21.15.111:0/2933390637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5a4004850 0x7fd5a4004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.282+0000 7fd5be59c700 1 -- 172.21.15.111:0/2933390637 reap_dead start 2023-03-29T10:58:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.282+0000 7fd5be59c700 1 --2- 172.21.15.111:0/2933390637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5a40052d0 0x7fd5a400ba10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009fc0 tx=0x7fd5a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca53717f2f5f338 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.283+0000 7fd5bcd99700 1 -- 172.21.15.111:0/2933390637 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0003a20 con 0x7fd5a40052d0 2023-03-29T10:58:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.283+0000 7fd5bcd99700 1 -- 172.21.15.111:0/2933390637 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b000f040 con 0x7fd5a40052d0 2023-03-29T10:58:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.283+0000 7fd5bcd99700 1 -- 172.21.15.111:0/2933390637 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0003de0 con 0x7fd5a40052d0 2023-03-29T10:58:52.284 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.283+0000 7fd5bffff700 1 -- 172.21.15.111:0/2933390637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5a40052d0 msgr2=0x7fd5a400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.283+0000 7fd5bffff700 1 --2- 172.21.15.111:0/2933390637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5a40052d0 0x7fd5a400ba10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009fc0 tx=0x7fd5a4005880 comp rx=0 tx=0).stop 2023-03-29T10:58:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.283+0000 7fd5bffff700 1 -- 172.21.15.111:0/2933390637 shutdown_connections 2023-03-29T10:58:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.283+0000 7fd5bffff700 1 --2- 172.21.15.111:0/2933390637 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5a40052d0 0x7fd5a400ba10 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.283+0000 7fd5bffff700 1 -- 172.21.15.111:0/2933390637 >> 172.21.15.111:0/2933390637 conn(0x7fd5b809a090 msgr2=0x7fd5b80d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.283+0000 7fd5bffff700 1 -- 172.21.15.111:0/2933390637 shutdown_connections 2023-03-29T10:58:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.283+0000 7fd5bffff700 1 -- 172.21.15.111:0/2933390637 wait complete. 2023-03-29T10:58:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.285+0000 7fd5bffff700 1 Processor -- start 2023-03-29T10:58:52.285 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.285+0000 7fd5bffff700 1 -- start start 2023-03-29T10:58:52.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.286+0000 7fd5bffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5a40052d0 0x7fd5b8156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:52.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.286+0000 7fd5bffff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5b8157e10 0x7fd5b8159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:52.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.286+0000 7fd5bffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5b80cc270 0x7fd5b80cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:52.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.286+0000 7fd5bffff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b815a200 con 0x7fd5b80cc270 2023-03-29T10:58:52.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.286+0000 7fd5bffff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b80ccc20 con 0x7fd5a40052d0 2023-03-29T10:58:52.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.286+0000 7fd5bffff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b81f2070 con 0x7fd5b8157e10 2023-03-29T10:58:52.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.286+0000 7fd5be59c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5b80cc270 0x7fd5b80cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:52.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.286+0000 7fd5be59c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5b80cc270 0x7fd5b80cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51726/0 (socket says 172.21.15.111:51726) 2023-03-29T10:58:52.286 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.286+0000 7fd5be59c700 1 -- 172.21.15.111:0/1800075634 learned_addr learned my addr 172.21.15.111:0/1800075634 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:52.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.287+0000 7fd5bdd9b700 1 --2- 172.21.15.111:0/1800075634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5a40052d0 0x7fd5b8156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:52.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.287+0000 7fd5bd59a700 1 --2- 172.21.15.111:0/1800075634 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5b8157e10 0x7fd5b8159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:52.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.287+0000 7fd5be59c700 1 -- 172.21.15.111:0/1800075634 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5b8157e10 msgr2=0x7fd5b8159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:52.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.287+0000 7fd5be59c700 1 --2- 172.21.15.111:0/1800075634 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5b8157e10 0x7fd5b8159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.287+0000 7fd5be59c700 1 -- 172.21.15.111:0/1800075634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5a40052d0 msgr2=0x7fd5b8156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:52.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.287+0000 7fd5be59c700 1 --2- 172.21.15.111:0/1800075634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5a40052d0 0x7fd5b8156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.287 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.287+0000 7fd5be59c700 1 -- 172.21.15.111:0/1800075634 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b80cba70 con 0x7fd5b80cc270 2023-03-29T10:58:52.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.287+0000 7fd5bd59a700 1 --2- 172.21.15.111:0/1800075634 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5b8157e10 0x7fd5b8159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:52.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.287+0000 7fd5be59c700 1 --2- 172.21.15.111:0/1800075634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5b80cc270 0x7fd5b80cc6e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd5b808a700 tx=0x7fd5b0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:52.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.287+0000 7fd5ae7fc700 1 -- 172.21.15.111:0/1800075634 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0003e90 con 0x7fd5b80cc270 2023-03-29T10:58:52.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.287+0000 7fd5ae7fc700 1 -- 172.21.15.111:0/1800075634 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b0004230 con 0x7fd5b80cc270 2023-03-29T10:58:52.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.287+0000 7fd5bffff700 1 -- 172.21.15.111:0/1800075634 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b80cb790 con 0x7fd5b80cc270 2023-03-29T10:58:52.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.287+0000 7fd5ae7fc700 1 -- 172.21.15.111:0/1800075634 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b00043e0 con 0x7fd5b80cc270 2023-03-29T10:58:52.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.287+0000 7fd5bffff700 1 -- 172.21.15.111:0/1800075634 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b815ced0 con 0x7fd5b80cc270 2023-03-29T10:58:52.288 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.288+0000 7fd5bffff700 1 -- 172.21.15.111:0/1800075634 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b80b6a20 con 0x7fd5b80cc270 2023-03-29T10:58:52.289 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.289+0000 7fd5ae7fc700 1 -- 172.21.15.111:0/1800075634 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fd5b0004540 con 0x7fd5b80cc270 2023-03-29T10:58:52.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.290+0000 7fd5ae7fc700 1 --2- 172.21.15.111:0/1800075634 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd59c069de0 0x7fd59c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:52.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.290+0000 7fd5ae7fc700 1 -- 172.21.15.111:0/1800075634 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd5b0089d50 con 0x7fd5b80cc270 2023-03-29T10:58:52.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.290+0000 7fd5bdd9b700 1 --2- 172.21.15.111:0/1800075634 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd59c069de0 0x7fd59c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:52.290 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.291+0000 7fd5bdd9b700 1 --2- 172.21.15.111:0/1800075634 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd59c069de0 0x7fd59c06c290 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4000c00 tx=0x7fd5b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:52.294 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.294+0000 7fd5ae7fc700 1 -- 172.21.15.111:0/1800075634 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fd5b0058cf0 con 0x7fd5b80cc270 2023-03-29T10:58:52.486 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.487+0000 7fd5bffff700 1 -- 172.21.15.111:0/1800075634 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd5b80d0e80 con 0x7fd5b80cc270 2023-03-29T10:58:52.487 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.487+0000 7fd5ae7fc700 1 -- 172.21.15.111:0/1800075634 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd5b0089070 con 0x7fd5b80cc270 2023-03-29T10:58:52.487 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:52.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.489+0000 7fd5bffff700 1 -- 172.21.15.111:0/1800075634 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd59c069de0 msgr2=0x7fd59c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:52.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.489+0000 7fd5bffff700 1 --2- 172.21.15.111:0/1800075634 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd59c069de0 0x7fd59c06c290 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4000c00 tx=0x7fd5b4009040 comp rx=0 tx=0).stop 2023-03-29T10:58:52.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.489+0000 7fd5bffff700 1 -- 172.21.15.111:0/1800075634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5b80cc270 msgr2=0x7fd5b80cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:52.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.489+0000 7fd5bffff700 1 --2- 172.21.15.111:0/1800075634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5b80cc270 0x7fd5b80cc6e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd5b808a700 tx=0x7fd5b0009750 comp rx=0 tx=0).stop 2023-03-29T10:58:52.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.489+0000 7fd5bffff700 1 -- 172.21.15.111:0/1800075634 shutdown_connections 2023-03-29T10:58:52.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.489+0000 7fd5bffff700 1 --2- 172.21.15.111:0/1800075634 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd59c069de0 0x7fd59c06c290 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.489 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.489+0000 7fd5bffff700 1 --2- 172.21.15.111:0/1800075634 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5b80cc270 0x7fd5b80cc6e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.489+0000 7fd5bffff700 1 --2- 172.21.15.111:0/1800075634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5a40052d0 0x7fd5b8156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.489+0000 7fd5bffff700 1 --2- 172.21.15.111:0/1800075634 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5b8157e10 0x7fd5b8159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.489+0000 7fd5bffff700 1 -- 172.21.15.111:0/1800075634 >> 172.21.15.111:0/1800075634 conn(0x7fd5b809a090 msgr2=0x7fd5b806f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:52.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.490+0000 7fd5bffff700 1 -- 172.21.15.111:0/1800075634 shutdown_connections 2023-03-29T10:58:52.490 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.490+0000 7fd5bffff700 1 -- 172.21.15.111:0/1800075634 wait complete. 2023-03-29T10:58:52.506 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-29T10:58:52.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.693+0000 7f6837fff700 1 Processor -- start 2023-03-29T10:58:52.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.693+0000 7f6837fff700 1 -- start start 2023-03-29T10:58:52.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.694+0000 7f6837fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f683015cf10 0x7f683006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:52.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.694+0000 7f6837fff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68300cc9e0 con 0x7f68300adf70 2023-03-29T10:58:52.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.694+0000 7f6837fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f683015de20 con 0x7f68300cd760 2023-03-29T10:58:52.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.694+0000 7f6837fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68300cd580 con 0x7f683015cf10 2023-03-29T10:58:52.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.695+0000 7f683559a700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f68300adf70 0x7f68300cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47412/0 (socket says 172.21.15.111:47412) 2023-03-29T10:58:52.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.695+0000 7f683559a700 1 -- 172.21.15.111:0/1763138566 learned_addr learned my addr 172.21.15.111:0/1763138566 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:52.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.695+0000 7f683659c700 1 --2- 172.21.15.111:0/1763138566 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f683015cf10 0x7f683006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:52.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.695+0000 7f6834d99700 1 -- 172.21.15.111:0/1763138566 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775605418 0 0) 0x7f68300cc9e0 con 0x7f68300adf70 2023-03-29T10:58:52.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.695+0000 7f6834d99700 1 -- 172.21.15.111:0/1763138566 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f681c0035b0 con 0x7f68300adf70 2023-03-29T10:58:52.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.695+0000 7f6834d99700 1 -- 172.21.15.111:0/1763138566 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244561606 0 0) 0x7f683015de20 con 0x7f68300cd760 2023-03-29T10:58:52.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.695+0000 7f6834d99700 1 -- 172.21.15.111:0/1763138566 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68300cc9e0 con 0x7f68300cd760 2023-03-29T10:58:52.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.695+0000 7f683659c700 1 -- 172.21.15.111:0/1763138566 >> v1:172.21.15.119:6789/0 conn(0x7f68300cd760 legacy=0x7f68300cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:52.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.695+0000 7f683659c700 1 -- 172.21.15.111:0/1763138566 >> v1:172.21.15.111:6789/0 conn(0x7f68300adf70 legacy=0x7f68300cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:52.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.695+0000 7f683659c700 1 -- 172.21.15.111:0/1763138566 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683015d860 con 0x7f683015cf10 2023-03-29T10:58:52.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.696+0000 7f683659c700 1 --2- 172.21.15.111:0/1763138566 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f683015cf10 0x7f683006c6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6828009dc0 tx=0x7f682800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=395706bf1b9caff9 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:52.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.696+0000 7f6834d99700 1 -- 172.21.15.111:0/1763138566 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682800e070 con 0x7f683015cf10 2023-03-29T10:58:52.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.696+0000 7f6834d99700 1 -- 172.21.15.111:0/1763138566 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682800bb20 con 0x7f683015cf10 2023-03-29T10:58:52.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.696+0000 7f6834d99700 1 -- 172.21.15.111:0/1763138566 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828012430 con 0x7f683015cf10 2023-03-29T10:58:52.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.696+0000 7f6837fff700 1 -- 172.21.15.111:0/1763138566 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f683015cf10 msgr2=0x7f683006c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:52.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.696+0000 7f6837fff700 1 --2- 172.21.15.111:0/1763138566 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f683015cf10 0x7f683006c6c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6828009dc0 tx=0x7f682800b330 comp rx=0 tx=0).stop 2023-03-29T10:58:52.696 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.696+0000 7f6837fff700 1 -- 172.21.15.111:0/1763138566 shutdown_connections 2023-03-29T10:58:52.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.696+0000 7f6837fff700 1 --2- 172.21.15.111:0/1763138566 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f683015cf10 0x7f683006c6c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.696+0000 7f6837fff700 1 -- 172.21.15.111:0/1763138566 >> 172.21.15.111:0/1763138566 conn(0x7f68300cf720 msgr2=0x7f68300cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:52.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.697+0000 7f6837fff700 1 -- 172.21.15.111:0/1763138566 shutdown_connections 2023-03-29T10:58:52.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.697+0000 7f6837fff700 1 -- 172.21.15.111:0/1763138566 wait complete. 2023-03-29T10:58:52.697 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.698+0000 7f6837fff700 1 Processor -- start 2023-03-29T10:58:52.698 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.698+0000 7f6837fff700 1 -- start start 2023-03-29T10:58:52.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.699+0000 7f6837fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68300adf70 0x7f68301572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:52.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.699+0000 7f6837fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68300cd760 0x7f68301f00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:52.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.699+0000 7f6837fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f683015cf10 0x7f68300b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:52.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.699+0000 7f6837fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68300c8a10 con 0x7f68300cd760 2023-03-29T10:58:52.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.699+0000 7f6837fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6830157810 con 0x7f68300adf70 2023-03-29T10:58:52.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.699+0000 7f6837fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68300c5c70 con 0x7f683015cf10 2023-03-29T10:58:52.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.699+0000 7f683559a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68300cd760 0x7f68301f00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:52.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.700+0000 7f683559a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68300cd760 0x7f68301f00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51742/0 (socket says 172.21.15.111:51742) 2023-03-29T10:58:52.699 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.700+0000 7f683559a700 1 -- 172.21.15.111:0/1125135978 learned_addr learned my addr 172.21.15.111:0/1125135978 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:52.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.700+0000 7f683559a700 1 -- 172.21.15.111:0/1125135978 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f683015cf10 msgr2=0x7f68300b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:52.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.700+0000 7f683559a700 1 --2- 172.21.15.111:0/1125135978 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f683015cf10 0x7f68300b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.700+0000 7f683559a700 1 -- 172.21.15.111:0/1125135978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68300adf70 msgr2=0x7f68301572d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:52.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.700+0000 7f683559a700 1 --2- 172.21.15.111:0/1125135978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68300adf70 0x7f68301572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.700 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.700+0000 7f683559a700 1 -- 172.21.15.111:0/1125135978 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6828009a70 con 0x7f68300cd760 2023-03-29T10:58:52.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.700+0000 7f683559a700 1 --2- 172.21.15.111:0/1125135978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68300cd760 0x7f68301f00c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f682000c700 tx=0x7f682000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:52.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.700+0000 7f68267fc700 1 -- 172.21.15.111:0/1125135978 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68200074e0 con 0x7f68300cd760 2023-03-29T10:58:52.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.700+0000 7f68267fc700 1 -- 172.21.15.111:0/1125135978 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6820007640 con 0x7f68300cd760 2023-03-29T10:58:52.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.700+0000 7f68267fc700 1 -- 172.21.15.111:0/1125135978 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6820005a30 con 0x7f68300cd760 2023-03-29T10:58:52.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.700+0000 7f6837fff700 1 -- 172.21.15.111:0/1125135978 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68300cc670 con 0x7f68300cd760 2023-03-29T10:58:52.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.700+0000 7f6837fff700 1 -- 172.21.15.111:0/1125135978 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683015db40 con 0x7f68300cd760 2023-03-29T10:58:52.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.701+0000 7f6837fff700 1 -- 172.21.15.111:0/1125135978 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6830071600 con 0x7f68300cd760 2023-03-29T10:58:52.701 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.702+0000 7f68267fc700 1 -- 172.21.15.111:0/1125135978 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f6820005b90 con 0x7f68300cd760 2023-03-29T10:58:52.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.703+0000 7f68267fc700 1 --2- 172.21.15.111:0/1125135978 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6814069de0 0x7f681406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:52.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.704+0000 7f68267fc700 1 -- 172.21.15.111:0/1125135978 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f6820087850 con 0x7f68300cd760 2023-03-29T10:58:52.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.704+0000 7f6835d9b700 1 --2- 172.21.15.111:0/1125135978 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6814069de0 0x7f681406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:52.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.704+0000 7f6835d9b700 1 --2- 172.21.15.111:0/1125135978 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6814069de0 0x7f681406c290 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f682c007fd0 tx=0x7f682c0079a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:52.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.708+0000 7f68267fc700 1 -- 172.21.15.111:0/1125135978 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f682004c160 con 0x7f68300cd760 2023-03-29T10:58:52.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.899+0000 7f6837fff700 1 -- 172.21.15.111:0/1125135978 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6830071b50 con 0x7f68300cd760 2023-03-29T10:58:52.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.900+0000 7f68267fc700 1 -- 172.21.15.111:0/1125135978 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6820052d90 con 0x7f68300cd760 2023-03-29T10:58:52.899 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-03-29T10:58:52.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.902+0000 7f6837fff700 1 -- 172.21.15.111:0/1125135978 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6814069de0 msgr2=0x7f681406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:52.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.902+0000 7f6837fff700 1 --2- 172.21.15.111:0/1125135978 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6814069de0 0x7f681406c290 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f682c007fd0 tx=0x7f682c0079a0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.902+0000 7f6837fff700 1 -- 172.21.15.111:0/1125135978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68300cd760 msgr2=0x7f68301f00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:52.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.902+0000 7f6837fff700 1 --2- 172.21.15.111:0/1125135978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68300cd760 0x7f68301f00c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f682000c700 tx=0x7f682000cac0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.902+0000 7f6837fff700 1 -- 172.21.15.111:0/1125135978 shutdown_connections 2023-03-29T10:58:52.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.902+0000 7f6837fff700 1 --2- 172.21.15.111:0/1125135978 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6814069de0 0x7f681406c290 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.902+0000 7f6837fff700 1 --2- 172.21.15.111:0/1125135978 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f683015cf10 0x7f68300b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.902+0000 7f6837fff700 1 --2- 172.21.15.111:0/1125135978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68300adf70 0x7f68301572d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.902+0000 7f6837fff700 1 --2- 172.21.15.111:0/1125135978 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f68300cd760 0x7f68301f00c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:52.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.902+0000 7f6837fff700 1 -- 172.21.15.111:0/1125135978 >> 172.21.15.111:0/1125135978 conn(0x7f68300cf720 msgr2=0x7f68300c9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:52.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.902+0000 7f6837fff700 1 -- 172.21.15.111:0/1125135978 shutdown_connections 2023-03-29T10:58:52.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:52.902+0000 7f6837fff700 1 -- 172.21.15.111:0/1125135978 wait complete. 2023-03-29T10:58:52.919 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-29T10:58:53.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.105+0000 7fb2575ff700 1 Processor -- start 2023-03-29T10:58:53.105 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.105+0000 7fb2575ff700 1 -- start start 2023-03-29T10:58:53.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.106+0000 7fb2575ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2500cfbf0 0x7fb2501608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:53.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.106+0000 7fb2575ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2500d0020 con 0x7fb2500b87a0 2023-03-29T10:58:53.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.106+0000 7fb2575ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2500cdc60 con 0x7fb2500d1100 2023-03-29T10:58:53.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.106+0000 7fb2575ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2500d02d0 con 0x7fb2500cfbf0 2023-03-29T10:58:53.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.107+0000 7fb25539b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fb2500b87a0 0x7fb2501601e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47418/0 (socket says 172.21.15.111:47418) 2023-03-29T10:58:53.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.107+0000 7fb25539b700 1 -- 172.21.15.111:0/4189756034 learned_addr learned my addr 172.21.15.111:0/4189756034 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:53.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.107+0000 7fb254b9a700 1 --2- 172.21.15.111:0/4189756034 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2500cfbf0 0x7fb2501608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:53.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.107+0000 7fb247fff700 1 -- 172.21.15.111:0/4189756034 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371387557 0 0) 0x7fb2500d0020 con 0x7fb2500b87a0 2023-03-29T10:58:53.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.107+0000 7fb247fff700 1 -- 172.21.15.111:0/4189756034 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb23c0035b0 con 0x7fb2500b87a0 2023-03-29T10:58:53.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.107+0000 7fb247fff700 1 -- 172.21.15.111:0/4189756034 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782954805 0 0) 0x7fb2500cdc60 con 0x7fb2500d1100 2023-03-29T10:58:53.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.107+0000 7fb247fff700 1 -- 172.21.15.111:0/4189756034 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2500d0020 con 0x7fb2500d1100 2023-03-29T10:58:53.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.107+0000 7fb254b9a700 1 -- 172.21.15.111:0/4189756034 >> v1:172.21.15.119:6789/0 conn(0x7fb2500d1100 legacy=0x7fb2500d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:53.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.107+0000 7fb254b9a700 1 -- 172.21.15.111:0/4189756034 >> v1:172.21.15.111:6789/0 conn(0x7fb2500b87a0 legacy=0x7fb2501601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:53.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.107+0000 7fb254b9a700 1 -- 172.21.15.111:0/4189756034 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2500cd6a0 con 0x7fb2500cfbf0 2023-03-29T10:58:53.107 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.108+0000 7fb254b9a700 1 --2- 172.21.15.111:0/4189756034 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2500cfbf0 0x7fb2501608f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb240009870 tx=0x7fb240009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da105042e1ccec89 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:53.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.108+0000 7fb247fff700 1 -- 172.21.15.111:0/4189756034 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb240011830 con 0x7fb2500cfbf0 2023-03-29T10:58:53.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.108+0000 7fb247fff700 1 -- 172.21.15.111:0/4189756034 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb240011990 con 0x7fb2500cfbf0 2023-03-29T10:58:53.108 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.108+0000 7fb247fff700 1 -- 172.21.15.111:0/4189756034 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb240011d20 con 0x7fb2500cfbf0 2023-03-29T10:58:53.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.108+0000 7fb2575ff700 1 -- 172.21.15.111:0/4189756034 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2500cfbf0 msgr2=0x7fb2501608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:53.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.108+0000 7fb2575ff700 1 --2- 172.21.15.111:0/4189756034 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2500cfbf0 0x7fb2501608f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb240009870 tx=0x7fb240009b80 comp rx=0 tx=0).stop 2023-03-29T10:58:53.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.109+0000 7fb2575ff700 1 -- 172.21.15.111:0/4189756034 shutdown_connections 2023-03-29T10:58:53.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.109+0000 7fb2575ff700 1 --2- 172.21.15.111:0/4189756034 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2500cfbf0 0x7fb2501608f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:53.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.109+0000 7fb2575ff700 1 -- 172.21.15.111:0/4189756034 >> 172.21.15.111:0/4189756034 conn(0x7fb250071b10 msgr2=0x7fb250071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:53.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.109+0000 7fb2575ff700 1 -- 172.21.15.111:0/4189756034 shutdown_connections 2023-03-29T10:58:53.109 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.109+0000 7fb2575ff700 1 -- 172.21.15.111:0/4189756034 wait complete. 2023-03-29T10:58:53.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.111+0000 7fb2575ff700 1 Processor -- start 2023-03-29T10:58:53.110 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.111+0000 7fb2575ff700 1 -- start start 2023-03-29T10:58:53.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.112+0000 7fb2575ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2500b87a0 0x7fb25015a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:53.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.112+0000 7fb2575ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2500cfbf0 0x7fb2500c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:53.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.112+0000 7fb2575ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2500d1100 0x7fb2501e6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:53.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.112+0000 7fb2575ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb25015a9e0 con 0x7fb2500b87a0 2023-03-29T10:58:53.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.112+0000 7fb2575ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2500c9490 con 0x7fb2500d1100 2023-03-29T10:58:53.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.112+0000 7fb2575ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2500c8ed0 con 0x7fb2500cfbf0 2023-03-29T10:58:53.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.113+0000 7fb25539b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2500b87a0 0x7fb25015a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:53.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.113+0000 7fb25539b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2500b87a0 0x7fb25015a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51746/0 (socket says 172.21.15.111:51746) 2023-03-29T10:58:53.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.113+0000 7fb25539b700 1 -- 172.21.15.111:0/1146922888 learned_addr learned my addr 172.21.15.111:0/1146922888 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:53.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.113+0000 7fb255b9c700 1 --2- 172.21.15.111:0/1146922888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2500d1100 0x7fb2501e6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:53.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.113+0000 7fb254b9a700 1 --2- 172.21.15.111:0/1146922888 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2500cfbf0 0x7fb2500c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:53.113 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.113+0000 7fb25539b700 1 -- 172.21.15.111:0/1146922888 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2500cfbf0 msgr2=0x7fb2500c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:53.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.113+0000 7fb25539b700 1 --2- 172.21.15.111:0/1146922888 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2500cfbf0 0x7fb2500c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:53.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.113+0000 7fb25539b700 1 -- 172.21.15.111:0/1146922888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2500d1100 msgr2=0x7fb2501e6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:53.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.113+0000 7fb25539b700 1 --2- 172.21.15.111:0/1146922888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2500d1100 0x7fb2501e6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:53.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.113+0000 7fb25539b700 1 -- 172.21.15.111:0/1146922888 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2400094a0 con 0x7fb2500b87a0 2023-03-29T10:58:53.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.113+0000 7fb25539b700 1 --2- 172.21.15.111:0/1146922888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2500b87a0 0x7fb25015a4a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb24c001eb0 tx=0x7fb24c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:53.114 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.114+0000 7fb245ffb700 1 -- 172.21.15.111:0/1146922888 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c005d70 con 0x7fb2500b87a0 2023-03-29T10:58:53.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.114+0000 7fb245ffb700 1 -- 172.21.15.111:0/1146922888 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb24c0070d0 con 0x7fb2500b87a0 2023-03-29T10:58:53.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.114+0000 7fb2575ff700 1 -- 172.21.15.111:0/1146922888 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb250160e30 con 0x7fb2500b87a0 2023-03-29T10:58:53.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.114+0000 7fb245ffb700 1 -- 172.21.15.111:0/1146922888 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb24c007460 con 0x7fb2500b87a0 2023-03-29T10:58:53.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.114+0000 7fb2575ff700 1 -- 172.21.15.111:0/1146922888 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2500cd980 con 0x7fb2500b87a0 2023-03-29T10:58:53.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.115+0000 7fb2575ff700 1 -- 172.21.15.111:0/1146922888 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb250160fc0 con 0x7fb2500b87a0 2023-03-29T10:58:53.115 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.116+0000 7fb245ffb700 1 -- 172.21.15.111:0/1146922888 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fb24c0075c0 con 0x7fb2500b87a0 2023-03-29T10:58:53.116 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.117+0000 7fb245ffb700 1 --2- 172.21.15.111:0/1146922888 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb234069de0 0x7fb23406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:53.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.117+0000 7fb245ffb700 1 -- 172.21.15.111:0/1146922888 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb24c087790 con 0x7fb2500b87a0 2023-03-29T10:58:53.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.117+0000 7fb254b9a700 1 --2- 172.21.15.111:0/1146922888 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb234069de0 0x7fb23406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:53.117 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.118+0000 7fb254b9a700 1 --2- 172.21.15.111:0/1146922888 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb234069de0 0x7fb23406c290 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb2400097c0 tx=0x7fb24001c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:53.121 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.121+0000 7fb245ffb700 1 -- 172.21.15.111:0/1146922888 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fb24c052c80 con 0x7fb2500b87a0 2023-03-29T10:58:53.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.313+0000 7fb2575ff700 1 -- 172.21.15.111:0/1146922888 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fb2500b52f0 con 0x7fb2500b87a0 2023-03-29T10:58:53.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.314+0000 7fb245ffb700 1 -- 172.21.15.111:0/1146922888 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7fb24c0562a0 con 0x7fb2500b87a0 2023-03-29T10:58:53.314 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:53.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.316+0000 7fb2575ff700 1 -- 172.21.15.111:0/1146922888 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb234069de0 msgr2=0x7fb23406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:53.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.316+0000 7fb2575ff700 1 --2- 172.21.15.111:0/1146922888 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb234069de0 0x7fb23406c290 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb2400097c0 tx=0x7fb24001c040 comp rx=0 tx=0).stop 2023-03-29T10:58:53.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.316+0000 7fb2575ff700 1 -- 172.21.15.111:0/1146922888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2500b87a0 msgr2=0x7fb25015a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:53.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.316+0000 7fb2575ff700 1 --2- 172.21.15.111:0/1146922888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2500b87a0 0x7fb25015a4a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb24c001eb0 tx=0x7fb24c00c7b0 comp rx=0 tx=0).stop 2023-03-29T10:58:53.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.317+0000 7fb2575ff700 1 -- 172.21.15.111:0/1146922888 shutdown_connections 2023-03-29T10:58:53.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.317+0000 7fb2575ff700 1 --2- 172.21.15.111:0/1146922888 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb234069de0 0x7fb23406c290 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:53.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.317+0000 7fb2575ff700 1 --2- 172.21.15.111:0/1146922888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb2500b87a0 0x7fb25015a4a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:53.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.317+0000 7fb2575ff700 1 --2- 172.21.15.111:0/1146922888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2500d1100 0x7fb2501e6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:53.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.317+0000 7fb2575ff700 1 --2- 172.21.15.111:0/1146922888 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb2500cfbf0 0x7fb2500c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:53.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.317+0000 7fb2575ff700 1 -- 172.21.15.111:0/1146922888 >> 172.21.15.111:0/1146922888 conn(0x7fb250071b10 msgr2=0x7fb2500b3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:53.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.317+0000 7fb2575ff700 1 -- 172.21.15.111:0/1146922888 shutdown_connections 2023-03-29T10:58:53.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.317+0000 7fb2575ff700 1 -- 172.21.15.111:0/1146922888 wait complete. 2023-03-29T10:58:53.334 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-03-29T10:58:53.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.518+0000 7fbd489ff700 1 Processor -- start 2023-03-29T10:58:53.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.518+0000 7fbd489ff700 1 -- start start 2023-03-29T10:58:53.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.519+0000 7fbd489ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbd440cf010 0x7fbd44160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:53.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.519+0000 7fbd489ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd440cd520 con 0x7fbd440cff50 2023-03-29T10:58:53.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.519+0000 7fbd489ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd440d2640 con 0x7fbd4415c590 2023-03-29T10:58:53.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.519+0000 7fbd489ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd441610d0 con 0x7fbd440cf010 2023-03-29T10:58:53.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.519+0000 7fbd3bfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbd440cf010 0x7fbd44160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:53.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.519+0000 7fbd42d9d700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fbd440cff50 0x7fbd440d2ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47432/0 (socket says 172.21.15.111:47432) 2023-03-29T10:58:53.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.519+0000 7fbd42d9d700 1 -- 172.21.15.111:0/2329152698 learned_addr learned my addr 172.21.15.111:0/2329152698 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:53.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.520+0000 7fbd3bfff700 1 -- 172.21.15.111:0/2329152698 >> v1:172.21.15.119:6789/0 conn(0x7fbd4415c590 legacy=0x7fbd440cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:53.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.520+0000 7fbd3bfff700 1 -- 172.21.15.111:0/2329152698 >> v1:172.21.15.111:6789/0 conn(0x7fbd440cff50 legacy=0x7fbd440d2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:53.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.520+0000 7fbd3bfff700 1 -- 172.21.15.111:0/2329152698 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd440b23c0 con 0x7fbd440cf010 2023-03-29T10:58:53.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.520+0000 7fbd3bfff700 1 --2- 172.21.15.111:0/2329152698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd440cf010 0x7fbd44160b90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009840 tx=0x7fbd2c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac20bfdb03a7440e server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:53.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.520+0000 7fbd41d9b700 1 -- 172.21.15.111:0/2329152698 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c011830 con 0x7fbd440cf010 2023-03-29T10:58:53.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.520+0000 7fbd41d9b700 1 -- 172.21.15.111:0/2329152698 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2c011990 con 0x7fbd440cf010 2023-03-29T10:58:53.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.520+0000 7fbd41d9b700 1 -- 172.21.15.111:0/2329152698 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2c011d20 con 0x7fbd440cf010 2023-03-29T10:58:53.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.521+0000 7fbd489ff700 1 -- 172.21.15.111:0/2329152698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd440cf010 msgr2=0x7fbd44160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:53.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.521+0000 7fbd489ff700 1 --2- 172.21.15.111:0/2329152698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd440cf010 0x7fbd44160b90 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c009840 tx=0x7fbd2c009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:53.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.521+0000 7fbd489ff700 1 -- 172.21.15.111:0/2329152698 shutdown_connections 2023-03-29T10:58:53.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.521+0000 7fbd489ff700 1 --2- 172.21.15.111:0/2329152698 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd440cf010 0x7fbd44160b90 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:53.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.521+0000 7fbd489ff700 1 -- 172.21.15.111:0/2329152698 >> 172.21.15.111:0/2329152698 conn(0x7fbd44071b10 msgr2=0x7fbd44071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:53.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.521+0000 7fbd489ff700 1 -- 172.21.15.111:0/2329152698 shutdown_connections 2023-03-29T10:58:53.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.521+0000 7fbd489ff700 1 -- 172.21.15.111:0/2329152698 wait complete. 2023-03-29T10:58:53.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.522+0000 7fbd489ff700 1 Processor -- start 2023-03-29T10:58:53.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.523+0000 7fbd489ff700 1 -- start start 2023-03-29T10:58:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.523+0000 7fbd489ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbd440cf010 0x7fbd440b7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.523+0000 7fbd489ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd440cff50 0x7fbd440c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.523+0000 7fbd489ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd4415c590 0x7fbd441e6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.523+0000 7fbd489ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd440b8910 con 0x7fbd4415c590 2023-03-29T10:58:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.523+0000 7fbd489ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd440b7d00 con 0x7fbd440cff50 2023-03-29T10:58:53.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.523+0000 7fbd489ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd440b7fe0 con 0x7fbd440cf010 2023-03-29T10:58:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.524+0000 7fbd42d9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd4415c590 0x7fbd441e6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.524+0000 7fbd42d9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd4415c590 0x7fbd441e6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51760/0 (socket says 172.21.15.111:51760) 2023-03-29T10:58:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.524+0000 7fbd42d9d700 1 -- 172.21.15.111:0/649635890 learned_addr learned my addr 172.21.15.111:0/649635890 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.524+0000 7fbd3bfff700 1 --2- 172.21.15.111:0/649635890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd440cff50 0x7fbd440c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:53.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.524+0000 7fbd4259c700 1 --2- 172.21.15.111:0/649635890 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbd440cf010 0x7fbd440b7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.524+0000 7fbd42d9d700 1 -- 172.21.15.111:0/649635890 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbd440cf010 msgr2=0x7fbd440b7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.524+0000 7fbd42d9d700 1 --2- 172.21.15.111:0/649635890 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbd440cf010 0x7fbd440b7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.524+0000 7fbd42d9d700 1 -- 172.21.15.111:0/649635890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd440cff50 msgr2=0x7fbd440c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.524+0000 7fbd42d9d700 1 --2- 172.21.15.111:0/649635890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd440cff50 0x7fbd440c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.524+0000 7fbd42d9d700 1 -- 172.21.15.111:0/649635890 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd2c0094a0 con 0x7fbd4415c590 2023-03-29T10:58:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.524+0000 7fbd3bfff700 1 --2- 172.21.15.111:0/649635890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd440cff50 0x7fbd440c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:58:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.524+0000 7fbd42d9d700 1 --2- 172.21.15.111:0/649635890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd4415c590 0x7fbd441e6d30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00cbc0 tx=0x7fbd3c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.525+0000 7fbd3affd700 1 -- 172.21.15.111:0/649635890 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c00a9e0 con 0x7fbd4415c590 2023-03-29T10:58:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.525+0000 7fbd3affd700 1 -- 172.21.15.111:0/649635890 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd3c00ab40 con 0x7fbd4415c590 2023-03-29T10:58:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.525+0000 7fbd3affd700 1 -- 172.21.15.111:0/649635890 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c012400 con 0x7fbd4415c590 2023-03-29T10:58:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.525+0000 7fbd489ff700 1 -- 172.21.15.111:0/649635890 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd440cd770 con 0x7fbd4415c590 2023-03-29T10:58:53.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.525+0000 7fbd489ff700 1 -- 172.21.15.111:0/649635890 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd440cf560 con 0x7fbd4415c590 2023-03-29T10:58:53.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.526+0000 7fbd489ff700 1 -- 172.21.15.111:0/649635890 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd4415c960 con 0x7fbd4415c590 2023-03-29T10:58:53.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.527+0000 7fbd3affd700 1 -- 172.21.15.111:0/649635890 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fbd3c01b3b0 con 0x7fbd4415c590 2023-03-29T10:58:53.527 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.528+0000 7fbd3affd700 1 --2- 172.21.15.111:0/649635890 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbd24069de0 0x7fbd2406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:53.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.528+0000 7fbd3affd700 1 -- 172.21.15.111:0/649635890 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbd3c087e30 con 0x7fbd4415c590 2023-03-29T10:58:53.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.528+0000 7fbd4259c700 1 --2- 172.21.15.111:0/649635890 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbd24069de0 0x7fbd2406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:53.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.529+0000 7fbd4259c700 1 --2- 172.21.15.111:0/649635890 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbd24069de0 0x7fbd2406c290 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbd34007fd0 tx=0x7fbd34009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:53.532 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.532+0000 7fbd3affd700 1 -- 172.21.15.111:0/649635890 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fbd3c04c160 con 0x7fbd4415c590 2023-03-29T10:58:53.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.740+0000 7fbd489ff700 1 -- 172.21.15.111:0/649635890 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbd440b8a50 con 0x7fbd4415c590 2023-03-29T10:58:53.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.740+0000 7fbd3affd700 1 -- 172.21.15.111:0/649635890 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbd3c053400 con 0x7fbd4415c590 2023-03-29T10:58:53.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.743+0000 7fbd489ff700 1 -- 172.21.15.111:0/649635890 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbd24069de0 msgr2=0x7fbd2406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:53.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.743+0000 7fbd489ff700 1 --2- 172.21.15.111:0/649635890 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbd24069de0 0x7fbd2406c290 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbd34007fd0 tx=0x7fbd34009040 comp rx=0 tx=0).stop 2023-03-29T10:58:53.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.743+0000 7fbd489ff700 1 -- 172.21.15.111:0/649635890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd4415c590 msgr2=0x7fbd441e6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:53.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.743+0000 7fbd489ff700 1 --2- 172.21.15.111:0/649635890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd4415c590 0x7fbd441e6d30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00cbc0 tx=0x7fbd3c00ced0 comp rx=0 tx=0).stop 2023-03-29T10:58:53.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.743+0000 7fbd489ff700 1 -- 172.21.15.111:0/649635890 shutdown_connections 2023-03-29T10:58:53.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.743+0000 7fbd489ff700 1 --2- 172.21.15.111:0/649635890 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbd24069de0 0x7fbd2406c290 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:53.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.743+0000 7fbd489ff700 1 --2- 172.21.15.111:0/649635890 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbd4415c590 0x7fbd441e6d30 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:53.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.743+0000 7fbd489ff700 1 --2- 172.21.15.111:0/649635890 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbd440cf010 0x7fbd440b7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:53.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.743+0000 7fbd489ff700 1 --2- 172.21.15.111:0/649635890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd440cff50 0x7fbd440c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:53.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.743+0000 7fbd489ff700 1 -- 172.21.15.111:0/649635890 >> 172.21.15.111:0/649635890 conn(0x7fbd44071b10 msgr2=0x7fbd440b4270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:53.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.743+0000 7fbd489ff700 1 -- 172.21.15.111:0/649635890 shutdown_connections 2023-03-29T10:58:53.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:53.743+0000 7fbd489ff700 1 -- 172.21.15.111:0/649635890 wait complete. 2023-03-29T10:58:53.744 INFO:teuthology.orchestra.run.smithi111.stderr:listed 0 entries 2023-03-29T10:58:53.759 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi119.front.sepia.ceph.com... 2023-03-29T10:58:53.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:53.760 DEBUG:teuthology.orchestra.run.smithi119:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-03-29T10:58:53.828 INFO:teuthology.orchestra.run:waiting for 300 2023-03-29T10:58:53.828 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-03-29T10:58:53.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:58:53.829 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-03-29T10:58:53.871 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-03-29T10:58:53.871 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-29T10:58:54.054 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.055+0000 7ff6a61ff700 1 Processor -- start 2023-03-29T10:58:54.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.055+0000 7ff6a61ff700 1 -- start start 2023-03-29T10:58:54.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.056+0000 7ff6a61ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6a015fea0 0x7ff6a00ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:54.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.056+0000 7ff6a61ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff6a00cede0 0x7ff6a00b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:54.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.056+0000 7ff6a61ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a006adf0 con 0x7ff6a00af620 2023-03-29T10:58:54.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.056+0000 7ff6a61ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a00cf260 con 0x7ff6a00cede0 2023-03-29T10:58:54.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.056+0000 7ff6a61ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a006b320 con 0x7ff6a015fea0 2023-03-29T10:58:54.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.056+0000 7ff69ffff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7ff6a00cede0 0x7ff6a00b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:54.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.056+0000 7ff697fff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7ff6a00af620 0x7ff6a00ce6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47434/0 (socket says 172.21.15.111:47434) 2023-03-29T10:58:54.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.056+0000 7ff697fff700 1 -- 172.21.15.111:0/1129049909 learned_addr learned my addr 172.21.15.111:0/1129049909 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:54.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.056+0000 7ff69f7fe700 1 --2- 172.21.15.111:0/1129049909 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6a015fea0 0x7ff6a00ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:54.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.057+0000 7ff69ffff700 1 -- 172.21.15.111:0/1129049909 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6a015fea0 msgr2=0x7ff6a00ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:54.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.057+0000 7ff69ffff700 1 --2- 172.21.15.111:0/1129049909 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6a015fea0 0x7ff6a00ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.057+0000 7ff69ffff700 1 -- 172.21.15.111:0/1129049909 >> v1:172.21.15.111:6789/0 conn(0x7ff6a00af620 legacy=0x7ff6a00ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:54.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.057+0000 7ff69ffff700 1 -- 172.21.15.111:0/1129049909 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a006b040 con 0x7ff6a00cede0 2023-03-29T10:58:54.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.057+0000 7ff69f7fe700 1 --2- 172.21.15.111:0/1129049909 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6a015fea0 0x7ff6a00ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:54.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.057+0000 7ff69effd700 1 -- 172.21.15.111:0/1129049909 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3683864243 0 0) 0x7ff6a006adf0 con 0x7ff6a00af620 2023-03-29T10:58:54.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.057+0000 7ff69ffff700 1 --2- 172.21.15.111:0/1129049909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6a00cede0 0x7ff6a00b01c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff698009d90 tx=0x7ff69800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95cef43634cce077 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:54.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.057+0000 7ff69effd700 1 -- 172.21.15.111:0/1129049909 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69800e070 con 0x7ff6a00cede0 2023-03-29T10:58:54.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.057+0000 7ff69effd700 1 -- 172.21.15.111:0/1129049909 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69800bbd0 con 0x7ff6a00cede0 2023-03-29T10:58:54.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.057+0000 7ff69effd700 1 -- 172.21.15.111:0/1129049909 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff698012430 con 0x7ff6a00cede0 2023-03-29T10:58:54.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.058+0000 7ff6a61ff700 1 -- 172.21.15.111:0/1129049909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6a00cede0 msgr2=0x7ff6a00b01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:54.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.058+0000 7ff6a61ff700 1 --2- 172.21.15.111:0/1129049909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6a00cede0 0x7ff6a00b01c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff698009d90 tx=0x7ff69800b330 comp rx=0 tx=0).stop 2023-03-29T10:58:54.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.058+0000 7ff6a61ff700 1 -- 172.21.15.111:0/1129049909 shutdown_connections 2023-03-29T10:58:54.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.058+0000 7ff6a61ff700 1 --2- 172.21.15.111:0/1129049909 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6a00cede0 0x7ff6a00b01c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.058+0000 7ff6a61ff700 1 --2- 172.21.15.111:0/1129049909 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6a015fea0 0x7ff6a00ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.058+0000 7ff6a61ff700 1 -- 172.21.15.111:0/1129049909 >> 172.21.15.111:0/1129049909 conn(0x7ff6a0099050 msgr2=0x7ff6a00cfcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:54.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.058+0000 7ff6a61ff700 1 -- 172.21.15.111:0/1129049909 shutdown_connections 2023-03-29T10:58:54.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.058+0000 7ff6a61ff700 1 -- 172.21.15.111:0/1129049909 wait complete. 2023-03-29T10:58:54.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.059+0000 7ff6a61ff700 1 Processor -- start 2023-03-29T10:58:54.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.060+0000 7ff6a61ff700 1 -- start start 2023-03-29T10:58:54.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.061+0000 7ff6a61ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a00af620 0x7ff6a0149180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:54.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.061+0000 7ff6a61ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6a00cede0 0x7ff6a01f2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:54.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.061+0000 7ff6a61ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6a015fea0 0x7ff6a00ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:54.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.061+0000 7ff6a61ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a01496c0 con 0x7ff6a00cede0 2023-03-29T10:58:54.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.061+0000 7ff6a61ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a015c580 con 0x7ff6a00af620 2023-03-29T10:58:54.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.061+0000 7ff6a61ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a0156e50 con 0x7ff6a015fea0 2023-03-29T10:58:54.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.061+0000 7ff697fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6a00cede0 0x7ff6a01f2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:54.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.061+0000 7ff697fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6a00cede0 0x7ff6a01f2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51774/0 (socket says 172.21.15.111:51774) 2023-03-29T10:58:54.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.061+0000 7ff697fff700 1 -- 172.21.15.111:0/3764554516 learned_addr learned my addr 172.21.15.111:0/3764554516 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:54.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.061+0000 7ff69f7fe700 1 --2- 172.21.15.111:0/3764554516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a00af620 0x7ff6a0149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:54.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.061+0000 7ff69ffff700 1 --2- 172.21.15.111:0/3764554516 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6a015fea0 0x7ff6a00ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:54.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.061+0000 7ff697fff700 1 -- 172.21.15.111:0/3764554516 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6a015fea0 msgr2=0x7ff6a00ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:54.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.061+0000 7ff697fff700 1 --2- 172.21.15.111:0/3764554516 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6a015fea0 0x7ff6a00ad7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.061+0000 7ff697fff700 1 -- 172.21.15.111:0/3764554516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a00af620 msgr2=0x7ff6a0149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:54.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.061+0000 7ff697fff700 1 --2- 172.21.15.111:0/3764554516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a00af620 0x7ff6a0149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.061+0000 7ff697fff700 1 -- 172.21.15.111:0/3764554516 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff698009a70 con 0x7ff6a00cede0 2023-03-29T10:58:54.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.061+0000 7ff69f7fe700 1 --2- 172.21.15.111:0/3764554516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a00af620 0x7ff6a0149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:54.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.061+0000 7ff697fff700 1 --2- 172.21.15.111:0/3764554516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6a00cede0 0x7ff6a01f2620 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff68800c550 tx=0x7ff68800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:54.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.062+0000 7ff69cff9700 1 -- 172.21.15.111:0/3764554516 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688004c80 con 0x7ff6a00cede0 2023-03-29T10:58:54.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.062+0000 7ff69cff9700 1 -- 172.21.15.111:0/3764554516 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff688004e00 con 0x7ff6a00cede0 2023-03-29T10:58:54.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.062+0000 7ff69cff9700 1 -- 172.21.15.111:0/3764554516 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688007310 con 0x7ff6a00cede0 2023-03-29T10:58:54.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.062+0000 7ff6a61ff700 1 -- 172.21.15.111:0/3764554516 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a00cc9a0 con 0x7ff6a00cede0 2023-03-29T10:58:54.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.062+0000 7ff6a61ff700 1 -- 172.21.15.111:0/3764554516 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a00cc6c0 con 0x7ff6a00cede0 2023-03-29T10:58:54.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.063+0000 7ff6a61ff700 1 -- 172.21.15.111:0/3764554516 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a015c730 con 0x7ff6a00cede0 2023-03-29T10:58:54.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.064+0000 7ff69cff9700 1 -- 172.21.15.111:0/3764554516 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7ff688007470 con 0x7ff6a00cede0 2023-03-29T10:58:54.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.065+0000 7ff69cff9700 1 --2- 172.21.15.111:0/3764554516 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff680069de0 0x7ff68006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:54.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.065+0000 7ff69cff9700 1 -- 172.21.15.111:0/3764554516 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff688087f50 con 0x7ff6a00cede0 2023-03-29T10:58:54.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.065+0000 7ff69f7fe700 1 --2- 172.21.15.111:0/3764554516 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff680069de0 0x7ff68006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:54.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.066+0000 7ff69f7fe700 1 --2- 172.21.15.111:0/3764554516 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff680069de0 0x7ff68006c290 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff6900099f0 tx=0x7ff690008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:54.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.069+0000 7ff69cff9700 1 -- 172.21.15.111:0/3764554516 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7ff6a015c730 con 0x7ff6a00cede0 2023-03-29T10:58:54.306 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.307+0000 7ff6a61ff700 1 -- 172.21.15.111:0/3764554516 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6a015c6f0 con 0x7ff6a00cede0 2023-03-29T10:58:54.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.307+0000 7ff69cff9700 1 -- 172.21.15.111:0/3764554516 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7ff6880534c0 con 0x7ff6a00cede0 2023-03-29T10:58:54.307 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:54.307 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/556826332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":556826332},{"type":"v1","addr":"172.21.15.111:6837","nonce":556826332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/814623168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":814623168},{"type":"v1","addr":"172.21.15.111:6835","nonce":814623168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/514422035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":514422035},{"type":"v1","addr":"172.21.15.119:6834","nonce":514422035}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-29T10:57:27.824135+0000","modified":"2023-03-29T10:57:53.367898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/3160800581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":3160800581},{"type":"v1","addr":"172.21.15.119:6835","nonce":3160800581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-03-29T10:58:54.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.309+0000 7ff6a61ff700 1 -- 172.21.15.111:0/3764554516 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff680069de0 msgr2=0x7ff68006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:54.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.309+0000 7ff6a61ff700 1 --2- 172.21.15.111:0/3764554516 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff680069de0 0x7ff68006c290 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff6900099f0 tx=0x7ff690008040 comp rx=0 tx=0).stop 2023-03-29T10:58:54.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.309+0000 7ff6a61ff700 1 -- 172.21.15.111:0/3764554516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6a00cede0 msgr2=0x7ff6a01f2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:54.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.309+0000 7ff6a61ff700 1 --2- 172.21.15.111:0/3764554516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6a00cede0 0x7ff6a01f2620 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff68800c550 tx=0x7ff68800c860 comp rx=0 tx=0).stop 2023-03-29T10:58:54.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.310+0000 7ff6a61ff700 1 -- 172.21.15.111:0/3764554516 shutdown_connections 2023-03-29T10:58:54.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.310+0000 7ff6a61ff700 1 --2- 172.21.15.111:0/3764554516 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7ff680069de0 0x7ff68006c290 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.310+0000 7ff6a61ff700 1 --2- 172.21.15.111:0/3764554516 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7ff6a015fea0 0x7ff6a00ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.310+0000 7ff6a61ff700 1 --2- 172.21.15.111:0/3764554516 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6a00af620 0x7ff6a0149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.310+0000 7ff6a61ff700 1 --2- 172.21.15.111:0/3764554516 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7ff6a00cede0 0x7ff6a01f2620 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.310+0000 7ff6a61ff700 1 -- 172.21.15.111:0/3764554516 >> 172.21.15.111:0/3764554516 conn(0x7ff6a0099050 msgr2=0x7ff6a00b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:54.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.310+0000 7ff6a61ff700 1 -- 172.21.15.111:0/3764554516 shutdown_connections 2023-03-29T10:58:54.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.310+0000 7ff6a61ff700 1 -- 172.21.15.111:0/3764554516 wait complete. 2023-03-29T10:58:54.310 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2023-03-29T10:58:54.326 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:58:54.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.509+0000 7fda837ff700 1 Processor -- start 2023-03-29T10:58:54.509 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.509+0000 7fda837ff700 1 -- start start 2023-03-29T10:58:54.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.510+0000 7fda837ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda7c15db70 0x7fda7c0cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:54.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.510+0000 7fda837ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fda7c0d2ea0 0x7fda7c0d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:54.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.510+0000 7fda837ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda7c0d3860 con 0x7fda7c0b2440 2023-03-29T10:58:54.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.510+0000 7fda837ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda7c0b2810 con 0x7fda7c0d2ea0 2023-03-29T10:58:54.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.510+0000 7fda837ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c15ce30 con 0x7fda7c15db70 2023-03-29T10:58:54.510 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.511+0000 7fda81d9c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fda7c0d2ea0 0x7fda7c0d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:54.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.511+0000 7fda8159b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fda7c0b2440 0x7fda7c160e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47442/0 (socket says 172.21.15.111:47442) 2023-03-29T10:58:54.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.511+0000 7fda8159b700 1 -- 172.21.15.111:0/2341490428 learned_addr learned my addr 172.21.15.111:0/2341490428 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:54.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.511+0000 7fda80d9a700 1 --2- 172.21.15.111:0/2341490428 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda7c15db70 0x7fda7c0cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:54.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.511+0000 7fda81d9c700 1 -- 172.21.15.111:0/2341490428 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda7c15db70 msgr2=0x7fda7c0cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:54.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.511+0000 7fda81d9c700 1 --2- 172.21.15.111:0/2341490428 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda7c15db70 0x7fda7c0cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.511+0000 7fda81d9c700 1 -- 172.21.15.111:0/2341490428 >> v1:172.21.15.111:6789/0 conn(0x7fda7c0b2440 legacy=0x7fda7c160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:54.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.511+0000 7fda81d9c700 1 -- 172.21.15.111:0/2341490428 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7c0cdee0 con 0x7fda7c0d2ea0 2023-03-29T10:58:54.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.511+0000 7fda73fff700 1 -- 172.21.15.111:0/2341490428 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 564148868 0 0) 0x7fda7c0d3860 con 0x7fda7c0b2440 2023-03-29T10:58:54.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.511+0000 7fda81d9c700 1 --2- 172.21.15.111:0/2341490428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda7c0d2ea0 0x7fda7c0d3320 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fda74009d90 tx=0x7fda7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fdb43ad6904c193 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:54.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.512+0000 7fda73fff700 1 -- 172.21.15.111:0/2341490428 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7400e070 con 0x7fda7c0d2ea0 2023-03-29T10:58:54.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.512+0000 7fda73fff700 1 -- 172.21.15.111:0/2341490428 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7400bb20 con 0x7fda7c0d2ea0 2023-03-29T10:58:54.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.512+0000 7fda73fff700 1 -- 172.21.15.111:0/2341490428 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74012430 con 0x7fda7c0d2ea0 2023-03-29T10:58:54.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.512+0000 7fda837ff700 1 -- 172.21.15.111:0/2341490428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda7c0d2ea0 msgr2=0x7fda7c0d3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:54.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.512+0000 7fda837ff700 1 --2- 172.21.15.111:0/2341490428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda7c0d2ea0 0x7fda7c0d3320 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fda74009d90 tx=0x7fda7400b330 comp rx=0 tx=0).stop 2023-03-29T10:58:54.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.512+0000 7fda837ff700 1 -- 172.21.15.111:0/2341490428 shutdown_connections 2023-03-29T10:58:54.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.512+0000 7fda837ff700 1 --2- 172.21.15.111:0/2341490428 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda7c0d2ea0 0x7fda7c0d3320 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.512+0000 7fda837ff700 1 --2- 172.21.15.111:0/2341490428 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda7c15db70 0x7fda7c0cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.512+0000 7fda837ff700 1 -- 172.21.15.111:0/2341490428 >> 172.21.15.111:0/2341490428 conn(0x7fda7c071b10 msgr2=0x7fda7c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:54.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.512+0000 7fda837ff700 1 -- 172.21.15.111:0/2341490428 shutdown_connections 2023-03-29T10:58:54.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.512+0000 7fda837ff700 1 -- 172.21.15.111:0/2341490428 wait complete. 2023-03-29T10:58:54.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.514+0000 7fda837ff700 1 Processor -- start 2023-03-29T10:58:54.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.514+0000 7fda837ff700 1 -- start start 2023-03-29T10:58:54.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.515+0000 7fda837ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda7c0b2440 0x7fda7c14b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:54.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.515+0000 7fda837ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda7c0d2ea0 0x7fda7c1e73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:54.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.515+0000 7fda837ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda7c15db70 0x7fda7c1e78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:54.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.515+0000 7fda837ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c0caca0 con 0x7fda7c15db70 2023-03-29T10:58:54.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.515+0000 7fda837ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c0ca0d0 con 0x7fda7c0d2ea0 2023-03-29T10:58:54.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.515+0000 7fda837ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c0ca370 con 0x7fda7c0b2440 2023-03-29T10:58:54.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.515+0000 7fda81d9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda7c15db70 0x7fda7c1e78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:54.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.515+0000 7fda81d9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda7c15db70 0x7fda7c1e78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51800/0 (socket says 172.21.15.111:51800) 2023-03-29T10:58:54.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.515+0000 7fda81d9c700 1 -- 172.21.15.111:0/2048415311 learned_addr learned my addr 172.21.15.111:0/2048415311 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:54.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.516+0000 7fda80d9a700 1 --2- 172.21.15.111:0/2048415311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda7c0d2ea0 0x7fda7c1e73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:54.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.516+0000 7fda8159b700 1 --2- 172.21.15.111:0/2048415311 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda7c0b2440 0x7fda7c14b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:54.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.516+0000 7fda81d9c700 1 -- 172.21.15.111:0/2048415311 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda7c0b2440 msgr2=0x7fda7c14b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:54.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.516+0000 7fda81d9c700 1 --2- 172.21.15.111:0/2048415311 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda7c0b2440 0x7fda7c14b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.516+0000 7fda81d9c700 1 -- 172.21.15.111:0/2048415311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda7c0d2ea0 msgr2=0x7fda7c1e73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:54.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.516+0000 7fda81d9c700 1 --2- 172.21.15.111:0/2048415311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda7c0d2ea0 0x7fda7c1e73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.516+0000 7fda81d9c700 1 -- 172.21.15.111:0/2048415311 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda74009a70 con 0x7fda7c15db70 2023-03-29T10:58:54.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.516+0000 7fda81d9c700 1 --2- 172.21.15.111:0/2048415311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda7c15db70 0x7fda7c1e78e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fda7c0d0ae0 tx=0x7fda74004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:54.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.516+0000 7fda71ffb700 1 -- 172.21.15.111:0/2048415311 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74012430 con 0x7fda7c15db70 2023-03-29T10:58:54.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.516+0000 7fda71ffb700 1 -- 172.21.15.111:0/2048415311 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda74020070 con 0x7fda7c15db70 2023-03-29T10:58:54.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.516+0000 7fda71ffb700 1 -- 172.21.15.111:0/2048415311 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda74004280 con 0x7fda7c15db70 2023-03-29T10:58:54.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.516+0000 7fda837ff700 1 -- 172.21.15.111:0/2048415311 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda7c15d090 con 0x7fda7c15db70 2023-03-29T10:58:54.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.517+0000 7fda837ff700 1 -- 172.21.15.111:0/2048415311 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda7c0cdc00 con 0x7fda7c15db70 2023-03-29T10:58:54.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.517+0000 7fda837ff700 1 -- 172.21.15.111:0/2048415311 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7c15d370 con 0x7fda7c15db70 2023-03-29T10:58:54.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.518+0000 7fda71ffb700 1 -- 172.21.15.111:0/2048415311 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fda740043e0 con 0x7fda7c15db70 2023-03-29T10:58:54.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.519+0000 7fda71ffb700 1 --2- 172.21.15.111:0/2048415311 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fda60069de0 0x7fda6006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:54.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.519+0000 7fda71ffb700 1 -- 172.21.15.111:0/2048415311 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fda74022070 con 0x7fda7c15db70 2023-03-29T10:58:54.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.519+0000 7fda8159b700 1 --2- 172.21.15.111:0/2048415311 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fda60069de0 0x7fda6006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:54.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.520+0000 7fda8159b700 1 --2- 172.21.15.111:0/2048415311 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fda60069de0 0x7fda6006c290 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fda78000f60 tx=0x7fda78007be0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:54.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.523+0000 7fda71ffb700 1 -- 172.21.15.111:0/2048415311 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fda74055690 con 0x7fda7c15db70 2023-03-29T10:58:54.719 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.720+0000 7fda837ff700 1 -- 172.21.15.111:0/2048415311 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda7c074430 con 0x7fda7c15db70 2023-03-29T10:58:54.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.721+0000 7fda71ffb700 1 -- 172.21.15.111:0/2048415311 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19892 (secure 0 0 0) 0x7fda74058cb0 con 0x7fda7c15db70 2023-03-29T10:58:54.721 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:54.721 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":30,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:58:09.476334+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.26","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:58:54.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.723+0000 7fda837ff700 1 -- 172.21.15.111:0/2048415311 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fda60069de0 msgr2=0x7fda6006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:54.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.723+0000 7fda837ff700 1 --2- 172.21.15.111:0/2048415311 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fda60069de0 0x7fda6006c290 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fda78000f60 tx=0x7fda78007be0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.723+0000 7fda837ff700 1 -- 172.21.15.111:0/2048415311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda7c15db70 msgr2=0x7fda7c1e78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:54.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.723+0000 7fda837ff700 1 --2- 172.21.15.111:0/2048415311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda7c15db70 0x7fda7c1e78e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fda7c0d0ae0 tx=0x7fda74004ad0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.723+0000 7fda837ff700 1 -- 172.21.15.111:0/2048415311 shutdown_connections 2023-03-29T10:58:54.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.723+0000 7fda837ff700 1 --2- 172.21.15.111:0/2048415311 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fda60069de0 0x7fda6006c290 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.723+0000 7fda837ff700 1 --2- 172.21.15.111:0/2048415311 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fda7c15db70 0x7fda7c1e78e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.723 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.723+0000 7fda837ff700 1 --2- 172.21.15.111:0/2048415311 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fda7c0b2440 0x7fda7c14b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.723+0000 7fda837ff700 1 --2- 172.21.15.111:0/2048415311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fda7c0d2ea0 0x7fda7c1e73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.723+0000 7fda837ff700 1 -- 172.21.15.111:0/2048415311 >> 172.21.15.111:0/2048415311 conn(0x7fda7c071b10 msgr2=0x7fda7c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:54.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.723+0000 7fda837ff700 1 -- 172.21.15.111:0/2048415311 shutdown_connections 2023-03-29T10:58:54.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.724+0000 7fda837ff700 1 -- 172.21.15.111:0/2048415311 wait complete. 2023-03-29T10:58:54.740 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-29T10:57:16.183979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-29T10:57:21.239809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-03-29T10:57:26.192124+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-03-29T10:57:26.796777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-03-29T10:57:29.874795+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-03-29T10:58:54.741 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-29T10:58:54.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.928+0000 7fea0e7ff700 1 Processor -- start 2023-03-29T10:58:54.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.928+0000 7fea0e7ff700 1 -- start start 2023-03-29T10:58:54.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.929+0000 7fea0e7ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea080b87a0 0x7fea080cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:54.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.929+0000 7fea0e7ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fea081601e0 0x7fea08160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:54.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.929+0000 7fea0e7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea08160ff0 con 0x7fea08160b70 2023-03-29T10:58:54.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.929+0000 7fea0e7ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea080d02d0 con 0x7fea081601e0 2023-03-29T10:58:54.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.929+0000 7fea0e7ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea080d0cb0 con 0x7fea080b87a0 2023-03-29T10:58:54.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.930+0000 7fe9fffff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fea081601e0 0x7fea08160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:54.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.930+0000 7fe9fffff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fea081601e0 0x7fea08160630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51812/0 (socket says 172.21.15.111:51812) 2023-03-29T10:58:54.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.930+0000 7fe9fffff700 1 -- 172.21.15.111:0/973159441 learned_addr learned my addr 172.21.15.111:0/973159441 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:54.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.930+0000 7fea07fff700 1 --2- 172.21.15.111:0/973159441 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea080b87a0 0x7fea080cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:54.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.930+0000 7fe9fffff700 1 -- 172.21.15.111:0/973159441 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea080b87a0 msgr2=0x7fea080cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:54.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.930+0000 7fe9fffff700 1 --2- 172.21.15.111:0/973159441 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea080b87a0 0x7fea080cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.930+0000 7fe9fffff700 1 -- 172.21.15.111:0/973159441 >> v1:172.21.15.119:6789/0 conn(0x7fea08160b70 legacy=0x7fea080d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:54.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.930+0000 7fe9fffff700 1 -- 172.21.15.111:0/973159441 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea080cdc60 con 0x7fea081601e0 2023-03-29T10:58:54.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.930+0000 7fe9fffff700 1 --2- 172.21.15.111:0/973159441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea081601e0 0x7fea08160630 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00098f0 tx=0x7fe9f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e1be9b99cc591f server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:54.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.930+0000 7fea077fe700 1 -- 172.21.15.111:0/973159441 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f000b8a0 con 0x7fea081601e0 2023-03-29T10:58:54.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.931+0000 7fea077fe700 1 -- 172.21.15.111:0/973159441 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f000ba00 con 0x7fea081601e0 2023-03-29T10:58:54.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.931+0000 7fea077fe700 1 -- 172.21.15.111:0/973159441 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f000bd90 con 0x7fea081601e0 2023-03-29T10:58:54.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.931+0000 7fea0e7ff700 1 -- 172.21.15.111:0/973159441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea081601e0 msgr2=0x7fea08160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:54.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.931+0000 7fea0e7ff700 1 --2- 172.21.15.111:0/973159441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea081601e0 0x7fea08160630 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00098f0 tx=0x7fe9f0009c00 comp rx=0 tx=0).stop 2023-03-29T10:58:54.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.931+0000 7fea0e7ff700 1 -- 172.21.15.111:0/973159441 shutdown_connections 2023-03-29T10:58:54.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.931+0000 7fea0e7ff700 1 --2- 172.21.15.111:0/973159441 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea080b87a0 0x7fea080cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.931+0000 7fea0e7ff700 1 --2- 172.21.15.111:0/973159441 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea081601e0 0x7fea08160630 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.931+0000 7fea0e7ff700 1 -- 172.21.15.111:0/973159441 >> 172.21.15.111:0/973159441 conn(0x7fea08071b10 msgr2=0x7fea08071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:54.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.931+0000 7fea0e7ff700 1 -- 172.21.15.111:0/973159441 shutdown_connections 2023-03-29T10:58:54.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.931+0000 7fea0e7ff700 1 -- 172.21.15.111:0/973159441 wait complete. 2023-03-29T10:58:54.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.933+0000 7fea0e7ff700 1 Processor -- start 2023-03-29T10:58:54.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.933+0000 7fea0e7ff700 1 -- start start 2023-03-29T10:58:54.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.934+0000 7fea0e7ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea080b87a0 0x7fea08151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:54.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.934+0000 7fea0e7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea081601e0 0x7fea0815eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:54.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.934+0000 7fea0e7ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea08160b70 0x7fea080b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:54.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.934+0000 7fea0e7ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea080cb020 con 0x7fea08160b70 2023-03-29T10:58:54.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.934+0000 7fea0e7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea081515d0 con 0x7fea081601e0 2023-03-29T10:58:54.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.934+0000 7fea0e7ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea080ca410 con 0x7fea080b87a0 2023-03-29T10:58:54.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.934+0000 7fea0cd9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea08160b70 0x7fea080b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:54.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.934+0000 7fea0cd9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea08160b70 0x7fea080b6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51818/0 (socket says 172.21.15.111:51818) 2023-03-29T10:58:54.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.934+0000 7fea0cd9c700 1 -- 172.21.15.111:0/972236264 learned_addr learned my addr 172.21.15.111:0/972236264 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:54.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.934+0000 7fea07fff700 1 --2- 172.21.15.111:0/972236264 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea080b87a0 0x7fea08151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:54.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.934+0000 7fe9fffff700 1 --2- 172.21.15.111:0/972236264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea081601e0 0x7fea0815eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:54.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.935+0000 7fea0cd9c700 1 -- 172.21.15.111:0/972236264 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea080b87a0 msgr2=0x7fea08151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:54.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.935+0000 7fea0cd9c700 1 --2- 172.21.15.111:0/972236264 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea080b87a0 0x7fea08151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.935+0000 7fea0cd9c700 1 -- 172.21.15.111:0/972236264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea081601e0 msgr2=0x7fea0815eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:54.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.935+0000 7fea0cd9c700 1 --2- 172.21.15.111:0/972236264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea081601e0 0x7fea0815eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:54.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.935+0000 7fea0cd9c700 1 -- 172.21.15.111:0/972236264 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f0009550 con 0x7fea08160b70 2023-03-29T10:58:54.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.935+0000 7fea07fff700 1 --2- 172.21.15.111:0/972236264 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea080b87a0 0x7fea08151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:54.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.935+0000 7fea0cd9c700 1 --2- 172.21.15.111:0/972236264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea08160b70 0x7fea080b6500 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fea0000cb50 tx=0x7fea0000cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:54.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.935+0000 7fea057fa700 1 -- 172.21.15.111:0/972236264 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea00003c00 con 0x7fea08160b70 2023-03-29T10:58:54.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.935+0000 7fea057fa700 1 -- 172.21.15.111:0/972236264 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea00003d60 con 0x7fea08160b70 2023-03-29T10:58:54.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.935+0000 7fea057fa700 1 -- 172.21.15.111:0/972236264 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0000ab70 con 0x7fea08160b70 2023-03-29T10:58:54.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.935+0000 7fea0e7ff700 1 -- 172.21.15.111:0/972236264 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea080cd980 con 0x7fea08160b70 2023-03-29T10:58:54.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.935+0000 7fea0e7ff700 1 -- 172.21.15.111:0/972236264 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea080cd6a0 con 0x7fea08160b70 2023-03-29T10:58:54.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.936+0000 7fea0e7ff700 1 -- 172.21.15.111:0/972236264 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea080d2110 con 0x7fea08160b70 2023-03-29T10:58:54.936 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.937+0000 7fea057fa700 1 -- 172.21.15.111:0/972236264 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fea0000acd0 con 0x7fea08160b70 2023-03-29T10:58:54.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.938+0000 7fea057fa700 1 --2- 172.21.15.111:0/972236264 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe9e8069de0 0x7fe9e806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:54.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.938+0000 7fea057fa700 1 -- 172.21.15.111:0/972236264 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fea00088410 con 0x7fea08160b70 2023-03-29T10:58:54.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.938+0000 7fea07fff700 1 --2- 172.21.15.111:0/972236264 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe9e8069de0 0x7fe9e806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:54.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.939+0000 7fea07fff700 1 --2- 172.21.15.111:0/972236264 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe9e8069de0 0x7fe9e806c290 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fea0815e7f0 tx=0x7fe9f8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:54.942 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:54.942+0000 7fea057fa700 1 -- 172.21.15.111:0/972236264 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fea000538f0 con 0x7fea08160b70 2023-03-29T10:58:55.135 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.135+0000 7fea0e7ff700 1 -- 172.21.15.111:0/972236264 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fea08074430 con 0x7fea08160b70 2023-03-29T10:58:55.206 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.136+0000 7fea057fa700 1 -- 172.21.15.111:0/972236264 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fea00056f10 con 0x7fea08160b70 2023-03-29T10:58:55.206 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-03-29T10:58:55.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.138+0000 7fea0e7ff700 1 -- 172.21.15.111:0/972236264 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe9e8069de0 msgr2=0x7fe9e806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:55.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.138+0000 7fea0e7ff700 1 --2- 172.21.15.111:0/972236264 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe9e8069de0 0x7fe9e806c290 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fea0815e7f0 tx=0x7fe9f8008040 comp rx=0 tx=0).stop 2023-03-29T10:58:55.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.138+0000 7fea0e7ff700 1 -- 172.21.15.111:0/972236264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea08160b70 msgr2=0x7fea080b6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:55.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.138+0000 7fea0e7ff700 1 --2- 172.21.15.111:0/972236264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea08160b70 0x7fea080b6500 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fea0000cb50 tx=0x7fea0000cf10 comp rx=0 tx=0).stop 2023-03-29T10:58:55.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.138+0000 7fea0e7ff700 1 -- 172.21.15.111:0/972236264 shutdown_connections 2023-03-29T10:58:55.207 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.138+0000 7fea0e7ff700 1 --2- 172.21.15.111:0/972236264 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fe9e8069de0 0x7fe9e806c290 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:55.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.138+0000 7fea0e7ff700 1 --2- 172.21.15.111:0/972236264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea08160b70 0x7fea080b6500 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:55.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.138+0000 7fea0e7ff700 1 --2- 172.21.15.111:0/972236264 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea080b87a0 0x7fea08151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:55.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.138+0000 7fea0e7ff700 1 --2- 172.21.15.111:0/972236264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea081601e0 0x7fea0815eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:55.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.138+0000 7fea0e7ff700 1 -- 172.21.15.111:0/972236264 >> 172.21.15.111:0/972236264 conn(0x7fea08071b10 msgr2=0x7fea080b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:55.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.138+0000 7fea0e7ff700 1 -- 172.21.15.111:0/972236264 shutdown_connections 2023-03-29T10:58:55.208 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.138+0000 7fea0e7ff700 1 -- 172.21.15.111:0/972236264 wait complete. 2023-03-29T10:58:55.208 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-29T10:58:55.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.395+0000 7f42bdfff700 1 Processor -- start 2023-03-29T10:58:55.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.395+0000 7f42bdfff700 1 -- start start 2023-03-29T10:58:55.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.396+0000 7f42bdfff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f42b80b87a0 0x7f42b80cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:55.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.396+0000 7f42bdfff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b80cdc60 con 0x7f42b8160d40 2023-03-29T10:58:55.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.396+0000 7f42bdfff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b80cd6a0 con 0x7f42b81601e0 2023-03-29T10:58:55.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.396+0000 7f42bdfff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b80d02d0 con 0x7f42b80b87a0 2023-03-29T10:58:55.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.396+0000 7f42b77fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f42b80b87a0 0x7f42b80cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:55.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.396+0000 7f42b7fff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f42b8160d40 0x7f42b80d1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47446/0 (socket says 172.21.15.111:47446) 2023-03-29T10:58:55.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.396+0000 7f42b7fff700 1 -- 172.21.15.111:0/3450364086 learned_addr learned my addr 172.21.15.111:0/3450364086 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:55.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.397+0000 7f42b67fc700 1 -- 172.21.15.111:0/3450364086 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438401674 0 0) 0x7f42b80cdc60 con 0x7f42b8160d40 2023-03-29T10:58:55.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.397+0000 7f42b67fc700 1 -- 172.21.15.111:0/3450364086 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42a00035b0 con 0x7f42b8160d40 2023-03-29T10:58:55.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.397+0000 7f42b77fe700 1 -- 172.21.15.111:0/3450364086 >> v1:172.21.15.119:6789/0 conn(0x7f42b81601e0 legacy=0x7f42b8160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:55.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.397+0000 7f42b77fe700 1 -- 172.21.15.111:0/3450364086 >> v1:172.21.15.111:6789/0 conn(0x7f42b8160d40 legacy=0x7f42b80d1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:55.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.397+0000 7f42b77fe700 1 -- 172.21.15.111:0/3450364086 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b80cd980 con 0x7f42b80b87a0 2023-03-29T10:58:55.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.397+0000 7f42b77fe700 1 --2- 172.21.15.111:0/3450364086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b80b87a0 0x7f42b80cfbf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009840 tx=0x7f42a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488749f2b1f3c897 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:55.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.397+0000 7f42b67fc700 1 -- 172.21.15.111:0/3450364086 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a8011830 con 0x7f42b80b87a0 2023-03-29T10:58:55.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.397+0000 7f42b67fc700 1 -- 172.21.15.111:0/3450364086 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a8011990 con 0x7f42b80b87a0 2023-03-29T10:58:55.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.397+0000 7f42b67fc700 1 -- 172.21.15.111:0/3450364086 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a8011d20 con 0x7f42b80b87a0 2023-03-29T10:58:55.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.398+0000 7f42bdfff700 1 -- 172.21.15.111:0/3450364086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b80b87a0 msgr2=0x7f42b80cfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:55.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.398+0000 7f42bdfff700 1 --2- 172.21.15.111:0/3450364086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b80b87a0 0x7f42b80cfbf0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009840 tx=0x7f42a8009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:55.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.398+0000 7f42bdfff700 1 -- 172.21.15.111:0/3450364086 shutdown_connections 2023-03-29T10:58:55.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.398+0000 7f42bdfff700 1 --2- 172.21.15.111:0/3450364086 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b80b87a0 0x7f42b80cfbf0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:55.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.398+0000 7f42bdfff700 1 -- 172.21.15.111:0/3450364086 >> 172.21.15.111:0/3450364086 conn(0x7f42b8071b10 msgr2=0x7f42b8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:55.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.398+0000 7f42bdfff700 1 -- 172.21.15.111:0/3450364086 shutdown_connections 2023-03-29T10:58:55.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.398+0000 7f42bdfff700 1 -- 172.21.15.111:0/3450364086 wait complete. 2023-03-29T10:58:55.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.400+0000 7f42bdfff700 1 Processor -- start 2023-03-29T10:58:55.399 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.400+0000 7f42bdfff700 1 -- start start 2023-03-29T10:58:55.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.401+0000 7f42bdfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42b80b87a0 0x7f42b81544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:55.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.401+0000 7f42bdfff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f42b81601e0 0x7f42b80cba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:55.400 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.401+0000 7f42bdfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b8160d40 0x7f42b80c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:55.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.401+0000 7f42bdfff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b80ca9d0 con 0x7f42b8160d40 2023-03-29T10:58:55.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.401+0000 7f42bdfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b80cacb0 con 0x7f42b80b87a0 2023-03-29T10:58:55.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.401+0000 7f42bdfff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b80caf90 con 0x7f42b81601e0 2023-03-29T10:58:55.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.401+0000 7f42b7fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b8160d40 0x7f42b80c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:55.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.401+0000 7f42b7fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b8160d40 0x7f42b80c9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51834/0 (socket says 172.21.15.111:51834) 2023-03-29T10:58:55.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.401+0000 7f42b7fff700 1 -- 172.21.15.111:0/4015322074 learned_addr learned my addr 172.21.15.111:0/4015322074 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:55.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.401+0000 7f42b77fe700 1 --2- 172.21.15.111:0/4015322074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42b80b87a0 0x7f42b81544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:55.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.401+0000 7f42b6ffd700 1 --2- 172.21.15.111:0/4015322074 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f42b81601e0 0x7f42b80cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:55.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.401+0000 7f42b7fff700 1 -- 172.21.15.111:0/4015322074 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f42b81601e0 msgr2=0x7f42b80cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:55.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.401+0000 7f42b7fff700 1 --2- 172.21.15.111:0/4015322074 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f42b81601e0 0x7f42b80cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:55.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.401+0000 7f42b7fff700 1 -- 172.21.15.111:0/4015322074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42b80b87a0 msgr2=0x7f42b81544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:55.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.401+0000 7f42b7fff700 1 --2- 172.21.15.111:0/4015322074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42b80b87a0 0x7f42b81544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:55.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.401+0000 7f42b7fff700 1 -- 172.21.15.111:0/4015322074 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a80094a0 con 0x7f42b8160d40 2023-03-29T10:58:55.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.402+0000 7f42b7fff700 1 --2- 172.21.15.111:0/4015322074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b8160d40 0x7f42b80c9f80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f42b000caa0 tx=0x7f42b000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:55.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.402+0000 7f429ffff700 1 -- 172.21.15.111:0/4015322074 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b000a9e0 con 0x7f42b8160d40 2023-03-29T10:58:55.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.402+0000 7f429ffff700 1 -- 172.21.15.111:0/4015322074 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b000ab40 con 0x7f42b8160d40 2023-03-29T10:58:55.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.402+0000 7f42bdfff700 1 -- 172.21.15.111:0/4015322074 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b80d19a0 con 0x7f42b8160d40 2023-03-29T10:58:55.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.402+0000 7f429ffff700 1 -- 172.21.15.111:0/4015322074 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b0012400 con 0x7f42b8160d40 2023-03-29T10:58:55.402 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.402+0000 7f42bdfff700 1 -- 172.21.15.111:0/4015322074 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b80d1810 con 0x7f42b8160d40 2023-03-29T10:58:55.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.403+0000 7f42bdfff700 1 -- 172.21.15.111:0/4015322074 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b80d1b30 con 0x7f42b8160d40 2023-03-29T10:58:55.403 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.404+0000 7f429ffff700 1 -- 172.21.15.111:0/4015322074 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f42b001b3b0 con 0x7f42b8160d40 2023-03-29T10:58:55.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.405+0000 7f429ffff700 1 --2- 172.21.15.111:0/4015322074 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f42a4069de0 0x7f42a406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:55.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.405+0000 7f429ffff700 1 -- 172.21.15.111:0/4015322074 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f42b00880c0 con 0x7f42b8160d40 2023-03-29T10:58:55.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.406+0000 7f42b77fe700 1 --2- 172.21.15.111:0/4015322074 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f42a4069de0 0x7f42a406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:55.405 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.406+0000 7f42b77fe700 1 --2- 172.21.15.111:0/4015322074 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f42a4069de0 0x7f42a406c290 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f42a80090b0 tx=0x7f42a8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:55.409 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.410+0000 7f429ffff700 1 -- 172.21.15.111:0/4015322074 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f42b00535d0 con 0x7f42b8160d40 2023-03-29T10:58:55.601 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.602+0000 7f42bdfff700 1 -- 172.21.15.111:0/4015322074 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f42b80b52f0 con 0x7f42b8160d40 2023-03-29T10:58:55.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.602+0000 7f429ffff700 1 -- 172.21.15.111:0/4015322074 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f42b0056bf0 con 0x7f42b8160d40 2023-03-29T10:58:55.602 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 8 2023-03-29T10:58:55.604 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.605+0000 7f42bdfff700 1 -- 172.21.15.111:0/4015322074 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f42a4069de0 msgr2=0x7f42a406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:55.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.605+0000 7f42bdfff700 1 --2- 172.21.15.111:0/4015322074 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f42a4069de0 0x7f42a406c290 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f42a80090b0 tx=0x7f42a8008030 comp rx=0 tx=0).stop 2023-03-29T10:58:55.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.605+0000 7f42bdfff700 1 -- 172.21.15.111:0/4015322074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b8160d40 msgr2=0x7f42b80c9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:55.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.605+0000 7f42bdfff700 1 --2- 172.21.15.111:0/4015322074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b8160d40 0x7f42b80c9f80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f42b000caa0 tx=0x7f42b000cdb0 comp rx=0 tx=0).stop 2023-03-29T10:58:55.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.605+0000 7f42bdfff700 1 -- 172.21.15.111:0/4015322074 shutdown_connections 2023-03-29T10:58:55.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.605+0000 7f42bdfff700 1 --2- 172.21.15.111:0/4015322074 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f42a4069de0 0x7f42a406c290 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:55.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.605+0000 7f42bdfff700 1 --2- 172.21.15.111:0/4015322074 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f42b8160d40 0x7f42b80c9f80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:55.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.605+0000 7f42bdfff700 1 --2- 172.21.15.111:0/4015322074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42b80b87a0 0x7f42b81544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:55.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.605+0000 7f42bdfff700 1 --2- 172.21.15.111:0/4015322074 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f42b81601e0 0x7f42b80cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:55.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.605+0000 7f42bdfff700 1 -- 172.21.15.111:0/4015322074 >> 172.21.15.111:0/4015322074 conn(0x7f42b8071b10 msgr2=0x7f42b80b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:55.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.605+0000 7f42bdfff700 1 -- 172.21.15.111:0/4015322074 shutdown_connections 2023-03-29T10:58:55.605 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.605+0000 7f42bdfff700 1 -- 172.21.15.111:0/4015322074 wait complete. 2023-03-29T10:58:55.621 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-03-29T10:58:55.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.809+0000 7f3b8a3ff700 1 Processor -- start 2023-03-29T10:58:55.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.809+0000 7f3b8a3ff700 1 -- start start 2023-03-29T10:58:55.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.810+0000 7f3b8a3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b840cf010 0x7f3b84160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:55.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.810+0000 7f3b8a3ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3b840cff50 0x7f3b840d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:55.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.810+0000 7f3b8a3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b840cd520 con 0x7f3b8415c590 2023-03-29T10:58:55.809 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.810+0000 7f3b8a3ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b840d2640 con 0x7f3b840cff50 2023-03-29T10:58:55.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.810+0000 7f3b8a3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b841610d0 con 0x7f3b840cf010 2023-03-29T10:58:55.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.810+0000 7f3b8899c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3b840cff50 0x7f3b840d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:55.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.810+0000 7f3b7bfff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b840cf010 0x7f3b84160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:55.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.810+0000 7f3b83fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3b8415c590 0x7f3b840cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.111:37306/0 (socket says 172.21.15.111:37306) 2023-03-29T10:58:55.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.811+0000 7f3b83fff700 1 -- 172.21.15.111:0/3842915884 learned_addr learned my addr 172.21.15.111:0/3842915884 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:55.810 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.811+0000 7f3b8899c700 1 --2- 172.21.15.111:0/3842915884 >> v2:172.21.15.119:3300/0 conn(0x7f3b840cff50 0x7f3b840d2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.111:48152/0 (socket says 172.21.15.111:48152) 2023-03-29T10:58:55.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.811+0000 7f3b8899c700 1 -- 172.21.15.111:0/3842915884 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b840cf010 msgr2=0x7f3b84160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:55.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.811+0000 7f3b8899c700 1 --2- 172.21.15.111:0/3842915884 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b840cf010 0x7f3b84160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:55.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.811+0000 7f3b8899c700 1 -- 172.21.15.111:0/3842915884 >> v1:172.21.15.119:6789/0 conn(0x7f3b8415c590 legacy=0x7f3b840cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:55.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.811+0000 7f3b8899c700 1 -- 172.21.15.111:0/3842915884 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b840b23c0 con 0x7f3b840cff50 2023-03-29T10:58:55.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.811+0000 7f3b8899c700 1 --2- 172.21.15.111:0/3842915884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b840cff50 0x7f3b840d2ad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009d90 tx=0x7f3b7c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33b22b6f1dbb1369 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:55.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.811+0000 7f3b837fe700 1 -- 172.21.15.111:0/3842915884 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c00e070 con 0x7f3b840cff50 2023-03-29T10:58:55.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.811+0000 7f3b837fe700 1 -- 172.21.15.111:0/3842915884 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7c00bb20 con 0x7f3b840cff50 2023-03-29T10:58:55.811 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.811+0000 7f3b837fe700 1 -- 172.21.15.111:0/3842915884 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c012430 con 0x7f3b840cff50 2023-03-29T10:58:55.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.812+0000 7f3b8a3ff700 1 -- 172.21.15.111:0/3842915884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b840cff50 msgr2=0x7f3b840d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:55.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.812+0000 7f3b8a3ff700 1 --2- 172.21.15.111:0/3842915884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b840cff50 0x7f3b840d2ad0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009d90 tx=0x7f3b7c00b330 comp rx=0 tx=0).stop 2023-03-29T10:58:55.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.812+0000 7f3b8a3ff700 1 -- 172.21.15.111:0/3842915884 shutdown_connections 2023-03-29T10:58:55.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.812+0000 7f3b8a3ff700 1 --2- 172.21.15.111:0/3842915884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b840cff50 0x7f3b840d2ad0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:55.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.812+0000 7f3b8a3ff700 1 --2- 172.21.15.111:0/3842915884 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b840cf010 0x7f3b84160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:55.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.812+0000 7f3b8a3ff700 1 -- 172.21.15.111:0/3842915884 >> 172.21.15.111:0/3842915884 conn(0x7f3b84071b10 msgr2=0x7f3b84071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:55.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.812+0000 7f3b8a3ff700 1 -- 172.21.15.111:0/3842915884 shutdown_connections 2023-03-29T10:58:55.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.812+0000 7f3b8a3ff700 1 -- 172.21.15.111:0/3842915884 wait complete. 2023-03-29T10:58:55.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.814+0000 7f3b8a3ff700 1 Processor -- start 2023-03-29T10:58:55.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.815+0000 7f3b8a3ff700 1 -- start start 2023-03-29T10:58:55.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.816+0000 7f3b8a3ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b840cf010 0x7f3b840b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:55.815 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.816+0000 7f3b8a3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b840cff50 0x7f3b8414b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:55.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.816+0000 7f3b8a3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8415c590 0x7f3b840b8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:55.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.816+0000 7f3b8a3ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b840b6550 con 0x7f3b840cf010 2023-03-29T10:58:55.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.816+0000 7f3b8a3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b840b6830 con 0x7f3b8415c590 2023-03-29T10:58:55.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.816+0000 7f3b8a3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b840b6b10 con 0x7f3b840cff50 2023-03-29T10:58:55.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.816+0000 7f3b83fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b840cf010 0x7f3b840b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:55.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.816+0000 7f3b83fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b840cf010 0x7f3b840b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51846/0 (socket says 172.21.15.111:51846) 2023-03-29T10:58:55.816 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.816+0000 7f3b83fff700 1 -- 172.21.15.111:0/1354991467 learned_addr learned my addr 172.21.15.111:0/1354991467 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:55.817 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.816+0000 7f3b7bfff700 1 --2- 172.21.15.111:0/1354991467 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b840cff50 0x7f3b8414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:55.817 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.816+0000 7f3b8899c700 1 --2- 172.21.15.111:0/1354991467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8415c590 0x7f3b840b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:55.817 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.816+0000 7f3b83fff700 1 -- 172.21.15.111:0/1354991467 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b840cff50 msgr2=0x7f3b8414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:55.817 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.816+0000 7f3b83fff700 1 --2- 172.21.15.111:0/1354991467 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b840cff50 0x7f3b8414b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:55.817 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.816+0000 7f3b83fff700 1 -- 172.21.15.111:0/1354991467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8415c590 msgr2=0x7f3b840b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:55.817 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.816+0000 7f3b83fff700 1 --2- 172.21.15.111:0/1354991467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8415c590 0x7f3b840b8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:55.817 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.816+0000 7f3b83fff700 1 -- 172.21.15.111:0/1354991467 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c009a70 con 0x7f3b840cf010 2023-03-29T10:58:55.817 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.817+0000 7f3b83fff700 1 --2- 172.21.15.111:0/1354991467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b840cf010 0x7f3b840b5d00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400c480 tx=0x7f3b7400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:55.817 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.817+0000 7f3b817fa700 1 -- 172.21.15.111:0/1354991467 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b74005d70 con 0x7f3b840cf010 2023-03-29T10:58:55.817 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.817+0000 7f3b817fa700 1 -- 172.21.15.111:0/1354991467 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b740070d0 con 0x7f3b840cf010 2023-03-29T10:58:55.817 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.817+0000 7f3b8a3ff700 1 -- 172.21.15.111:0/1354991467 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b840cd770 con 0x7f3b840cf010 2023-03-29T10:58:55.817 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.817+0000 7f3b817fa700 1 -- 172.21.15.111:0/1354991467 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b74007440 con 0x7f3b840cf010 2023-03-29T10:58:55.818 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.817+0000 7f3b8a3ff700 1 -- 172.21.15.111:0/1354991467 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b840cf560 con 0x7f3b840cf010 2023-03-29T10:58:55.818 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.818+0000 7f3b8a3ff700 1 -- 172.21.15.111:0/1354991467 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b8415c960 con 0x7f3b840cf010 2023-03-29T10:58:55.818 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.819+0000 7f3b817fa700 1 -- 172.21.15.111:0/1354991467 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f3b740075a0 con 0x7f3b840cf010 2023-03-29T10:58:55.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.820+0000 7f3b817fa700 1 --2- 172.21.15.111:0/1354991467 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3b64069de0 0x7f3b6406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:55.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.820+0000 7f3b817fa700 1 -- 172.21.15.111:0/1354991467 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3b74087890 con 0x7f3b840cf010 2023-03-29T10:58:55.820 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.820+0000 7f3b7bfff700 1 --2- 172.21.15.111:0/1354991467 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3b64069de0 0x7f3b6406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:55.820 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.821+0000 7f3b7bfff700 1 --2- 172.21.15.111:0/1354991467 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3b64069de0 0x7f3b6406c290 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3b840b45c0 tx=0x7f3b6c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:55.823 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:55.824+0000 7f3b817fa700 1 -- 172.21.15.111:0/1354991467 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f3b7404c160 con 0x7f3b840cf010 2023-03-29T10:58:56.024 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.025+0000 7f3b8a3ff700 1 -- 172.21.15.111:0/1354991467 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3b840b51f0 con 0x7f3b840cf010 2023-03-29T10:58:56.025 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.025+0000 7f3b817fa700 1 -- 172.21.15.111:0/1354991467 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3b74017020 con 0x7f3b840cf010 2023-03-29T10:58:56.025 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:56.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.028+0000 7f3b8a3ff700 1 -- 172.21.15.111:0/1354991467 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3b64069de0 msgr2=0x7f3b6406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:56.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.028+0000 7f3b8a3ff700 1 --2- 172.21.15.111:0/1354991467 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3b64069de0 0x7f3b6406c290 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3b840b45c0 tx=0x7f3b6c008040 comp rx=0 tx=0).stop 2023-03-29T10:58:56.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.028+0000 7f3b8a3ff700 1 -- 172.21.15.111:0/1354991467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b840cf010 msgr2=0x7f3b840b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:56.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.028+0000 7f3b8a3ff700 1 --2- 172.21.15.111:0/1354991467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b840cf010 0x7f3b840b5d00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3b7400c480 tx=0x7f3b7400c840 comp rx=0 tx=0).stop 2023-03-29T10:58:56.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.028+0000 7f3b8a3ff700 1 -- 172.21.15.111:0/1354991467 shutdown_connections 2023-03-29T10:58:56.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.028+0000 7f3b8a3ff700 1 --2- 172.21.15.111:0/1354991467 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3b64069de0 0x7f3b6406c290 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.028+0000 7f3b8a3ff700 1 --2- 172.21.15.111:0/1354991467 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3b840cf010 0x7f3b840b5d00 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.028+0000 7f3b8a3ff700 1 --2- 172.21.15.111:0/1354991467 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b8415c590 0x7f3b840b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.028+0000 7f3b8a3ff700 1 --2- 172.21.15.111:0/1354991467 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3b840cff50 0x7f3b8414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.028+0000 7f3b8a3ff700 1 -- 172.21.15.111:0/1354991467 >> 172.21.15.111:0/1354991467 conn(0x7f3b84071b10 msgr2=0x7f3b840b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:56.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.028+0000 7f3b8a3ff700 1 -- 172.21.15.111:0/1354991467 shutdown_connections 2023-03-29T10:58:56.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.028+0000 7f3b8a3ff700 1 -- 172.21.15.111:0/1354991467 wait complete. 2023-03-29T10:58:56.044 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-03-29T10:58:56.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.231+0000 7f5425fff700 1 Processor -- start 2023-03-29T10:58:56.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.232+0000 7f5425fff700 1 -- start start 2023-03-29T10:58:56.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.233+0000 7f5425fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f54200cd760 0x7f54200cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:56.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.233+0000 7f5425fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f54200adf70 0x7f54200cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:56.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.233+0000 7f5425fff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54200cc9e0 con 0x7f542015cf10 2023-03-29T10:58:56.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.233+0000 7f5425fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f542015de20 con 0x7f54200cd760 2023-03-29T10:58:56.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.233+0000 7f5425fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54200ce070 con 0x7f54200adf70 2023-03-29T10:58:56.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.233+0000 7f541ffff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f542015cf10 0x7f542006c6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47448/0 (socket says 172.21.15.111:47448) 2023-03-29T10:58:56.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.233+0000 7f541ffff700 1 -- 172.21.15.111:0/2179586124 learned_addr learned my addr 172.21.15.111:0/2179586124 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:56.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.233+0000 7f5417fff700 1 --2- 172.21.15.111:0/2179586124 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f54200adf70 0x7f54200cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:56.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.233+0000 7f541effd700 1 -- 172.21.15.111:0/2179586124 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981666695 0 0) 0x7f54200cc9e0 con 0x7f542015cf10 2023-03-29T10:58:56.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.233+0000 7f541f7fe700 1 --2- 172.21.15.111:0/2179586124 >> v2:172.21.15.119:3300/0 conn(0x7f54200cd760 0x7f54200cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:56.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.233+0000 7f541effd700 1 -- 172.21.15.111:0/2179586124 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f540c0035b0 con 0x7f542015cf10 2023-03-29T10:58:56.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f541effd700 1 -- 172.21.15.111:0/2179586124 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4181021681 0 0) 0x7f540c0035b0 con 0x7f542015cf10 2023-03-29T10:58:56.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f541effd700 1 -- 172.21.15.111:0/2179586124 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f54200adf70 msgr2=0x7f54200cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:56.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f541effd700 1 --2- 172.21.15.111:0/2179586124 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f54200adf70 0x7f54200cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f541effd700 1 -- 172.21.15.111:0/2179586124 >> v2:172.21.15.119:3300/0 conn(0x7f54200cd760 msgr2=0x7f54200cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:56.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f541effd700 1 --2- 172.21.15.111:0/2179586124 >> v2:172.21.15.119:3300/0 conn(0x7f54200cd760 0x7f54200cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f541effd700 1 -- 172.21.15.111:0/2179586124 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f542015d860 con 0x7f542015cf10 2023-03-29T10:58:56.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f541effd700 1 -- 172.21.15.111:0/2179586124 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f54180019f0 con 0x7f542015cf10 2023-03-29T10:58:56.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f541effd700 1 -- 172.21.15.111:0/2179586124 >> v1:172.21.15.111:6789/0 conn(0x7f542015cf10 legacy=0x7f542006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:56.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f541effd700 1 --2- 172.21.15.111:0/2179586124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f540c003a00 0x7f540c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:56.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f541effd700 1 --2- 172.21.15.111:0/2179586124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f540c004430 0x7f540c0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:56.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f541effd700 1 --2- 172.21.15.111:0/2179586124 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f54200adf70 0x7f540c004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:56.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f541effd700 1 -- 172.21.15.111:0/2179586124 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542015de20 con 0x7f540c003a00 2023-03-29T10:58:56.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f541effd700 1 -- 172.21.15.111:0/2179586124 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54200ce070 con 0x7f540c004430 2023-03-29T10:58:56.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f541effd700 1 -- 172.21.15.111:0/2179586124 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540c00dc20 con 0x7f54200adf70 2023-03-29T10:58:56.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f541f7fe700 1 --2- 172.21.15.111:0/2179586124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f540c003a00 0x7f540c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:56.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f5417fff700 1 --2- 172.21.15.111:0/2179586124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f540c004430 0x7f540c0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:56.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f541ffff700 1 --2- 172.21.15.111:0/2179586124 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f54200adf70 0x7f540c004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:56.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f5417fff700 1 -- 172.21.15.111:0/2179586124 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f54200adf70 msgr2=0x7f540c004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:56.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f5417fff700 1 --2- 172.21.15.111:0/2179586124 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f54200adf70 0x7f540c004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f5417fff700 1 -- 172.21.15.111:0/2179586124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f540c003a00 msgr2=0x7f540c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:56.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f5417fff700 1 --2- 172.21.15.111:0/2179586124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f540c003a00 0x7f540c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f5417fff700 1 -- 172.21.15.111:0/2179586124 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c0035b0 con 0x7f540c004430 2023-03-29T10:58:56.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.234+0000 7f541f7fe700 1 --2- 172.21.15.111:0/2179586124 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f540c003a00 0x7f540c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:58:56.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.235+0000 7f5417fff700 1 --2- 172.21.15.111:0/2179586124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f540c004430 0x7f540c0048f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5408009b40 tx=0x7f540c005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a60b752d5c489d84 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:56.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.235+0000 7f541effd700 1 -- 172.21.15.111:0/2179586124 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5408005a20 con 0x7f540c004430 2023-03-29T10:58:56.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.235+0000 7f541effd700 1 -- 172.21.15.111:0/2179586124 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5408005b80 con 0x7f540c004430 2023-03-29T10:58:56.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.235+0000 7f541effd700 1 -- 172.21.15.111:0/2179586124 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540800e6c0 con 0x7f540c004430 2023-03-29T10:58:56.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.235+0000 7f5425fff700 1 -- 172.21.15.111:0/2179586124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f540c004430 msgr2=0x7f540c0048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:56.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.235+0000 7f5425fff700 1 --2- 172.21.15.111:0/2179586124 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f540c004430 0x7f540c0048f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5408009b40 tx=0x7f540c005880 comp rx=0 tx=0).stop 2023-03-29T10:58:56.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.235+0000 7f541ffff700 1 -- 172.21.15.111:0/2179586124 reap_dead start 2023-03-29T10:58:56.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.236+0000 7f5425fff700 1 -- 172.21.15.111:0/2179586124 shutdown_connections 2023-03-29T10:58:56.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.236+0000 7f5425fff700 1 -- 172.21.15.111:0/2179586124 >> 172.21.15.111:0/2179586124 conn(0x7f54200cf720 msgr2=0x7f54200cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:56.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.236+0000 7f5425fff700 1 -- 172.21.15.111:0/2179586124 shutdown_connections 2023-03-29T10:58:56.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.236+0000 7f5425fff700 1 -- 172.21.15.111:0/2179586124 wait complete. 2023-03-29T10:58:56.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.237+0000 7f5425fff700 1 Processor -- start 2023-03-29T10:58:56.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.238+0000 7f5425fff700 1 -- start start 2023-03-29T10:58:56.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.238+0000 7f5425fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f540c004430 0x7f54200ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:56.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.239+0000 7f5425fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54200afc10 0x7f54200b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:56.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.239+0000 7f5425fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54200b0b40 0x7f5420152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:56.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.239+0000 7f5425fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54200b0fb0 con 0x7f54200b0b40 2023-03-29T10:58:56.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.239+0000 7f5425fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54200af780 con 0x7f54200afc10 2023-03-29T10:58:56.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.239+0000 7f5425fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54200aeb70 con 0x7f540c004430 2023-03-29T10:58:56.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.239+0000 7f541ffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54200b0b40 0x7f5420152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:56.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.239+0000 7f541ffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54200b0b40 0x7f5420152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51862/0 (socket says 172.21.15.111:51862) 2023-03-29T10:58:56.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.239+0000 7f5417fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54200afc10 0x7f54200b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:56.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.239+0000 7f541ffff700 1 -- 172.21.15.111:0/4076716677 learned_addr learned my addr 172.21.15.111:0/4076716677 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:56.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.239+0000 7f541f7fe700 1 --2- 172.21.15.111:0/4076716677 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f540c004430 0x7f54200ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:56.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.239+0000 7f5417fff700 1 -- 172.21.15.111:0/4076716677 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f540c004430 msgr2=0x7f54200ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:56.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.239+0000 7f5417fff700 1 --2- 172.21.15.111:0/4076716677 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f540c004430 0x7f54200ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.239+0000 7f5417fff700 1 -- 172.21.15.111:0/4076716677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54200b0b40 msgr2=0x7f5420152d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:56.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.239+0000 7f5417fff700 1 --2- 172.21.15.111:0/4076716677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54200b0b40 0x7f5420152d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.239+0000 7f5417fff700 1 -- 172.21.15.111:0/4076716677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54200cc670 con 0x7f54200afc10 2023-03-29T10:58:56.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.239+0000 7f541ffff700 1 --2- 172.21.15.111:0/4076716677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54200b0b40 0x7f5420152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-03-29T10:58:56.241 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.240+0000 7f5417fff700 1 --2- 172.21.15.111:0/4076716677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54200afc10 0x7f54200b0600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5408009b10 tx=0x7f54080047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:56.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.240+0000 7f541cff9700 1 -- 172.21.15.111:0/4076716677 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5408020070 con 0x7f54200afc10 2023-03-29T10:58:56.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.240+0000 7f541cff9700 1 -- 172.21.15.111:0/4076716677 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5408004260 con 0x7f54200afc10 2023-03-29T10:58:56.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.240+0000 7f541cff9700 1 -- 172.21.15.111:0/4076716677 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5408005a20 con 0x7f54200afc10 2023-03-29T10:58:56.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.240+0000 7f5425fff700 1 -- 172.21.15.111:0/4076716677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f542015db40 con 0x7f54200afc10 2023-03-29T10:58:56.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.240+0000 7f5425fff700 1 -- 172.21.15.111:0/4076716677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5420071600 con 0x7f54200afc10 2023-03-29T10:58:56.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.241+0000 7f5425fff700 1 -- 172.21.15.111:0/4076716677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5420071b50 con 0x7f54200afc10 2023-03-29T10:58:56.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.242+0000 7f541cff9700 1 -- 172.21.15.111:0/4076716677 <== mon.1 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f54080036a0 con 0x7f54200afc10 2023-03-29T10:58:56.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.243+0000 7f541cff9700 1 --2- 172.21.15.111:0/4076716677 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5400069de0 0x7f540006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:56.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.243+0000 7f541cff9700 1 -- 172.21.15.111:0/4076716677 <== mon.1 v2:172.21.15.119:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f5408089060 con 0x7f54200afc10 2023-03-29T10:58:56.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.243+0000 7f541f7fe700 1 --2- 172.21.15.111:0/4076716677 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5400069de0 0x7f540006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:56.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.244+0000 7f541f7fe700 1 --2- 172.21.15.111:0/4076716677 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5400069de0 0x7f540006c290 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f542006cbd0 tx=0x7f5410015040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:56.248 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.248+0000 7f541cff9700 1 -- 172.21.15.111:0/4076716677 <== mon.1 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f54080bb160 con 0x7f54200afc10 2023-03-29T10:58:56.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.441+0000 7f5425fff700 1 -- 172.21.15.111:0/4076716677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f54200b4a10 con 0x7f54200afc10 2023-03-29T10:58:56.441 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.442+0000 7f541cff9700 1 -- 172.21.15.111:0/4076716677 <== mon.1 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f54080545d0 con 0x7f54200afc10 2023-03-29T10:58:56.441 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 64 2023-03-29T10:58:56.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.444+0000 7f5425fff700 1 -- 172.21.15.111:0/4076716677 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5400069de0 msgr2=0x7f540006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:56.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.444+0000 7f5425fff700 1 --2- 172.21.15.111:0/4076716677 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5400069de0 0x7f540006c290 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f542006cbd0 tx=0x7f5410015040 comp rx=0 tx=0).stop 2023-03-29T10:58:56.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.444+0000 7f5425fff700 1 -- 172.21.15.111:0/4076716677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54200afc10 msgr2=0x7f54200b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:56.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.444+0000 7f5425fff700 1 --2- 172.21.15.111:0/4076716677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54200afc10 0x7f54200b0600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5408009b10 tx=0x7f54080047c0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.444+0000 7f5425fff700 1 -- 172.21.15.111:0/4076716677 shutdown_connections 2023-03-29T10:58:56.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.444+0000 7f5425fff700 1 --2- 172.21.15.111:0/4076716677 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5400069de0 0x7f540006c290 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.444+0000 7f5425fff700 1 --2- 172.21.15.111:0/4076716677 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f54200b0b40 0x7f5420152d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.444+0000 7f5425fff700 1 --2- 172.21.15.111:0/4076716677 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f540c004430 0x7f54200ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.444+0000 7f5425fff700 1 --2- 172.21.15.111:0/4076716677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54200afc10 0x7f54200b0600 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.444+0000 7f5425fff700 1 -- 172.21.15.111:0/4076716677 >> 172.21.15.111:0/4076716677 conn(0x7f54200cf720 msgr2=0x7f542015aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:56.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.444+0000 7f5425fff700 1 -- 172.21.15.111:0/4076716677 shutdown_connections 2023-03-29T10:58:56.444 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.444+0000 7f5425fff700 1 -- 172.21.15.111:0/4076716677 wait complete. 2023-03-29T10:58:56.461 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-03-29T10:58:56.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.649+0000 7f25c39ff700 1 Processor -- start 2023-03-29T10:58:56.648 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.649+0000 7f25c39ff700 1 -- start start 2023-03-29T10:58:56.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.650+0000 7f25c39ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f25bc0b87a0 0x7f25bc0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:56.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.650+0000 7f25c39ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25bc1601e0 0x7f25bc160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:56.649 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.650+0000 7f25c39ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25bc160ff0 con 0x7f25bc160b70 2023-03-29T10:58:56.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.650+0000 7f25c39ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25bc0d02d0 con 0x7f25bc0b87a0 2023-03-29T10:58:56.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.650+0000 7f25c39ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc0d0cb0 con 0x7f25bc1601e0 2023-03-29T10:58:56.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.650+0000 7f25c1f9c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f25bc160b70 0x7f25bc0d2de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47464/0 (socket says 172.21.15.111:47464) 2023-03-29T10:58:56.650 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.650+0000 7f25c1f9c700 1 -- 172.21.15.111:0/3038143413 learned_addr learned my addr 172.21.15.111:0/3038143413 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:56.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.650+0000 7f25b3fff700 1 -- 172.21.15.111:0/3038143413 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4239705560 0 0) 0x7f25bc160ff0 con 0x7f25bc160b70 2023-03-29T10:58:56.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.650+0000 7f25b3fff700 1 -- 172.21.15.111:0/3038143413 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25a80035b0 con 0x7f25bc160b70 2023-03-29T10:58:56.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.651+0000 7f25c0f9a700 1 --2- 172.21.15.111:0/3038143413 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25bc1601e0 0x7f25bc160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:56.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.651+0000 7f25c179b700 1 --2- 172.21.15.111:0/3038143413 >> v2:172.21.15.119:3300/0 conn(0x7f25bc0b87a0 0x7f25bc0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:56.651 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.651+0000 7f25b3fff700 1 -- 172.21.15.111:0/3038143413 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1158898496 0 0) 0x7f25a80035b0 con 0x7f25bc160b70 2023-03-29T10:58:56.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.651+0000 7f25b3fff700 1 -- 172.21.15.111:0/3038143413 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25bc1601e0 msgr2=0x7f25bc160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:56.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.651+0000 7f25b3fff700 1 --2- 172.21.15.111:0/3038143413 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25bc1601e0 0x7f25bc160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.651+0000 7f25b3fff700 1 -- 172.21.15.111:0/3038143413 >> v2:172.21.15.119:3300/0 conn(0x7f25bc0b87a0 msgr2=0x7f25bc0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:56.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.651+0000 7f25b3fff700 1 --2- 172.21.15.111:0/3038143413 >> v2:172.21.15.119:3300/0 conn(0x7f25bc0b87a0 0x7f25bc0cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.651+0000 7f25b3fff700 1 -- 172.21.15.111:0/3038143413 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc0cdc60 con 0x7f25bc160b70 2023-03-29T10:58:56.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.651+0000 7f25b3fff700 1 -- 172.21.15.111:0/3038143413 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f25b40019f0 con 0x7f25bc160b70 2023-03-29T10:58:56.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.651+0000 7f25b3fff700 1 -- 172.21.15.111:0/3038143413 >> v1:172.21.15.111:6789/0 conn(0x7f25bc160b70 legacy=0x7f25bc0d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:56.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.651+0000 7f25b3fff700 1 --2- 172.21.15.111:0/3038143413 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25bc1601e0 0x7f25a8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:56.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.651+0000 7f25b3fff700 1 --2- 172.21.15.111:0/3038143413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25a8003fd0 0x7f25a8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:56.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.651+0000 7f25b3fff700 1 --2- 172.21.15.111:0/3038143413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25a8004d90 0x7f25a8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:56.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.651+0000 7f25b3fff700 1 -- 172.21.15.111:0/3038143413 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc0d02d0 con 0x7f25a8004d90 2023-03-29T10:58:56.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.651+0000 7f25b3fff700 1 -- 172.21.15.111:0/3038143413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a800ba10 con 0x7f25a8003fd0 2023-03-29T10:58:56.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.651+0000 7f25b3fff700 1 -- 172.21.15.111:0/3038143413 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a800bb50 con 0x7f25bc1601e0 2023-03-29T10:58:56.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.651+0000 7f25c1f9c700 1 --2- 172.21.15.111:0/3038143413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25a8004d90 0x7f25a8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:56.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.651+0000 7f25c179b700 1 --2- 172.21.15.111:0/3038143413 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25bc1601e0 0x7f25a8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:56.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.652+0000 7f25c0f9a700 1 --2- 172.21.15.111:0/3038143413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25a8003fd0 0x7f25a8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:56.652 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.652+0000 7f25c1f9c700 1 -- 172.21.15.111:0/3038143413 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25bc1601e0 msgr2=0x7f25a8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:56.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.652+0000 7f25c1f9c700 1 --2- 172.21.15.111:0/3038143413 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25bc1601e0 0x7f25a8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.652+0000 7f25c1f9c700 1 -- 172.21.15.111:0/3038143413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25a8003fd0 msgr2=0x7f25a8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:56.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.652+0000 7f25c1f9c700 1 --2- 172.21.15.111:0/3038143413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25a8003fd0 0x7f25a8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.652+0000 7f25c1f9c700 1 -- 172.21.15.111:0/3038143413 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a80035b0 con 0x7f25a8004d90 2023-03-29T10:58:56.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.652+0000 7f25c1f9c700 1 --2- 172.21.15.111:0/3038143413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25a8004d90 0x7f25a8005200 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f25b400a790 tx=0x7f25a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b3bc1a4b375c07 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:56.653 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.652+0000 7f25b3fff700 1 -- 172.21.15.111:0/3038143413 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b400a250 con 0x7f25a8004d90 2023-03-29T10:58:56.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.652+0000 7f25b3fff700 1 -- 172.21.15.111:0/3038143413 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b4012600 con 0x7f25a8004d90 2023-03-29T10:58:56.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.652+0000 7f25b3fff700 1 -- 172.21.15.111:0/3038143413 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b40129b0 con 0x7f25a8004d90 2023-03-29T10:58:56.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.653+0000 7f25c39ff700 1 -- 172.21.15.111:0/3038143413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25a8004d90 msgr2=0x7f25a8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:56.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.653+0000 7f25c39ff700 1 --2- 172.21.15.111:0/3038143413 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25a8004d90 0x7f25a8005200 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f25b400a790 tx=0x7f25a8005880 comp rx=0 tx=0).stop 2023-03-29T10:58:56.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.653+0000 7f25c1f9c700 1 -- 172.21.15.111:0/3038143413 reap_dead start 2023-03-29T10:58:56.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.653+0000 7f25c39ff700 1 -- 172.21.15.111:0/3038143413 shutdown_connections 2023-03-29T10:58:56.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.653+0000 7f25c39ff700 1 -- 172.21.15.111:0/3038143413 >> 172.21.15.111:0/3038143413 conn(0x7f25bc071b10 msgr2=0x7f25bc071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:56.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.653+0000 7f25c39ff700 1 -- 172.21.15.111:0/3038143413 shutdown_connections 2023-03-29T10:58:56.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.653+0000 7f25c39ff700 1 -- 172.21.15.111:0/3038143413 wait complete. 2023-03-29T10:58:56.654 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.654+0000 7f25c39ff700 1 Processor -- start 2023-03-29T10:58:56.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.655+0000 7f25c39ff700 1 -- start start 2023-03-29T10:58:56.655 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.655+0000 7f25c39ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25a8004d90 0x7f25bc15b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:56.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.655+0000 7f25c39ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25bc15e170 0x7f25bc150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:56.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.655+0000 7f25c39ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25bc0c8420 0x7f25bc0c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:56.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.655+0000 7f25c39ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc0c8de0 con 0x7f25bc15e170 2023-03-29T10:58:56.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.655+0000 7f25c39ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc15eea0 con 0x7f25bc0c8420 2023-03-29T10:58:56.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.655+0000 7f25c39ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc15c250 con 0x7f25a8004d90 2023-03-29T10:58:56.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.656+0000 7f25c0f9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25bc15e170 0x7f25bc150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:56.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.656+0000 7f25c0f9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25bc15e170 0x7f25bc150970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51882/0 (socket says 172.21.15.111:51882) 2023-03-29T10:58:56.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.656+0000 7f25c0f9a700 1 -- 172.21.15.111:0/2406262025 learned_addr learned my addr 172.21.15.111:0/2406262025 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:56.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.656+0000 7f25c179b700 1 --2- 172.21.15.111:0/2406262025 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25a8004d90 0x7f25bc15b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:56.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.656+0000 7f25c1f9c700 1 --2- 172.21.15.111:0/2406262025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25bc0c8420 0x7f25bc0c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:56.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.656+0000 7f25c0f9a700 1 -- 172.21.15.111:0/2406262025 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25a8004d90 msgr2=0x7f25bc15b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:56.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.656+0000 7f25c0f9a700 1 --2- 172.21.15.111:0/2406262025 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25a8004d90 0x7f25bc15b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.656+0000 7f25c0f9a700 1 -- 172.21.15.111:0/2406262025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25bc0c8420 msgr2=0x7f25bc0c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:56.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.656+0000 7f25c0f9a700 1 --2- 172.21.15.111:0/2406262025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25bc0c8420 0x7f25bc0c88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.656 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.656+0000 7f25c0f9a700 1 -- 172.21.15.111:0/2406262025 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc0cd980 con 0x7f25bc15e170 2023-03-29T10:58:56.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.656+0000 7f25c0f9a700 1 --2- 172.21.15.111:0/2406262025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25bc15e170 0x7f25bc150970 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f25ac010870 tx=0x7f25ac010b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:56.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.657+0000 7f25b1ffb700 1 -- 172.21.15.111:0/2406262025 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac016670 con 0x7f25bc15e170 2023-03-29T10:58:56.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.657+0000 7f25b1ffb700 1 -- 172.21.15.111:0/2406262025 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac0167d0 con 0x7f25bc15e170 2023-03-29T10:58:56.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.657+0000 7f25c39ff700 1 -- 172.21.15.111:0/2406262025 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25bc0cd6a0 con 0x7f25bc15e170 2023-03-29T10:58:56.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.657+0000 7f25b1ffb700 1 -- 172.21.15.111:0/2406262025 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac016b40 con 0x7f25bc15e170 2023-03-29T10:58:56.657 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.657+0000 7f25c39ff700 1 -- 172.21.15.111:0/2406262025 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc0d2110 con 0x7f25bc15e170 2023-03-29T10:58:56.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.658+0000 7f25c39ff700 1 -- 172.21.15.111:0/2406262025 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25bc15c0b0 con 0x7f25bc15e170 2023-03-29T10:58:56.658 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.659+0000 7f25b1ffb700 1 -- 172.21.15.111:0/2406262025 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f25ac016ca0 con 0x7f25bc15e170 2023-03-29T10:58:56.659 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.660+0000 7f25b1ffb700 1 --2- 172.21.15.111:0/2406262025 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f25a0069de0 0x7f25a006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:56.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.660+0000 7f25b1ffb700 1 -- 172.21.15.111:0/2406262025 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f25ac088560 con 0x7f25bc15e170 2023-03-29T10:58:56.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.660+0000 7f25c179b700 1 --2- 172.21.15.111:0/2406262025 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f25a0069de0 0x7f25a006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:56.660 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.661+0000 7f25c179b700 1 --2- 172.21.15.111:0/2406262025 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f25a0069de0 0x7f25a006c290 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009ce0 tx=0x7f25b8009210 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:56.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.665+0000 7f25b1ffb700 1 -- 172.21.15.111:0/2406262025 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f25bc15c0b0 con 0x7f25bc15e170 2023-03-29T10:58:56.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.856+0000 7f25c39ff700 1 -- 172.21.15.111:0/2406262025 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f25bc15c0b0 con 0x7f25bc15e170 2023-03-29T10:58:56.856 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.857+0000 7f25b1ffb700 1 -- 172.21.15.111:0/2406262025 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v30) v1 ==== 103+0+11 (secure 0 0 0) 0x7f25bc15c0b0 con 0x7f25bc15e170 2023-03-29T10:58:56.856 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 32 2023-03-29T10:58:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.859+0000 7f25c39ff700 1 -- 172.21.15.111:0/2406262025 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f25a0069de0 msgr2=0x7f25a006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.859+0000 7f25c39ff700 1 --2- 172.21.15.111:0/2406262025 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f25a0069de0 0x7f25a006c290 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009ce0 tx=0x7f25b8009210 comp rx=0 tx=0).stop 2023-03-29T10:58:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.859+0000 7f25c39ff700 1 -- 172.21.15.111:0/2406262025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25bc15e170 msgr2=0x7f25bc150970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.859+0000 7f25c39ff700 1 --2- 172.21.15.111:0/2406262025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25bc15e170 0x7f25bc150970 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f25ac010870 tx=0x7f25ac010b80 comp rx=0 tx=0).stop 2023-03-29T10:58:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.859+0000 7f25c39ff700 1 -- 172.21.15.111:0/2406262025 shutdown_connections 2023-03-29T10:58:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.859+0000 7f25c39ff700 1 --2- 172.21.15.111:0/2406262025 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f25a0069de0 0x7f25a006c290 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.859+0000 7f25c39ff700 1 --2- 172.21.15.111:0/2406262025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25bc0c8420 0x7f25bc0c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.859+0000 7f25c39ff700 1 --2- 172.21.15.111:0/2406262025 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f25a8004d90 0x7f25bc15b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.859+0000 7f25c39ff700 1 --2- 172.21.15.111:0/2406262025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f25bc15e170 0x7f25bc150970 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.859+0000 7f25c39ff700 1 -- 172.21.15.111:0/2406262025 >> 172.21.15.111:0/2406262025 conn(0x7f25bc071b10 msgr2=0x7f25bc0d3700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.859+0000 7f25c39ff700 1 -- 172.21.15.111:0/2406262025 shutdown_connections 2023-03-29T10:58:56.859 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:56.859+0000 7f25c39ff700 1 -- 172.21.15.111:0/2406262025 wait complete. 2023-03-29T10:58:56.876 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-29T10:58:57.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.059+0000 7fcbb89ff700 1 Processor -- start 2023-03-29T10:58:57.058 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.059+0000 7fcbb89ff700 1 -- start start 2023-03-29T10:58:57.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.060+0000 7fcbb89ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbb40b8100 0x7fcbb41675e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.059 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.060+0000 7fcbb89ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb40cdc10 con 0x7fcbb40cf970 2023-03-29T10:58:57.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.060+0000 7fcbb89ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb40b6c60 con 0x7fcbb40cfd40 2023-03-29T10:58:57.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.060+0000 7fcbb89ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb40d08d0 con 0x7fcbb40b8100 2023-03-29T10:58:57.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.060+0000 7fcbb259c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fcbb40cf970 0x7fcbb40cf110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47468/0 (socket says 172.21.15.111:47468) 2023-03-29T10:58:57.060 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.060+0000 7fcbb259c700 1 -- 172.21.15.111:0/2563395459 learned_addr learned my addr 172.21.15.111:0/2563395459 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:57.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.060+0000 7fcbb159a700 1 -- 172.21.15.111:0/2563395459 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1447439787 0 0) 0x7fcbb40cdc10 con 0x7fcbb40cf970 2023-03-29T10:58:57.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.061+0000 7fcbb2d9d700 1 --2- 172.21.15.111:0/2563395459 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbb40b8100 0x7fcbb41675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:57.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.061+0000 7fcbb159a700 1 -- 172.21.15.111:0/2563395459 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb9c0035b0 con 0x7fcbb40cf970 2023-03-29T10:58:57.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.061+0000 7fcbb159a700 1 -- 172.21.15.111:0/2563395459 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 900331572 0 0) 0x7fcb9c0035b0 con 0x7fcbb40cf970 2023-03-29T10:58:57.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.061+0000 7fcbb159a700 1 -- 172.21.15.111:0/2563395459 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbb40b8100 msgr2=0x7fcbb41675e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.061+0000 7fcbb159a700 1 --2- 172.21.15.111:0/2563395459 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbb40b8100 0x7fcbb41675e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.061+0000 7fcbb159a700 1 -- 172.21.15.111:0/2563395459 >> v1:172.21.15.119:6789/0 conn(0x7fcbb40cfd40 legacy=0x7fcbb40d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.061+0000 7fcbb159a700 1 -- 172.21.15.111:0/2563395459 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb40cd650 con 0x7fcbb40cf970 2023-03-29T10:58:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.061+0000 7fcbb159a700 1 -- 172.21.15.111:0/2563395459 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7fcba4001670 con 0x7fcbb40cf970 2023-03-29T10:58:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.061+0000 7fcbb159a700 1 -- 172.21.15.111:0/2563395459 >> v1:172.21.15.111:6789/0 conn(0x7fcbb40cf970 legacy=0x7fcbb40cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.061+0000 7fcbb159a700 1 --2- 172.21.15.111:0/2563395459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb9c003aa0 0x7fcb9c003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.061+0000 7fcbb159a700 1 --2- 172.21.15.111:0/2563395459 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbb40b8100 0x7fcb9c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.061+0000 7fcbb159a700 1 --2- 172.21.15.111:0/2563395459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb9c004d90 0x7fcb9c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.061+0000 7fcbb159a700 1 -- 172.21.15.111:0/2563395459 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb40d08d0 con 0x7fcb9c004d90 2023-03-29T10:58:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.061+0000 7fcbb159a700 1 -- 172.21.15.111:0/2563395459 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c00bbf0 con 0x7fcb9c003aa0 2023-03-29T10:58:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.061+0000 7fcbb159a700 1 -- 172.21.15.111:0/2563395459 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c00bdb0 con 0x7fcbb40b8100 2023-03-29T10:58:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.061+0000 7fcbb2d9d700 1 --2- 172.21.15.111:0/2563395459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb9c004d90 0x7fcb9c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.061+0000 7fcbb259c700 1 --2- 172.21.15.111:0/2563395459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb9c003aa0 0x7fcb9c003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.061+0000 7fcbb1d9b700 1 --2- 172.21.15.111:0/2563395459 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbb40b8100 0x7fcb9c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.062+0000 7fcbb2d9d700 1 -- 172.21.15.111:0/2563395459 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbb40b8100 msgr2=0x7fcb9c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.062+0000 7fcbb2d9d700 1 --2- 172.21.15.111:0/2563395459 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbb40b8100 0x7fcb9c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.062+0000 7fcbb2d9d700 1 -- 172.21.15.111:0/2563395459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb9c003aa0 msgr2=0x7fcb9c003eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:57.062 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.062+0000 7fcbb2d9d700 1 --2- 172.21.15.111:0/2563395459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb9c003aa0 0x7fcb9c003eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.062+0000 7fcbb2d9d700 1 -- 172.21.15.111:0/2563395459 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c0035b0 con 0x7fcb9c004d90 2023-03-29T10:58:57.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.062+0000 7fcbb1d9b700 1 --2- 172.21.15.111:0/2563395459 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbb40b8100 0x7fcb9c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:57.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.062+0000 7fcbb259c700 1 --2- 172.21.15.111:0/2563395459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb9c003aa0 0x7fcb9c003eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:57.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.062+0000 7fcbb2d9d700 1 --2- 172.21.15.111:0/2563395459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb9c004d90 0x7fcb9c005200 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009fc0 tx=0x7fcb9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc284da70365b577 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:57.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.062+0000 7fcbb159a700 1 -- 172.21.15.111:0/2563395459 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac00ec30 con 0x7fcb9c004d90 2023-03-29T10:58:57.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.062+0000 7fcbb159a700 1 -- 172.21.15.111:0/2563395459 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbac00f040 con 0x7fcb9c004d90 2023-03-29T10:58:57.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.062+0000 7fcbb159a700 1 -- 172.21.15.111:0/2563395459 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac014480 con 0x7fcb9c004d90 2023-03-29T10:58:57.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.063+0000 7fcbb89ff700 1 -- 172.21.15.111:0/2563395459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb9c004d90 msgr2=0x7fcb9c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:57.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.063+0000 7fcbb89ff700 1 --2- 172.21.15.111:0/2563395459 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcb9c004d90 0x7fcb9c005200 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcbac009fc0 tx=0x7fcb9c005880 comp rx=0 tx=0).stop 2023-03-29T10:58:57.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.063+0000 7fcbb2d9d700 1 -- 172.21.15.111:0/2563395459 reap_dead start 2023-03-29T10:58:57.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.063+0000 7fcbb89ff700 1 -- 172.21.15.111:0/2563395459 shutdown_connections 2023-03-29T10:58:57.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.063+0000 7fcbb89ff700 1 -- 172.21.15.111:0/2563395459 >> 172.21.15.111:0/2563395459 conn(0x7fcbb4071b10 msgr2=0x7fcbb4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:57.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.063+0000 7fcbb89ff700 1 -- 172.21.15.111:0/2563395459 shutdown_connections 2023-03-29T10:58:57.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.063+0000 7fcbb89ff700 1 -- 172.21.15.111:0/2563395459 wait complete. 2023-03-29T10:58:57.064 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.065+0000 7fcbb89ff700 1 Processor -- start 2023-03-29T10:58:57.065 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.065+0000 7fcbb89ff700 1 -- start start 2023-03-29T10:58:57.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.066+0000 7fcbb89ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbb4150bd0 0x7fcbb40c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.066+0000 7fcbb89ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbb415f000 0x7fcbb40b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.066+0000 7fcbb89ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbb415f470 0x7fcbb40b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.067+0000 7fcbb89ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb40ca240 con 0x7fcbb415f470 2023-03-29T10:58:57.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.067+0000 7fcbb89ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb415f9f0 con 0x7fcbb4150bd0 2023-03-29T10:58:57.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.067+0000 7fcbb89ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb40ca490 con 0x7fcbb415f000 2023-03-29T10:58:57.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.067+0000 7fcbb2d9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbb415f470 0x7fcbb40b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:57.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.067+0000 7fcbb2d9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbb415f470 0x7fcbb40b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51894/0 (socket says 172.21.15.111:51894) 2023-03-29T10:58:57.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.067+0000 7fcbb2d9d700 1 -- 172.21.15.111:0/1478668766 learned_addr learned my addr 172.21.15.111:0/1478668766 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:57.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.067+0000 7fcbb259c700 1 --2- 172.21.15.111:0/1478668766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbb4150bd0 0x7fcbb40c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:57.067 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.067+0000 7fcbb1d9b700 1 --2- 172.21.15.111:0/1478668766 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbb415f000 0x7fcbb40b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:57.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.067+0000 7fcbb2d9d700 1 -- 172.21.15.111:0/1478668766 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbb415f000 msgr2=0x7fcbb40b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:57.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.067+0000 7fcbb2d9d700 1 --2- 172.21.15.111:0/1478668766 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbb415f000 0x7fcbb40b36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.067+0000 7fcbb2d9d700 1 -- 172.21.15.111:0/1478668766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbb4150bd0 msgr2=0x7fcbb40c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:57.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.067+0000 7fcbb2d9d700 1 --2- 172.21.15.111:0/1478668766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbb4150bd0 0x7fcbb40c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.067+0000 7fcbb2d9d700 1 -- 172.21.15.111:0/1478668766 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb40b68f0 con 0x7fcbb415f470 2023-03-29T10:58:57.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.067+0000 7fcbb259c700 1 --2- 172.21.15.111:0/1478668766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbb4150bd0 0x7fcbb40c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:57.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.067+0000 7fcbb2d9d700 1 --2- 172.21.15.111:0/1478668766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbb415f470 0x7fcbb40b3c30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcbac016000 tx=0x7fcbac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:57.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.068+0000 7fcba2ffd700 1 -- 172.21.15.111:0/1478668766 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac007000 con 0x7fcbb415f470 2023-03-29T10:58:57.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.068+0000 7fcba2ffd700 1 -- 172.21.15.111:0/1478668766 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbac022070 con 0x7fcbb415f470 2023-03-29T10:58:57.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.068+0000 7fcbb89ff700 1 -- 172.21.15.111:0/1478668766 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb40cd930 con 0x7fcbb415f470 2023-03-29T10:58:57.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.068+0000 7fcba2ffd700 1 -- 172.21.15.111:0/1478668766 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac029480 con 0x7fcbb415f470 2023-03-29T10:58:57.068 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.068+0000 7fcbb89ff700 1 -- 172.21.15.111:0/1478668766 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb40d36d0 con 0x7fcbb415f470 2023-03-29T10:58:57.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.069+0000 7fcbb89ff700 1 -- 172.21.15.111:0/1478668766 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb4074430 con 0x7fcbb415f470 2023-03-29T10:58:57.069 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.070+0000 7fcba2ffd700 1 -- 172.21.15.111:0/1478668766 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fcbac0295e0 con 0x7fcbb415f470 2023-03-29T10:58:57.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.071+0000 7fcba2ffd700 1 --2- 172.21.15.111:0/1478668766 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fcb94069de0 0x7fcb9406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.071+0000 7fcba2ffd700 1 -- 172.21.15.111:0/1478668766 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcbac024070 con 0x7fcbb415f470 2023-03-29T10:58:57.070 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.071+0000 7fcbb259c700 1 --2- 172.21.15.111:0/1478668766 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fcb94069de0 0x7fcb9406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:57.071 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.071+0000 7fcbb259c700 1 --2- 172.21.15.111:0/1478668766 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fcb94069de0 0x7fcb9406c290 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcba4001eb0 tx=0x7fcba4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:57.074 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.075+0000 7fcba2ffd700 1 -- 172.21.15.111:0/1478668766 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fcbac0555a0 con 0x7fcbb415f470 2023-03-29T10:58:57.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.308+0000 7fcbb89ff700 1 -- 172.21.15.111:0/1478668766 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbb4072fc0 con 0x7fcbb415f470 2023-03-29T10:58:57.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.308+0000 7fcba2ffd700 1 -- 172.21.15.111:0/1478668766 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fcbac018020 con 0x7fcbb415f470 2023-03-29T10:58:57.308 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:57.308 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/556826332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":556826332},{"type":"v1","addr":"172.21.15.111:6837","nonce":556826332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/814623168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":814623168},{"type":"v1","addr":"172.21.15.111:6835","nonce":814623168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/514422035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":514422035},{"type":"v1","addr":"172.21.15.119:6834","nonce":514422035}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-29T10:57:27.824135+0000","modified":"2023-03-29T10:57:53.367898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/3160800581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":3160800581},{"type":"v1","addr":"172.21.15.119:6835","nonce":3160800581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-03-29T10:58:57.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.311+0000 7fcbb89ff700 1 -- 172.21.15.111:0/1478668766 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fcb94069de0 msgr2=0x7fcb9406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:57.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.311+0000 7fcbb89ff700 1 --2- 172.21.15.111:0/1478668766 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fcb94069de0 0x7fcb9406c290 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcba4001eb0 tx=0x7fcba4009040 comp rx=0 tx=0).stop 2023-03-29T10:58:57.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.311+0000 7fcbb89ff700 1 -- 172.21.15.111:0/1478668766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbb415f470 msgr2=0x7fcbb40b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:57.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.311+0000 7fcbb89ff700 1 --2- 172.21.15.111:0/1478668766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbb415f470 0x7fcbb40b3c30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fcbac016000 tx=0x7fcbac009750 comp rx=0 tx=0).stop 2023-03-29T10:58:57.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.311+0000 7fcbb89ff700 1 -- 172.21.15.111:0/1478668766 shutdown_connections 2023-03-29T10:58:57.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.311+0000 7fcbb89ff700 1 --2- 172.21.15.111:0/1478668766 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fcb94069de0 0x7fcb9406c290 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.311+0000 7fcbb89ff700 1 --2- 172.21.15.111:0/1478668766 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fcbb415f470 0x7fcbb40b3c30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.311+0000 7fcbb89ff700 1 --2- 172.21.15.111:0/1478668766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbb4150bd0 0x7fcbb40c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.311+0000 7fcbb89ff700 1 --2- 172.21.15.111:0/1478668766 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fcbb415f000 0x7fcbb40b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.311+0000 7fcbb89ff700 1 -- 172.21.15.111:0/1478668766 >> 172.21.15.111:0/1478668766 conn(0x7fcbb4071b10 msgr2=0x7fcbb40b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:57.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.311+0000 7fcbb89ff700 1 -- 172.21.15.111:0/1478668766 shutdown_connections 2023-03-29T10:58:57.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.311+0000 7fcbb89ff700 1 -- 172.21.15.111:0/1478668766 wait complete. 2023-03-29T10:58:57.312 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2023-03-29T10:58:57.328 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:58:57.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.511+0000 7f4bc01ff700 1 Processor -- start 2023-03-29T10:58:57.511 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.511+0000 7f4bc01ff700 1 -- start start 2023-03-29T10:58:57.512 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.513+0000 7f4bc01ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bb815c950 0x7f4bb80cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.513+0000 7f4bc01ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4bb80d3a30 0x7f4bb80b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.513+0000 7f4bc01ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb80cdc80 con 0x7f4bb80cf950 2023-03-29T10:58:57.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.513+0000 7f4bc01ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb80cd6c0 con 0x7f4bb80d3a30 2023-03-29T10:58:57.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.513+0000 7f4bc01ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb80cd9a0 con 0x7f4bb815c950 2023-03-29T10:58:57.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.513+0000 7f4bbdf9b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f4bb80cf950 0x7f4bb80cfd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:47482/0 (socket says 172.21.15.111:47482) 2023-03-29T10:58:57.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.513+0000 7f4bbdf9b700 1 -- 172.21.15.111:0/3465704077 learned_addr learned my addr 172.21.15.111:0/3465704077 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:57.513 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.513+0000 7f4bbe79c700 1 --2- 172.21.15.111:0/3465704077 >> v2:172.21.15.111:3300/0 conn(0x7f4bb80d3a30 0x7f4bb80b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:57.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.513+0000 7f4bbd79a700 1 --2- 172.21.15.111:0/3465704077 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bb815c950 0x7f4bb80cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:57.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.513+0000 7f4bbcf99700 1 -- 172.21.15.111:0/3465704077 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975564053 0 0) 0x7f4bb80cdc80 con 0x7f4bb80cf950 2023-03-29T10:58:57.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.514+0000 7f4bbcf99700 1 -- 172.21.15.111:0/3465704077 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ba40035b0 con 0x7f4bb80cf950 2023-03-29T10:58:57.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.514+0000 7f4bbe79c700 1 -- 172.21.15.111:0/3465704077 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bb815c950 msgr2=0x7f4bb80cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:57.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.514+0000 7f4bbe79c700 1 --2- 172.21.15.111:0/3465704077 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bb815c950 0x7f4bb80cf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.514+0000 7f4bbe79c700 1 -- 172.21.15.111:0/3465704077 >> v1:172.21.15.111:6789/0 conn(0x7f4bb80cf950 legacy=0x7f4bb80cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:57.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.514+0000 7f4bbe79c700 1 -- 172.21.15.111:0/3465704077 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb80d16c0 con 0x7f4bb80d3a30 2023-03-29T10:58:57.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.514+0000 7f4bbe79c700 1 --2- 172.21.15.111:0/3465704077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb80d3a30 0x7f4bb80b5d90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009d90 tx=0x7f4bb000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd2778087ffb90c server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:57.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.514+0000 7f4bbcf99700 1 -- 172.21.15.111:0/3465704077 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb000e070 con 0x7f4bb80d3a30 2023-03-29T10:58:57.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.514+0000 7f4bbcf99700 1 -- 172.21.15.111:0/3465704077 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb000bb20 con 0x7f4bb80d3a30 2023-03-29T10:58:57.514 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.514+0000 7f4bbcf99700 1 -- 172.21.15.111:0/3465704077 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0012430 con 0x7f4bb80d3a30 2023-03-29T10:58:57.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.515+0000 7f4bc01ff700 1 -- 172.21.15.111:0/3465704077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb80d3a30 msgr2=0x7f4bb80b5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:57.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.515+0000 7f4bc01ff700 1 --2- 172.21.15.111:0/3465704077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb80d3a30 0x7f4bb80b5d90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009d90 tx=0x7f4bb000b330 comp rx=0 tx=0).stop 2023-03-29T10:58:57.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.515+0000 7f4bc01ff700 1 -- 172.21.15.111:0/3465704077 shutdown_connections 2023-03-29T10:58:57.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.515+0000 7f4bc01ff700 1 --2- 172.21.15.111:0/3465704077 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb80d3a30 0x7f4bb80b5d90 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.515+0000 7f4bc01ff700 1 --2- 172.21.15.111:0/3465704077 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bb815c950 0x7f4bb80cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.515+0000 7f4bc01ff700 1 -- 172.21.15.111:0/3465704077 >> 172.21.15.111:0/3465704077 conn(0x7f4bb8071b10 msgr2=0x7f4bb8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:57.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.515+0000 7f4bc01ff700 1 -- 172.21.15.111:0/3465704077 shutdown_connections 2023-03-29T10:58:57.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.515+0000 7f4bc01ff700 1 -- 172.21.15.111:0/3465704077 wait complete. 2023-03-29T10:58:57.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.516+0000 7f4bc01ff700 1 Processor -- start 2023-03-29T10:58:57.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.517+0000 7f4bc01ff700 1 -- start start 2023-03-29T10:58:57.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.517+0000 7f4bc01ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb80cf950 0x7f4bb81539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.517+0000 7f4bc01ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bb80d3a30 0x7f4bb815f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.517+0000 7f4bc01ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bb815c950 0x7f4bb815f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.518+0000 7f4bc01ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb815fbf0 con 0x7f4bb80cf950 2023-03-29T10:58:57.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.518+0000 7f4bc01ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb81f6150 con 0x7f4bb80d3a30 2023-03-29T10:58:57.517 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.518+0000 7f4bc01ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb81f65e0 con 0x7f4bb815c950 2023-03-29T10:58:57.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.518+0000 7f4bbdf9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb80cf950 0x7f4bb81539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:57.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.518+0000 7f4bbdf9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb80cf950 0x7f4bb81539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51914/0 (socket says 172.21.15.111:51914) 2023-03-29T10:58:57.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.518+0000 7f4bbdf9b700 1 -- 172.21.15.111:0/1432288655 learned_addr learned my addr 172.21.15.111:0/1432288655 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:57.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.518+0000 7f4bbdf9b700 1 -- 172.21.15.111:0/1432288655 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bb815c950 msgr2=0x7f4bb815f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:57.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.518+0000 7f4bbdf9b700 1 --2- 172.21.15.111:0/1432288655 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bb815c950 0x7f4bb815f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.518+0000 7f4bbdf9b700 1 -- 172.21.15.111:0/1432288655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bb80d3a30 msgr2=0x7f4bb815f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:57.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.518+0000 7f4bbdf9b700 1 --2- 172.21.15.111:0/1432288655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bb80d3a30 0x7f4bb815f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.518+0000 7f4bbdf9b700 1 -- 172.21.15.111:0/1432288655 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb0009a70 con 0x7f4bb80cf950 2023-03-29T10:58:57.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.518+0000 7f4bbdf9b700 1 --2- 172.21.15.111:0/1432288655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb80cf950 0x7f4bb81539c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400c4d0 tx=0x7f4bb400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:57.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.519+0000 7f4bae7fc700 1 -- 172.21.15.111:0/1432288655 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4005d70 con 0x7f4bb80cf950 2023-03-29T10:58:57.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.519+0000 7f4bae7fc700 1 -- 172.21.15.111:0/1432288655 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb40070d0 con 0x7f4bb80cf950 2023-03-29T10:58:57.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.519+0000 7f4bc01ff700 1 -- 172.21.15.111:0/1432288655 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb80cf630 con 0x7f4bb80cf950 2023-03-29T10:58:57.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.519+0000 7f4bae7fc700 1 -- 172.21.15.111:0/1432288655 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4007460 con 0x7f4bb80cf950 2023-03-29T10:58:57.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.519+0000 7f4bc01ff700 1 -- 172.21.15.111:0/1432288655 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb80d19a0 con 0x7f4bb80cf950 2023-03-29T10:58:57.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.520+0000 7f4bc01ff700 1 -- 172.21.15.111:0/1432288655 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb80b62d0 con 0x7f4bb80cf950 2023-03-29T10:58:57.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.520+0000 7f4bae7fc700 1 -- 172.21.15.111:0/1432288655 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f4bb40075c0 con 0x7f4bb80cf950 2023-03-29T10:58:57.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.521+0000 7f4bae7fc700 1 --2- 172.21.15.111:0/1432288655 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4b9c069de0 0x7f4b9c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.521+0000 7f4bae7fc700 1 -- 172.21.15.111:0/1432288655 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4bb4087850 con 0x7f4bb80cf950 2023-03-29T10:58:57.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.522+0000 7f4bbd79a700 1 --2- 172.21.15.111:0/1432288655 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4b9c069de0 0x7f4b9c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:57.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.522+0000 7f4bbd79a700 1 --2- 172.21.15.111:0/1432288655 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4b9c069de0 0x7f4b9c06c290 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80d08f0 tx=0x7f4ba8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:57.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.525+0000 7f4bae7fc700 1 -- 172.21.15.111:0/1432288655 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f4bb4052dc0 con 0x7f4bb80cf950 2023-03-29T10:58:57.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.722+0000 7f4bc01ff700 1 -- 172.21.15.111:0/1432288655 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bb8074430 con 0x7f4bb80cf950 2023-03-29T10:58:57.722 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.723+0000 7f4bae7fc700 1 -- 172.21.15.111:0/1432288655 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19892 (secure 0 0 0) 0x7f4bb40563e0 con 0x7f4bb80cf950 2023-03-29T10:58:57.723 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:57.723 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":30,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:58:09.476334+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.26","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:58:57.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.725+0000 7f4bc01ff700 1 -- 172.21.15.111:0/1432288655 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4b9c069de0 msgr2=0x7f4b9c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:57.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.725+0000 7f4bc01ff700 1 --2- 172.21.15.111:0/1432288655 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4b9c069de0 0x7f4b9c06c290 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4bb80d08f0 tx=0x7f4ba8008040 comp rx=0 tx=0).stop 2023-03-29T10:58:57.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.725+0000 7f4bc01ff700 1 -- 172.21.15.111:0/1432288655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb80cf950 msgr2=0x7f4bb81539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:57.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.725+0000 7f4bc01ff700 1 --2- 172.21.15.111:0/1432288655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb80cf950 0x7f4bb81539c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400c4d0 tx=0x7f4bb400c890 comp rx=0 tx=0).stop 2023-03-29T10:58:57.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.726+0000 7f4bc01ff700 1 -- 172.21.15.111:0/1432288655 shutdown_connections 2023-03-29T10:58:57.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.726+0000 7f4bc01ff700 1 --2- 172.21.15.111:0/1432288655 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4b9c069de0 0x7f4b9c06c290 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.726+0000 7f4bc01ff700 1 --2- 172.21.15.111:0/1432288655 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bb80cf950 0x7f4bb81539c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.726+0000 7f4bc01ff700 1 --2- 172.21.15.111:0/1432288655 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bb815c950 0x7f4bb815f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.726+0000 7f4bc01ff700 1 --2- 172.21.15.111:0/1432288655 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bb80d3a30 0x7f4bb815f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.726+0000 7f4bc01ff700 1 -- 172.21.15.111:0/1432288655 >> 172.21.15.111:0/1432288655 conn(0x7f4bb8071b10 msgr2=0x7f4bb8071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:57.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.726+0000 7f4bc01ff700 1 -- 172.21.15.111:0/1432288655 shutdown_connections 2023-03-29T10:58:57.726 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.726+0000 7f4bc01ff700 1 -- 172.21.15.111:0/1432288655 wait complete. 2023-03-29T10:58:57.742 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-03-29T10:58:57.743 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-29T10:58:57.926 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.926+0000 7fea579ff700 1 Processor -- start 2023-03-29T10:58:57.926 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.926+0000 7fea579ff700 1 -- start start 2023-03-29T10:58:57.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.927+0000 7fea579ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fea50158d30 0x7fea500ca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.927+0000 7fea579ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea500cda50 0x7fea500ca870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.927+0000 7fea579ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea500cc9b0 con 0x7fea50070d60 2023-03-29T10:58:57.927 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.927+0000 7fea579ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea50071dd0 con 0x7fea50158d30 2023-03-29T10:58:57.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.927+0000 7fea579ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea500ccc90 con 0x7fea500cda50 2023-03-29T10:58:57.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.928+0000 7fea5579b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fea50158d30 0x7fea500ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:57.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.928+0000 7fea5579b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fea50158d30 0x7fea500ca330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51916/0 (socket says 172.21.15.111:51916) 2023-03-29T10:58:57.928 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.928+0000 7fea5579b700 1 -- 172.21.15.111:0/1553850057 learned_addr learned my addr 172.21.15.111:0/1553850057 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:57.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.928+0000 7fea54f9a700 1 --2- 172.21.15.111:0/1553850057 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea500cda50 0x7fea500ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:57.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.928+0000 7fea5579b700 1 -- 172.21.15.111:0/1553850057 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea500cda50 msgr2=0x7fea500ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:57.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.928+0000 7fea5579b700 1 --2- 172.21.15.111:0/1553850057 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea500cda50 0x7fea500ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.928+0000 7fea5579b700 1 -- 172.21.15.111:0/1553850057 >> v1:172.21.15.119:6789/0 conn(0x7fea50070d60 legacy=0x7fea5015c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:57.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.928+0000 7fea5579b700 1 -- 172.21.15.111:0/1553850057 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea500ccf70 con 0x7fea50158d30 2023-03-29T10:58:57.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.928+0000 7fea5579b700 1 --2- 172.21.15.111:0/1553850057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea50158d30 0x7fea500ca330 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00bb90 tx=0x7fea4c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ba2174deeff270 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:57.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.929+0000 7fea47fff700 1 -- 172.21.15.111:0/1553850057 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c00dc40 con 0x7fea50158d30 2023-03-29T10:58:57.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.929+0000 7fea47fff700 1 -- 172.21.15.111:0/1553850057 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4c00dda0 con 0x7fea50158d30 2023-03-29T10:58:57.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.929+0000 7fea47fff700 1 -- 172.21.15.111:0/1553850057 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c017630 con 0x7fea50158d30 2023-03-29T10:58:57.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.929+0000 7fea579ff700 1 -- 172.21.15.111:0/1553850057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea50158d30 msgr2=0x7fea500ca330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:57.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.929+0000 7fea579ff700 1 --2- 172.21.15.111:0/1553850057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea50158d30 0x7fea500ca330 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00bb90 tx=0x7fea4c00bea0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.930 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.929+0000 7fea579ff700 1 -- 172.21.15.111:0/1553850057 shutdown_connections 2023-03-29T10:58:57.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.929+0000 7fea579ff700 1 --2- 172.21.15.111:0/1553850057 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea50158d30 0x7fea500ca330 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.929+0000 7fea579ff700 1 --2- 172.21.15.111:0/1553850057 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea500cda50 0x7fea500ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.929+0000 7fea579ff700 1 -- 172.21.15.111:0/1553850057 >> 172.21.15.111:0/1553850057 conn(0x7fea500cb6c0 msgr2=0x7fea500cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:57.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.929+0000 7fea579ff700 1 -- 172.21.15.111:0/1553850057 shutdown_connections 2023-03-29T10:58:57.931 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.929+0000 7fea579ff700 1 -- 172.21.15.111:0/1553850057 wait complete. 2023-03-29T10:58:57.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.932+0000 7fea579ff700 1 Processor -- start 2023-03-29T10:58:57.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.932+0000 7fea579ff700 1 -- start start 2023-03-29T10:58:57.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.933+0000 7fea579ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea50070d60 0x7fea50156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.933+0000 7fea579ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea500cda50 0x7fea500c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.933+0000 7fea579ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea50158d30 0x7fea500c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.933+0000 7fea579ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea500a74a0 con 0x7fea50070d60 2023-03-29T10:58:57.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.933+0000 7fea579ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea500a7760 con 0x7fea50158d30 2023-03-29T10:58:57.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.933+0000 7fea579ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea500a7a40 con 0x7fea500cda50 2023-03-29T10:58:57.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.933+0000 7fea5579b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea50070d60 0x7fea50156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:57.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.933+0000 7fea5579b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea50070d60 0x7fea50156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51920/0 (socket says 172.21.15.111:51920) 2023-03-29T10:58:57.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.933+0000 7fea5579b700 1 -- 172.21.15.111:0/920462630 learned_addr learned my addr 172.21.15.111:0/920462630 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:57.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.933+0000 7fea54f9a700 1 --2- 172.21.15.111:0/920462630 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea500cda50 0x7fea500c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:57.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.933+0000 7fea55f9c700 1 --2- 172.21.15.111:0/920462630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea50158d30 0x7fea500c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:57.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.934+0000 7fea5579b700 1 -- 172.21.15.111:0/920462630 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea500cda50 msgr2=0x7fea500c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:57.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.934+0000 7fea5579b700 1 --2- 172.21.15.111:0/920462630 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea500cda50 0x7fea500c4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.934+0000 7fea5579b700 1 -- 172.21.15.111:0/920462630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea50158d30 msgr2=0x7fea500c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:57.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.934+0000 7fea5579b700 1 --2- 172.21.15.111:0/920462630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea50158d30 0x7fea500c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:57.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.934+0000 7fea5579b700 1 -- 172.21.15.111:0/920462630 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c00b7f0 con 0x7fea50070d60 2023-03-29T10:58:57.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.934+0000 7fea5579b700 1 --2- 172.21.15.111:0/920462630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea50070d60 0x7fea50156e80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fea4c019040 tx=0x7fea4c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:57.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.934+0000 7fea45ffb700 1 -- 172.21.15.111:0/920462630 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c017e10 con 0x7fea50070d60 2023-03-29T10:58:57.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.934+0000 7fea45ffb700 1 -- 172.21.15.111:0/920462630 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4c004050 con 0x7fea50070d60 2023-03-29T10:58:57.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.934+0000 7fea579ff700 1 -- 172.21.15.111:0/920462630 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea500cd530 con 0x7fea50070d60 2023-03-29T10:58:57.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.934+0000 7fea45ffb700 1 -- 172.21.15.111:0/920462630 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4c004400 con 0x7fea50070d60 2023-03-29T10:58:57.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.934+0000 7fea579ff700 1 -- 172.21.15.111:0/920462630 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea500cd250 con 0x7fea50070d60 2023-03-29T10:58:57.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.935+0000 7fea579ff700 1 -- 172.21.15.111:0/920462630 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea500711e0 con 0x7fea50070d60 2023-03-29T10:58:57.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.936+0000 7fea45ffb700 1 -- 172.21.15.111:0/920462630 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fea4c015070 con 0x7fea50070d60 2023-03-29T10:58:57.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.937+0000 7fea45ffb700 1 --2- 172.21.15.111:0/920462630 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fea34069de0 0x7fea3406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:57.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.937+0000 7fea45ffb700 1 -- 172.21.15.111:0/920462630 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fea4c089810 con 0x7fea50070d60 2023-03-29T10:58:57.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.937+0000 7fea54f9a700 1 --2- 172.21.15.111:0/920462630 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fea34069de0 0x7fea3406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:57.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.938+0000 7fea54f9a700 1 --2- 172.21.15.111:0/920462630 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fea34069de0 0x7fea3406c290 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fea500b24e0 tx=0x7fea40008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:57.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:57.941+0000 7fea45ffb700 1 -- 172.21.15.111:0/920462630 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fea4c054e20 con 0x7fea50070d60 2023-03-29T10:58:58.176 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.176+0000 7fea579ff700 1 -- 172.21.15.111:0/920462630 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea500711e0 con 0x7fea50070d60 2023-03-29T10:58:58.177 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.177+0000 7fea45ffb700 1 -- 172.21.15.111:0/920462630 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fea4c058440 con 0x7fea50070d60 2023-03-29T10:58:58.177 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:58.177 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/556826332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":556826332},{"type":"v1","addr":"172.21.15.111:6837","nonce":556826332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/814623168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":814623168},{"type":"v1","addr":"172.21.15.111:6835","nonce":814623168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/514422035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":514422035},{"type":"v1","addr":"172.21.15.119:6834","nonce":514422035}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-29T10:57:27.824135+0000","modified":"2023-03-29T10:57:53.367898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/3160800581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":3160800581},{"type":"v1","addr":"172.21.15.119:6835","nonce":3160800581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-03-29T10:58:58.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.179+0000 7fea579ff700 1 -- 172.21.15.111:0/920462630 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fea34069de0 msgr2=0x7fea3406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:58.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.179+0000 7fea579ff700 1 --2- 172.21.15.111:0/920462630 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fea34069de0 0x7fea3406c290 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fea500b24e0 tx=0x7fea40008040 comp rx=0 tx=0).stop 2023-03-29T10:58:58.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.179+0000 7fea579ff700 1 -- 172.21.15.111:0/920462630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea50070d60 msgr2=0x7fea50156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:58.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.179+0000 7fea579ff700 1 --2- 172.21.15.111:0/920462630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea50070d60 0x7fea50156e80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fea4c019040 tx=0x7fea4c004750 comp rx=0 tx=0).stop 2023-03-29T10:58:58.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.180+0000 7fea579ff700 1 -- 172.21.15.111:0/920462630 shutdown_connections 2023-03-29T10:58:58.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.180+0000 7fea579ff700 1 --2- 172.21.15.111:0/920462630 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fea34069de0 0x7fea3406c290 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:58.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.180+0000 7fea579ff700 1 --2- 172.21.15.111:0/920462630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fea50070d60 0x7fea50156e80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:58.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.180+0000 7fea579ff700 1 --2- 172.21.15.111:0/920462630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea50158d30 0x7fea500c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:58.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.180+0000 7fea579ff700 1 --2- 172.21.15.111:0/920462630 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fea500cda50 0x7fea500c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:58.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.180+0000 7fea579ff700 1 -- 172.21.15.111:0/920462630 >> 172.21.15.111:0/920462630 conn(0x7fea500cb6c0 msgr2=0x7fea500c7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:58.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.180+0000 7fea579ff700 1 -- 172.21.15.111:0/920462630 shutdown_connections 2023-03-29T10:58:58.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.180+0000 7fea579ff700 1 -- 172.21.15.111:0/920462630 wait complete. 2023-03-29T10:58:58.182 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2023-03-29T10:58:58.197 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-29T10:58:58.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.381+0000 7fd7be5ff700 1 Processor -- start 2023-03-29T10:58:58.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.381+0000 7fd7be5ff700 1 -- start start 2023-03-29T10:58:58.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.382+0000 7fd7be5ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd7b815d2d0 0x7fd7b815d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:58.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.382+0000 7fd7be5ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b815dcb0 con 0x7fd7b8071420 2023-03-29T10:58:58.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.382+0000 7fd7be5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b815e150 con 0x7fd7b80b2ed0 2023-03-29T10:58:58.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.382+0000 7fd7be5ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b80718c0 con 0x7fd7b815d2d0 2023-03-29T10:58:58.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.382+0000 7fd7b7fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd7b815d2d0 0x7fd7b815d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:58.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.382+0000 7fd7b77fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fd7b8071420 0x7fd7b80b27c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:40986/0 (socket says 172.21.15.111:40986) 2023-03-29T10:58:58.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.382+0000 7fd7b77fe700 1 -- 172.21.15.111:0/1199848264 learned_addr learned my addr 172.21.15.111:0/1199848264 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:58.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.383+0000 7fd7b7fff700 1 -- 172.21.15.111:0/1199848264 >> v1:172.21.15.119:6789/0 conn(0x7fd7b80b2ed0 legacy=0x7fd7b80cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:58.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.383+0000 7fd7b7fff700 1 -- 172.21.15.111:0/1199848264 >> v1:172.21.15.111:6789/0 conn(0x7fd7b8071420 legacy=0x7fd7b80b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:58.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.383+0000 7fd7b7fff700 1 -- 172.21.15.111:0/1199848264 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b80b35b0 con 0x7fd7b815d2d0 2023-03-29T10:58:58.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.383+0000 7fd7b6ffd700 1 -- 172.21.15.111:0/1199848264 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2775153029 0 0) 0x7fd7b815dcb0 con 0x7fd7b8071420 2023-03-29T10:58:58.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.383+0000 7fd7b7fff700 1 --2- 172.21.15.111:0/1199848264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b815d2d0 0x7fd7b815d6a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009840 tx=0x7fd7a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c020532313e646ae server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:58.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.383+0000 7fd7b6ffd700 1 -- 172.21.15.111:0/1199848264 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8011830 con 0x7fd7b815d2d0 2023-03-29T10:58:58.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.383+0000 7fd7b6ffd700 1 -- 172.21.15.111:0/1199848264 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a8011990 con 0x7fd7b815d2d0 2023-03-29T10:58:58.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.383+0000 7fd7b6ffd700 1 -- 172.21.15.111:0/1199848264 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8011d20 con 0x7fd7b815d2d0 2023-03-29T10:58:58.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.384+0000 7fd7be5ff700 1 -- 172.21.15.111:0/1199848264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b815d2d0 msgr2=0x7fd7b815d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:58.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.384+0000 7fd7be5ff700 1 --2- 172.21.15.111:0/1199848264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b815d2d0 0x7fd7b815d6a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009840 tx=0x7fd7a8009b50 comp rx=0 tx=0).stop 2023-03-29T10:58:58.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.384+0000 7fd7be5ff700 1 -- 172.21.15.111:0/1199848264 shutdown_connections 2023-03-29T10:58:58.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.384+0000 7fd7be5ff700 1 --2- 172.21.15.111:0/1199848264 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b815d2d0 0x7fd7b815d6a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:58.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.384+0000 7fd7be5ff700 1 -- 172.21.15.111:0/1199848264 >> 172.21.15.111:0/1199848264 conn(0x7fd7b806eec0 msgr2=0x7fd7b806f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:58.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.384+0000 7fd7be5ff700 1 -- 172.21.15.111:0/1199848264 shutdown_connections 2023-03-29T10:58:58.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.384+0000 7fd7be5ff700 1 -- 172.21.15.111:0/1199848264 wait complete. 2023-03-29T10:58:58.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.386+0000 7fd7be5ff700 1 Processor -- start 2023-03-29T10:58:58.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.386+0000 7fd7be5ff700 1 -- start start 2023-03-29T10:58:58.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.387+0000 7fd7be5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7b8071420 0x7fd7b80b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:58.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.387+0000 7fd7be5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd7b80b2ed0 0x7fd7b80b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:58.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.387+0000 7fd7be5ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b815d2d0 0x7fd7b80c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:58.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.387+0000 7fd7be5ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b8157380 con 0x7fd7b815d2d0 2023-03-29T10:58:58.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.387+0000 7fd7be5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b81575d0 con 0x7fd7b8071420 2023-03-29T10:58:58.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.387+0000 7fd7be5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b81578b0 con 0x7fd7b80b2ed0 2023-03-29T10:58:58.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.387+0000 7fd7bcb9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b815d2d0 0x7fd7b80c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:58.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.387+0000 7fd7bcb9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b815d2d0 0x7fd7b80c6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:51998/0 (socket says 172.21.15.111:51998) 2023-03-29T10:58:58.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.387+0000 7fd7bcb9c700 1 -- 172.21.15.111:0/279520672 learned_addr learned my addr 172.21.15.111:0/279520672 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:58.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.387+0000 7fd7b77fe700 1 --2- 172.21.15.111:0/279520672 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd7b80b2ed0 0x7fd7b80b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:58.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.387+0000 7fd7b7fff700 1 --2- 172.21.15.111:0/279520672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7b8071420 0x7fd7b80b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:58.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.387+0000 7fd7bcb9c700 1 -- 172.21.15.111:0/279520672 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd7b80b2ed0 msgr2=0x7fd7b80b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:58.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.387+0000 7fd7bcb9c700 1 --2- 172.21.15.111:0/279520672 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd7b80b2ed0 0x7fd7b80b3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:58.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.387+0000 7fd7bcb9c700 1 -- 172.21.15.111:0/279520672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7b8071420 msgr2=0x7fd7b80b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:58.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.387+0000 7fd7bcb9c700 1 --2- 172.21.15.111:0/279520672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7b8071420 0x7fd7b80b19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:58.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.387+0000 7fd7bcb9c700 1 -- 172.21.15.111:0/279520672 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a80094a0 con 0x7fd7b815d2d0 2023-03-29T10:58:58.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.388+0000 7fd7bcb9c700 1 --2- 172.21.15.111:0/279520672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b815d2d0 0x7fd7b80c6d60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000cbc0 tx=0x7fd7b000ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:58.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.388+0000 7fd7b4ff9700 1 -- 172.21.15.111:0/279520672 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0003c00 con 0x7fd7b815d2d0 2023-03-29T10:58:58.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.388+0000 7fd7b4ff9700 1 -- 172.21.15.111:0/279520672 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b0003d60 con 0x7fd7b815d2d0 2023-03-29T10:58:58.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.388+0000 7fd7be5ff700 1 -- 172.21.15.111:0/279520672 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b80ced20 con 0x7fd7b815d2d0 2023-03-29T10:58:58.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.388+0000 7fd7b4ff9700 1 -- 172.21.15.111:0/279520672 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b000aae0 con 0x7fd7b815d2d0 2023-03-29T10:58:58.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.388+0000 7fd7be5ff700 1 -- 172.21.15.111:0/279520672 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b80cea40 con 0x7fd7b815d2d0 2023-03-29T10:58:58.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.389+0000 7fd7be5ff700 1 -- 172.21.15.111:0/279520672 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b80cf000 con 0x7fd7b815d2d0 2023-03-29T10:58:58.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.390+0000 7fd7b4ff9700 1 -- 172.21.15.111:0/279520672 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fd7b000ac40 con 0x7fd7b815d2d0 2023-03-29T10:58:58.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.391+0000 7fd7b4ff9700 1 --2- 172.21.15.111:0/279520672 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd7a4069de0 0x7fd7a406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:58.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.391+0000 7fd7b4ff9700 1 -- 172.21.15.111:0/279520672 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fd7b00882c0 con 0x7fd7b815d2d0 2023-03-29T10:58:58.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.391+0000 7fd7b7fff700 1 --2- 172.21.15.111:0/279520672 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd7a4069de0 0x7fd7a406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:58.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.392+0000 7fd7b7fff700 1 --2- 172.21.15.111:0/279520672 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd7a4069de0 0x7fd7a406c290 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80090b0 tx=0x7fd7a801c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:58.394 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.395+0000 7fd7b4ff9700 1 -- 172.21.15.111:0/279520672 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fd7b00537a0 con 0x7fd7b815d2d0 2023-03-29T10:58:58.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.635+0000 7fd7be5ff700 1 -- 172.21.15.111:0/279520672 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7b8157a70 con 0x7fd7b815d2d0 2023-03-29T10:58:58.635 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.636+0000 7fd7b4ff9700 1 -- 172.21.15.111:0/279520672 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4670 (secure 0 0 0) 0x7fd7b0056dc0 con 0x7fd7b815d2d0 2023-03-29T10:58:58.636 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:58.636 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4253,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6837/556826332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":556826332},{"type":"v1","addr":"172.21.15.111:6837","nonce":556826332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/814623168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":814623168},{"type":"v1","addr":"172.21.15.111:6835","nonce":814623168}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4293,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/514422035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":514422035},{"type":"v1","addr":"172.21.15.119:6834","nonce":514422035}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-29T10:57:27.824135+0000","modified":"2023-03-29T10:57:53.367898+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/3160800581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":3160800581},{"type":"v1","addr":"172.21.15.119:6835","nonce":3160800581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-03-29T10:58:58.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.638+0000 7fd7be5ff700 1 -- 172.21.15.111:0/279520672 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd7a4069de0 msgr2=0x7fd7a406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:58.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.638+0000 7fd7be5ff700 1 --2- 172.21.15.111:0/279520672 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd7a4069de0 0x7fd7a406c290 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80090b0 tx=0x7fd7a801c040 comp rx=0 tx=0).stop 2023-03-29T10:58:58.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.638+0000 7fd7be5ff700 1 -- 172.21.15.111:0/279520672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b815d2d0 msgr2=0x7fd7b80c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:58.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.638+0000 7fd7be5ff700 1 --2- 172.21.15.111:0/279520672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b815d2d0 0x7fd7b80c6d60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd7b000cbc0 tx=0x7fd7b000ced0 comp rx=0 tx=0).stop 2023-03-29T10:58:58.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.638+0000 7fd7be5ff700 1 -- 172.21.15.111:0/279520672 shutdown_connections 2023-03-29T10:58:58.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.638+0000 7fd7be5ff700 1 --2- 172.21.15.111:0/279520672 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd7a4069de0 0x7fd7a406c290 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:58.638 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.638+0000 7fd7be5ff700 1 --2- 172.21.15.111:0/279520672 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd7b815d2d0 0x7fd7b80c6d60 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:58.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.638+0000 7fd7be5ff700 1 --2- 172.21.15.111:0/279520672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7b8071420 0x7fd7b80b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:58.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.638+0000 7fd7be5ff700 1 --2- 172.21.15.111:0/279520672 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd7b80b2ed0 0x7fd7b80b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:58.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.638+0000 7fd7be5ff700 1 -- 172.21.15.111:0/279520672 >> 172.21.15.111:0/279520672 conn(0x7fd7b806eec0 msgr2=0x7fd7b8153110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:58.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.638+0000 7fd7be5ff700 1 -- 172.21.15.111:0/279520672 shutdown_connections 2023-03-29T10:58:58.639 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.638+0000 7fd7be5ff700 1 -- 172.21.15.111:0/279520672 wait complete. 2023-03-29T10:58:58.639 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 9 2023-03-29T10:58:58.655 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:58:58.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.838+0000 7f60ff5ff700 1 Processor -- start 2023-03-29T10:58:58.838 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.838+0000 7f60ff5ff700 1 -- start start 2023-03-29T10:58:58.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.839+0000 7f60ff5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60f80b2430 0x7f60f806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:58.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.839+0000 7f60ff5ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f60f80b5db0 0x7f60f806c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:58.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.839+0000 7f60ff5ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60f80d26f0 con 0x7f60f8160f40 2023-03-29T10:58:58.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.839+0000 7f60ff5ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f80cd750 con 0x7f60f80b5db0 2023-03-29T10:58:58.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.839+0000 7f60ff5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f80cda30 con 0x7f60f80b2430 2023-03-29T10:58:58.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.839+0000 7f60fcb9a700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f60f8160f40 0x7f60f80d0d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:40992/0 (socket says 172.21.15.111:40992) 2023-03-29T10:58:58.839 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.839+0000 7f60fcb9a700 1 -- 172.21.15.111:0/3076452223 learned_addr learned my addr 172.21.15.111:0/3076452223 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:58.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.840+0000 7f60fdb9c700 1 --2- 172.21.15.111:0/3076452223 >> v2:172.21.15.111:3300/0 conn(0x7f60f80b5db0 0x7f60f806c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:58.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.840+0000 7f60fd39b700 1 --2- 172.21.15.111:0/3076452223 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60f80b2430 0x7f60f806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:58.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.840+0000 7f60effff700 1 -- 172.21.15.111:0/3076452223 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190324604 0 0) 0x7f60f80d26f0 con 0x7f60f8160f40 2023-03-29T10:58:58.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.840+0000 7f60effff700 1 -- 172.21.15.111:0/3076452223 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60e40035b0 con 0x7f60f8160f40 2023-03-29T10:58:58.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.840+0000 7f60fdb9c700 1 -- 172.21.15.111:0/3076452223 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60f80b2430 msgr2=0x7f60f806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:58.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.840+0000 7f60fdb9c700 1 --2- 172.21.15.111:0/3076452223 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60f80b2430 0x7f60f806bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:58.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.840+0000 7f60fdb9c700 1 -- 172.21.15.111:0/3076452223 >> v1:172.21.15.111:6789/0 conn(0x7f60f8160f40 legacy=0x7f60f80d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:58.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.840+0000 7f60fdb9c700 1 -- 172.21.15.111:0/3076452223 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f80d2940 con 0x7f60f80b5db0 2023-03-29T10:58:58.840 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.840+0000 7f60fdb9c700 1 --2- 172.21.15.111:0/3076452223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60f80b5db0 0x7f60f806c130 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009d90 tx=0x7f60f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e39f7570654bd75 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:58.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.840+0000 7f60effff700 1 -- 172.21.15.111:0/3076452223 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f000e070 con 0x7f60f80b5db0 2023-03-29T10:58:58.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.841+0000 7f60effff700 1 -- 172.21.15.111:0/3076452223 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f000bb20 con 0x7f60f80b5db0 2023-03-29T10:58:58.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.841+0000 7f60effff700 1 -- 172.21.15.111:0/3076452223 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f0012430 con 0x7f60f80b5db0 2023-03-29T10:58:58.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.841+0000 7f60ff5ff700 1 -- 172.21.15.111:0/3076452223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60f80b5db0 msgr2=0x7f60f806c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:58.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.841+0000 7f60ff5ff700 1 --2- 172.21.15.111:0/3076452223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60f80b5db0 0x7f60f806c130 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f60f0009d90 tx=0x7f60f000b330 comp rx=0 tx=0).stop 2023-03-29T10:58:58.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.841+0000 7f60ff5ff700 1 -- 172.21.15.111:0/3076452223 shutdown_connections 2023-03-29T10:58:58.841 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.841+0000 7f60ff5ff700 1 --2- 172.21.15.111:0/3076452223 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60f80b5db0 0x7f60f806c130 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:58.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.841+0000 7f60ff5ff700 1 --2- 172.21.15.111:0/3076452223 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60f80b2430 0x7f60f806bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:58.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.841+0000 7f60ff5ff700 1 -- 172.21.15.111:0/3076452223 >> 172.21.15.111:0/3076452223 conn(0x7f60f8071b10 msgr2=0x7f60f8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:58.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.841+0000 7f60ff5ff700 1 -- 172.21.15.111:0/3076452223 shutdown_connections 2023-03-29T10:58:58.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.841+0000 7f60ff5ff700 1 -- 172.21.15.111:0/3076452223 wait complete. 2023-03-29T10:58:58.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.843+0000 7f60ff5ff700 1 Processor -- start 2023-03-29T10:58:58.842 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.843+0000 7f60ff5ff700 1 -- start start 2023-03-29T10:58:58.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.844+0000 7f60ff5ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60f80b2430 0x7f60f80cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:58.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.844+0000 7f60ff5ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60f80b5db0 0x7f60f81f1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:58.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.844+0000 7f60ff5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60f8160f40 0x7f60f81f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:58.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.844+0000 7f60ff5ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f80cbf80 con 0x7f60f80b5db0 2023-03-29T10:58:58.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.844+0000 7f60ff5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f80cca40 con 0x7f60f8160f40 2023-03-29T10:58:58.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.844+0000 7f60ff5ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f8148100 con 0x7f60f80b2430 2023-03-29T10:58:58.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.844+0000 7f60fcb9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60f80b5db0 0x7f60f81f1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:58.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.844+0000 7f60fcb9a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60f80b5db0 0x7f60f81f1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52004/0 (socket says 172.21.15.111:52004) 2023-03-29T10:58:58.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.844+0000 7f60fcb9a700 1 -- 172.21.15.111:0/4103251225 learned_addr learned my addr 172.21.15.111:0/4103251225 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:58.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.844+0000 7f60fd39b700 1 --2- 172.21.15.111:0/4103251225 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60f80b2430 0x7f60f80cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:58.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.844+0000 7f60fcb9a700 1 -- 172.21.15.111:0/4103251225 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60f80b2430 msgr2=0x7f60f80cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:58.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.844+0000 7f60fdb9c700 1 --2- 172.21.15.111:0/4103251225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60f8160f40 0x7f60f81f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:58.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.845+0000 7f60fcb9a700 1 --2- 172.21.15.111:0/4103251225 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60f80b2430 0x7f60f80cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:58.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.845+0000 7f60fcb9a700 1 -- 172.21.15.111:0/4103251225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60f8160f40 msgr2=0x7f60f81f17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:58.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.845+0000 7f60fcb9a700 1 --2- 172.21.15.111:0/4103251225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60f8160f40 0x7f60f81f17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:58.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.845+0000 7f60fcb9a700 1 -- 172.21.15.111:0/4103251225 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f0009a70 con 0x7f60f80b5db0 2023-03-29T10:58:58.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.845+0000 7f60fd39b700 1 --2- 172.21.15.111:0/4103251225 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60f80b2430 0x7f60f80cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:58.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.845+0000 7f60fdb9c700 1 --2- 172.21.15.111:0/4103251225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60f8160f40 0x7f60f81f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:58:58.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.845+0000 7f60fcb9a700 1 --2- 172.21.15.111:0/4103251225 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60f80b5db0 0x7f60f81f1260 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f60e8005910 tx=0x7f60e800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:58.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.845+0000 7f60edffb700 1 -- 172.21.15.111:0/4103251225 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e8005d70 con 0x7f60f80b5db0 2023-03-29T10:58:58.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.845+0000 7f60edffb700 1 -- 172.21.15.111:0/4103251225 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60e80070d0 con 0x7f60f80b5db0 2023-03-29T10:58:58.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.845+0000 7f60ff5ff700 1 -- 172.21.15.111:0/4103251225 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f80d1480 con 0x7f60f80b5db0 2023-03-29T10:58:58.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.845+0000 7f60edffb700 1 -- 172.21.15.111:0/4103251225 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60e8007440 con 0x7f60f80b5db0 2023-03-29T10:58:58.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.845+0000 7f60ff5ff700 1 -- 172.21.15.111:0/4103251225 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f80d2c20 con 0x7f60f80b5db0 2023-03-29T10:58:58.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.846+0000 7f60ff5ff700 1 -- 172.21.15.111:0/4103251225 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f80d1610 con 0x7f60f80b5db0 2023-03-29T10:58:58.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.847+0000 7f60edffb700 1 -- 172.21.15.111:0/4103251225 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f60e80075a0 con 0x7f60f80b5db0 2023-03-29T10:58:58.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.848+0000 7f60edffb700 1 --2- 172.21.15.111:0/4103251225 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f60dc069de0 0x7f60dc06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:58.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.848+0000 7f60edffb700 1 -- 172.21.15.111:0/4103251225 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f60e8087650 con 0x7f60f80b5db0 2023-03-29T10:58:58.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.848+0000 7f60fd39b700 1 --2- 172.21.15.111:0/4103251225 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f60dc069de0 0x7f60dc06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:58.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.849+0000 7f60fd39b700 1 --2- 172.21.15.111:0/4103251225 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f60dc069de0 0x7f60dc06c290 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f60f4009b60 tx=0x7f60f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:58.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:58.852+0000 7f60edffb700 1 -- 172.21.15.111:0/4103251225 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f60e804c160 con 0x7f60f80b5db0 2023-03-29T10:58:59.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.046+0000 7f60ff5ff700 1 -- 172.21.15.111:0/4103251225 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60f8148430 con 0x7f60f80b5db0 2023-03-29T10:58:59.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.047+0000 7f60edffb700 1 -- 172.21.15.111:0/4103251225 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+19892 (secure 0 0 0) 0x7f60e8052c70 con 0x7f60f80b5db0 2023-03-29T10:58:59.047 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:58:59.047 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":30,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:58:09.476334+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:57:26.192124+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-03-29T10:57:26.796777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-03-29T10:57:29.874795+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.26","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:58:59.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.049+0000 7f60ff5ff700 1 -- 172.21.15.111:0/4103251225 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f60dc069de0 msgr2=0x7f60dc06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:59.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.049+0000 7f60ff5ff700 1 --2- 172.21.15.111:0/4103251225 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f60dc069de0 0x7f60dc06c290 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f60f4009b60 tx=0x7f60f4008040 comp rx=0 tx=0).stop 2023-03-29T10:58:59.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.049+0000 7f60ff5ff700 1 -- 172.21.15.111:0/4103251225 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60f80b5db0 msgr2=0x7f60f81f1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:59.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.049+0000 7f60ff5ff700 1 --2- 172.21.15.111:0/4103251225 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60f80b5db0 0x7f60f81f1260 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f60e8005910 tx=0x7f60e800c860 comp rx=0 tx=0).stop 2023-03-29T10:58:59.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.049+0000 7f60ff5ff700 1 -- 172.21.15.111:0/4103251225 shutdown_connections 2023-03-29T10:58:59.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.049+0000 7f60ff5ff700 1 --2- 172.21.15.111:0/4103251225 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f60dc069de0 0x7f60dc06c290 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.049+0000 7f60ff5ff700 1 --2- 172.21.15.111:0/4103251225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60f8160f40 0x7f60f81f17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.049+0000 7f60ff5ff700 1 --2- 172.21.15.111:0/4103251225 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f60f80b2430 0x7f60f80cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.049+0000 7f60ff5ff700 1 --2- 172.21.15.111:0/4103251225 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f60f80b5db0 0x7f60f81f1260 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.049+0000 7f60ff5ff700 1 -- 172.21.15.111:0/4103251225 >> 172.21.15.111:0/4103251225 conn(0x7f60f8071b10 msgr2=0x7f60f806bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:59.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.050+0000 7f60ff5ff700 1 -- 172.21.15.111:0/4103251225 shutdown_connections 2023-03-29T10:58:59.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.050+0000 7f60ff5ff700 1 -- 172.21.15.111:0/4103251225 wait complete. 2023-03-29T10:58:59.067 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-03-29T10:58:59.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.251+0000 7f1f3ddff700 1 Processor -- start 2023-03-29T10:58:59.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.251+0000 7f1f3ddff700 1 -- start start 2023-03-29T10:58:59.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.253+0000 7f1f3ddff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1f38158da0 0x7f1f3806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:59.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.253+0000 7f1f3ddff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1f3806cc00 0x7f1f3806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:59.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.253+0000 7f1f3ddff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f380ca890 con 0x7f1f381589d0 2023-03-29T10:58:59.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.253+0000 7f1f3ddff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f380cd860 con 0x7f1f38158da0 2023-03-29T10:58:59.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.253+0000 7f1f3ddff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380ca5b0 con 0x7f1f3806cc00 2023-03-29T10:58:59.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.253+0000 7f1f2ffff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1f38158da0 0x7f1f3806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:59.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.253+0000 7f1f2ffff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1f38158da0 0x7f1f3806c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52016/0 (socket says 172.21.15.111:52016) 2023-03-29T10:58:59.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.253+0000 7f1f2ffff700 1 -- 172.21.15.111:0/1854031724 learned_addr learned my addr 172.21.15.111:0/1854031724 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:59.253 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.253+0000 7f1f37fff700 1 --2- 172.21.15.111:0/1854031724 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1f3806cc00 0x7f1f3806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:59.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.253+0000 7f1f2ffff700 1 -- 172.21.15.111:0/1854031724 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1f3806cc00 msgr2=0x7f1f3806a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:59.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.253+0000 7f1f2ffff700 1 --2- 172.21.15.111:0/1854031724 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1f3806cc00 0x7f1f3806a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.253+0000 7f1f2ffff700 1 -- 172.21.15.111:0/1854031724 >> v1:172.21.15.119:6789/0 conn(0x7f1f381589d0 legacy=0x7f1f380cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:59.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.253+0000 7f1f2ffff700 1 -- 172.21.15.111:0/1854031724 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f380cc670 con 0x7f1f38158da0 2023-03-29T10:58:59.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.254+0000 7f1f2ffff700 1 --2- 172.21.15.111:0/1854031724 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f38158da0 0x7f1f3806c6c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1f20006fd0 tx=0x7f1f20009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c413c849cd38db5c server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:59.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.254+0000 7f1f36ffd700 1 -- 172.21.15.111:0/1854031724 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f20005e30 con 0x7f1f38158da0 2023-03-29T10:58:59.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.254+0000 7f1f36ffd700 1 -- 172.21.15.111:0/1854031724 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f20004030 con 0x7f1f38158da0 2023-03-29T10:58:59.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.254+0000 7f1f36ffd700 1 -- 172.21.15.111:0/1854031724 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f200043e0 con 0x7f1f38158da0 2023-03-29T10:58:59.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.254+0000 7f1f3ddff700 1 -- 172.21.15.111:0/1854031724 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f38158da0 msgr2=0x7f1f3806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:59.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.254+0000 7f1f3ddff700 1 --2- 172.21.15.111:0/1854031724 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f38158da0 0x7f1f3806c6c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1f20006fd0 tx=0x7f1f20009bd0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.255+0000 7f1f3ddff700 1 -- 172.21.15.111:0/1854031724 shutdown_connections 2023-03-29T10:58:59.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.255+0000 7f1f3ddff700 1 --2- 172.21.15.111:0/1854031724 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1f3806cc00 0x7f1f3806a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.255+0000 7f1f3ddff700 1 --2- 172.21.15.111:0/1854031724 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f38158da0 0x7f1f3806c6c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.255+0000 7f1f3ddff700 1 -- 172.21.15.111:0/1854031724 >> 172.21.15.111:0/1854031724 conn(0x7f1f380cf720 msgr2=0x7f1f380cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:59.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.255+0000 7f1f3ddff700 1 -- 172.21.15.111:0/1854031724 shutdown_connections 2023-03-29T10:58:59.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.255+0000 7f1f3ddff700 1 -- 172.21.15.111:0/1854031724 wait complete. 2023-03-29T10:58:59.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.257+0000 7f1f3ddff700 1 Processor -- start 2023-03-29T10:58:59.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.257+0000 7f1f3ddff700 1 -- start start 2023-03-29T10:58:59.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.258+0000 7f1f3ddff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1f3806cc00 0x7f1f3815bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:59.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.258+0000 7f1f3ddff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f381589d0 0x7f1f380c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:59.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.258+0000 7f1f3ddff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f38158da0 0x7f1f381574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:59.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.258+0000 7f1f3ddff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3815c0e0 con 0x7f1f38158da0 2023-03-29T10:58:59.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.258+0000 7f1f3ddff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380c81e0 con 0x7f1f381589d0 2023-03-29T10:58:59.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.258+0000 7f1f3ddff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380c7330 con 0x7f1f3806cc00 2023-03-29T10:58:59.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.258+0000 7f1f37fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f38158da0 0x7f1f381574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:59.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.259+0000 7f1f37fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f38158da0 0x7f1f381574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52028/0 (socket says 172.21.15.111:52028) 2023-03-29T10:58:59.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.259+0000 7f1f37fff700 1 -- 172.21.15.111:0/4044990899 learned_addr learned my addr 172.21.15.111:0/4044990899 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:59.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.259+0000 7f1f2ffff700 1 --2- 172.21.15.111:0/4044990899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f381589d0 0x7f1f380c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:59.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.259+0000 7f1f377fe700 1 --2- 172.21.15.111:0/4044990899 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1f3806cc00 0x7f1f3815bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:59.259 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.259+0000 7f1f37fff700 1 -- 172.21.15.111:0/4044990899 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1f3806cc00 msgr2=0x7f1f3815bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:59.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.259+0000 7f1f37fff700 1 --2- 172.21.15.111:0/4044990899 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1f3806cc00 0x7f1f3815bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.259+0000 7f1f37fff700 1 -- 172.21.15.111:0/4044990899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f381589d0 msgr2=0x7f1f380c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:59.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.259+0000 7f1f37fff700 1 --2- 172.21.15.111:0/4044990899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f381589d0 0x7f1f380c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.259+0000 7f1f37fff700 1 -- 172.21.15.111:0/4044990899 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f20009500 con 0x7f1f38158da0 2023-03-29T10:58:59.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.259+0000 7f1f2ffff700 1 --2- 172.21.15.111:0/4044990899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f381589d0 0x7f1f380c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:59.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.259+0000 7f1f37fff700 1 --2- 172.21.15.111:0/4044990899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f38158da0 0x7f1f381574e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009dd0 tx=0x7f1f3000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:59.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.259+0000 7f1f34ff9700 1 -- 172.21.15.111:0/4044990899 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f30009c40 con 0x7f1f38158da0 2023-03-29T10:58:59.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.259+0000 7f1f34ff9700 1 -- 172.21.15.111:0/4044990899 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3000f040 con 0x7f1f38158da0 2023-03-29T10:58:59.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.259+0000 7f1f3ddff700 1 -- 172.21.15.111:0/4044990899 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f380ccc30 con 0x7f1f38158da0 2023-03-29T10:58:59.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.259+0000 7f1f34ff9700 1 -- 172.21.15.111:0/4044990899 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f30014480 con 0x7f1f38158da0 2023-03-29T10:58:59.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.260+0000 7f1f3ddff700 1 -- 172.21.15.111:0/4044990899 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f380cc950 con 0x7f1f38158da0 2023-03-29T10:58:59.260 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.260+0000 7f1f3ddff700 1 -- 172.21.15.111:0/4044990899 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3806a920 con 0x7f1f38158da0 2023-03-29T10:58:59.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.261+0000 7f1f34ff9700 1 -- 172.21.15.111:0/4044990899 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f1f300145e0 con 0x7f1f38158da0 2023-03-29T10:58:59.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.262+0000 7f1f34ff9700 1 --2- 172.21.15.111:0/4044990899 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1f18069de0 0x7f1f1806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:59.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.262+0000 7f1f34ff9700 1 -- 172.21.15.111:0/4044990899 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1f30088c20 con 0x7f1f38158da0 2023-03-29T10:58:59.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.263+0000 7f1f377fe700 1 --2- 172.21.15.111:0/4044990899 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1f18069de0 0x7f1f1806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:59.262 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.263+0000 7f1f377fe700 1 --2- 172.21.15.111:0/4044990899 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1f18069de0 0x7f1f1806c290 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1f28007fd0 tx=0x7f1f28009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:59.266 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.266+0000 7f1f34ff9700 1 -- 172.21.15.111:0/4044990899 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f1f300540a0 con 0x7f1f38158da0 2023-03-29T10:58:59.500 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.501+0000 7f1f3ddff700 1 -- 172.21.15.111:0/4044990899 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1f380c8370 con 0x7f1f38158da0 2023-03-29T10:58:59.680 INFO:tasks.ceph.mon.a.smithi111.stderr:2023-03-29T10:58:59.680+0000 7f79c2ff9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-03-29T10:58:59.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.690+0000 7f1f34ff9700 1 -- 172.21.15.111:0/4044990899 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1f30018090 con 0x7f1f38158da0 2023-03-29T10:58:59.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.692+0000 7f1f3ddff700 1 -- 172.21.15.111:0/4044990899 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1f18069de0 msgr2=0x7f1f1806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:59.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.692+0000 7f1f3ddff700 1 --2- 172.21.15.111:0/4044990899 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1f18069de0 0x7f1f1806c290 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1f28007fd0 tx=0x7f1f28009040 comp rx=0 tx=0).stop 2023-03-29T10:58:59.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.692+0000 7f1f3ddff700 1 -- 172.21.15.111:0/4044990899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f38158da0 msgr2=0x7f1f381574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:59.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.692+0000 7f1f3ddff700 1 --2- 172.21.15.111:0/4044990899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f38158da0 0x7f1f381574e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009dd0 tx=0x7f1f3000c3a0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.693+0000 7f1f3ddff700 1 -- 172.21.15.111:0/4044990899 shutdown_connections 2023-03-29T10:58:59.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.693+0000 7f1f3ddff700 1 --2- 172.21.15.111:0/4044990899 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1f18069de0 0x7f1f1806c290 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.693+0000 7f1f3ddff700 1 --2- 172.21.15.111:0/4044990899 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1f38158da0 0x7f1f381574e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.693+0000 7f1f3ddff700 1 --2- 172.21.15.111:0/4044990899 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f1f3806cc00 0x7f1f3815bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.693+0000 7f1f3ddff700 1 --2- 172.21.15.111:0/4044990899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f381589d0 0x7f1f380c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.693+0000 7f1f3ddff700 1 -- 172.21.15.111:0/4044990899 >> 172.21.15.111:0/4044990899 conn(0x7f1f380cf720 msgr2=0x7f1f3806d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:59.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.693+0000 7f1f3ddff700 1 -- 172.21.15.111:0/4044990899 shutdown_connections 2023-03-29T10:58:59.693 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.693+0000 7f1f3ddff700 1 -- 172.21.15.111:0/4044990899 wait complete. 2023-03-29T10:58:59.693 INFO:teuthology.orchestra.run.smithi111.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-03-29T10:58:59.700 INFO:tasks.ceph.mds.d.smithi119.stderr: -5846> 2023-03-29T10:57:33.551+0000 7ff9fd1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-03-29T10:58:59.710 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-03-29T10:58:59.752 INFO:tasks.ceph.mds.d.smithi119.stdout:starting mds.d at 2023-03-29T10:58:59.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.898+0000 7fbea51ff700 1 Processor -- start 2023-03-29T10:58:59.898 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.899+0000 7fbea51ff700 1 -- start start 2023-03-29T10:58:59.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.900+0000 7fbea51ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbea01589d0 0x7fbea00cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:59.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.900+0000 7fbea51ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbea00cd690 0x7fbea006c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:59.899 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.900+0000 7fbea51ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea00ca890 con 0x7fbea0158da0 2023-03-29T10:58:59.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.900+0000 7fbea51ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbea00ca5b0 con 0x7fbea00cd690 2023-03-29T10:58:59.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.900+0000 7fbea51ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea00cc670 con 0x7fbea01589d0 2023-03-29T10:58:59.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.900+0000 7fbe9f59e700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fbea0158da0 0x7fbea006a3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:40994/0 (socket says 172.21.15.111:40994) 2023-03-29T10:58:59.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.900+0000 7fbe9f59e700 1 -- 172.21.15.111:0/3847697349 learned_addr learned my addr 172.21.15.111:0/3847697349 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:58:59.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.900+0000 7fbe97fff700 1 --2- 172.21.15.111:0/3847697349 >> v2:172.21.15.119:3300/0 conn(0x7fbea00cd690 0x7fbea006c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:59.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.900+0000 7fbe9ed9d700 1 --2- 172.21.15.111:0/3847697349 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbea01589d0 0x7fbea00cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:59.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.900+0000 7fbe9e59c700 1 -- 172.21.15.111:0/3847697349 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747941332 0 0) 0x7fbea00ca890 con 0x7fbea0158da0 2023-03-29T10:58:59.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.900+0000 7fbe9e59c700 1 -- 172.21.15.111:0/3847697349 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe8c0035b0 con 0x7fbea0158da0 2023-03-29T10:58:59.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.901+0000 7fbe9e59c700 1 -- 172.21.15.111:0/3847697349 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1428325470 0 0) 0x7fbe8c0035b0 con 0x7fbea0158da0 2023-03-29T10:58:59.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.901+0000 7fbe9e59c700 1 -- 172.21.15.111:0/3847697349 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbea01589d0 msgr2=0x7fbea00cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:59.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.901+0000 7fbe9e59c700 1 --2- 172.21.15.111:0/3847697349 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbea01589d0 0x7fbea00cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.901+0000 7fbe9e59c700 1 -- 172.21.15.111:0/3847697349 >> v2:172.21.15.119:3300/0 conn(0x7fbea00cd690 msgr2=0x7fbea006c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:59.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.901+0000 7fbe9e59c700 1 --2- 172.21.15.111:0/3847697349 >> v2:172.21.15.119:3300/0 conn(0x7fbea00cd690 0x7fbea006c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.901+0000 7fbe9e59c700 1 -- 172.21.15.111:0/3847697349 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea00cc950 con 0x7fbea0158da0 2023-03-29T10:58:59.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.901+0000 7fbe9e59c700 1 -- 172.21.15.111:0/3847697349 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7fbe980019f0 con 0x7fbea0158da0 2023-03-29T10:58:59.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.901+0000 7fbe9e59c700 1 -- 172.21.15.111:0/3847697349 >> v1:172.21.15.111:6789/0 conn(0x7fbea0158da0 legacy=0x7fbea006a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:59.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.901+0000 7fbe9e59c700 1 --2- 172.21.15.111:0/3847697349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe8c003a00 0x7fbe8c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:59.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.901+0000 7fbe9e59c700 1 --2- 172.21.15.111:0/3847697349 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbea01589d0 0x7fbe8c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:59.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.901+0000 7fbe9e59c700 1 --2- 172.21.15.111:0/3847697349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe8c004d90 0x7fbe8c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:59.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.901+0000 7fbe9e59c700 1 -- 172.21.15.111:0/3847697349 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea00ca5b0 con 0x7fbe8c003a00 2023-03-29T10:58:59.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.901+0000 7fbe9e59c700 1 -- 172.21.15.111:0/3847697349 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea00cc670 con 0x7fbe8c004d90 2023-03-29T10:58:59.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.901+0000 7fbe9e59c700 1 -- 172.21.15.111:0/3847697349 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c00ba10 con 0x7fbea01589d0 2023-03-29T10:58:59.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.901+0000 7fbe9ed9d700 1 --2- 172.21.15.111:0/3847697349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe8c003a00 0x7fbe8c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:59.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.901+0000 7fbe97fff700 1 --2- 172.21.15.111:0/3847697349 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbea01589d0 0x7fbe8c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:59.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.901+0000 7fbe9f59e700 1 --2- 172.21.15.111:0/3847697349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe8c004d90 0x7fbe8c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:59.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.902+0000 7fbe9ed9d700 1 -- 172.21.15.111:0/3847697349 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbea01589d0 msgr2=0x7fbe8c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:59.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.902+0000 7fbe9ed9d700 1 --2- 172.21.15.111:0/3847697349 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbea01589d0 0x7fbe8c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.902+0000 7fbe9ed9d700 1 -- 172.21.15.111:0/3847697349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe8c004d90 msgr2=0x7fbe8c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:58:59.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.902+0000 7fbe9ed9d700 1 --2- 172.21.15.111:0/3847697349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe8c004d90 0x7fbe8c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.902+0000 7fbe9ed9d700 1 -- 172.21.15.111:0/3847697349 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c0035b0 con 0x7fbe8c003a00 2023-03-29T10:58:59.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.902+0000 7fbe9ed9d700 1 --2- 172.21.15.111:0/3847697349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe8c003a00 0x7fbe8c003e10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbe900096f0 tx=0x7fbe8c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02bc314317d57dc server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:59.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.902+0000 7fbe9e59c700 1 -- 172.21.15.111:0/3847697349 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9000e660 con 0x7fbe8c003a00 2023-03-29T10:58:59.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.902+0000 7fbe9e59c700 1 -- 172.21.15.111:0/3847697349 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9000e7c0 con 0x7fbe8c003a00 2023-03-29T10:58:59.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.902+0000 7fbe9e59c700 1 -- 172.21.15.111:0/3847697349 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9000eb50 con 0x7fbe8c003a00 2023-03-29T10:58:59.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.902+0000 7fbea51ff700 1 -- 172.21.15.111:0/3847697349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe8c003a00 msgr2=0x7fbe8c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:59.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.902+0000 7fbea51ff700 1 --2- 172.21.15.111:0/3847697349 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe8c003a00 0x7fbe8c003e10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbe900096f0 tx=0x7fbe8c005880 comp rx=0 tx=0).stop 2023-03-29T10:58:59.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.902+0000 7fbe9f59e700 1 -- 172.21.15.111:0/3847697349 reap_dead start 2023-03-29T10:58:59.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.903+0000 7fbea51ff700 1 -- 172.21.15.111:0/3847697349 shutdown_connections 2023-03-29T10:58:59.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.903+0000 7fbea51ff700 1 -- 172.21.15.111:0/3847697349 >> 172.21.15.111:0/3847697349 conn(0x7fbea00cf720 msgr2=0x7fbea00cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:58:59.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.903+0000 7fbea51ff700 1 -- 172.21.15.111:0/3847697349 shutdown_connections 2023-03-29T10:58:59.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.903+0000 7fbea51ff700 1 -- 172.21.15.111:0/3847697349 wait complete. 2023-03-29T10:58:59.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.904+0000 7fbea51ff700 1 Processor -- start 2023-03-29T10:58:59.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.905+0000 7fbea51ff700 1 -- start start 2023-03-29T10:58:59.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.905+0000 7fbea51ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe8c003a00 0x7fbea00c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:59.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.905+0000 7fbea51ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbea01f4e00 0x7fbea01f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:59.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.905+0000 7fbea51ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbea01f5270 0x7fbea0159840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:59.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.905+0000 7fbea51ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea0157dd0 con 0x7fbe8c003a00 2023-03-29T10:58:59.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.905+0000 7fbea51ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea00c5fc0 con 0x7fbea01f4e00 2023-03-29T10:58:59.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.905+0000 7fbea51ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea01f6140 con 0x7fbea01f5270 2023-03-29T10:58:59.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.906+0000 7fbe9ed9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe8c003a00 0x7fbea00c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:59.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.906+0000 7fbe9ed9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe8c003a00 0x7fbea00c6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52032/0 (socket says 172.21.15.111:52032) 2023-03-29T10:58:59.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.906+0000 7fbe9ed9d700 1 -- 172.21.15.111:0/1789507249 learned_addr learned my addr 172.21.15.111:0/1789507249 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:58:59.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.906+0000 7fbe9f59e700 1 --2- 172.21.15.111:0/1789507249 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbea01f5270 0x7fbea0159840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:59.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.906+0000 7fbe97fff700 1 --2- 172.21.15.111:0/1789507249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbea01f4e00 0x7fbea01f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:59.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.906+0000 7fbe9ed9d700 1 -- 172.21.15.111:0/1789507249 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbea01f5270 msgr2=0x7fbea0159840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:59.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.906+0000 7fbe9ed9d700 1 --2- 172.21.15.111:0/1789507249 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbea01f5270 0x7fbea0159840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.906+0000 7fbe9ed9d700 1 -- 172.21.15.111:0/1789507249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbea01f4e00 msgr2=0x7fbea01f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:58:59.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.906+0000 7fbe9ed9d700 1 --2- 172.21.15.111:0/1789507249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbea01f4e00 0x7fbea01f56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:58:59.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.906+0000 7fbe9ed9d700 1 -- 172.21.15.111:0/1789507249 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea006aaf0 con 0x7fbe8c003a00 2023-03-29T10:58:59.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.906+0000 7fbe97fff700 1 --2- 172.21.15.111:0/1789507249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbea01f4e00 0x7fbea01f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:58:59.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.906+0000 7fbe9ed9d700 1 --2- 172.21.15.111:0/1789507249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe8c003a00 0x7fbea00c6450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000e4b0 tx=0x7fbe900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:59.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.907+0000 7fbe977fe700 1 -- 172.21.15.111:0/1789507249 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9000e660 con 0x7fbe8c003a00 2023-03-29T10:58:59.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.907+0000 7fbe977fe700 1 -- 172.21.15.111:0/1789507249 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe900038c0 con 0x7fbe8c003a00 2023-03-29T10:58:59.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.907+0000 7fbe977fe700 1 -- 172.21.15.111:0/1789507249 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90006e60 con 0x7fbe8c003a00 2023-03-29T10:58:59.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.907+0000 7fbea51ff700 1 -- 172.21.15.111:0/1789507249 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea00ccc30 con 0x7fbe8c003a00 2023-03-29T10:58:59.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.907+0000 7fbea51ff700 1 -- 172.21.15.111:0/1789507249 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea006ac80 con 0x7fbe8c003a00 2023-03-29T10:58:59.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.908+0000 7fbea51ff700 1 -- 172.21.15.111:0/1789507249 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea00b4a10 con 0x7fbe8c003a00 2023-03-29T10:58:59.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.909+0000 7fbe977fe700 1 -- 172.21.15.111:0/1789507249 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fbe90003c70 con 0x7fbe8c003a00 2023-03-29T10:58:59.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.910+0000 7fbe977fe700 1 --2- 172.21.15.111:0/1789507249 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbe80069de0 0x7fbe8006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:58:59.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.910+0000 7fbe977fe700 1 -- 172.21.15.111:0/1789507249 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6957+0+0 (secure 0 0 0) 0x7fbe90088e50 con 0x7fbe8c003a00 2023-03-29T10:58:59.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.910+0000 7fbe97fff700 1 --2- 172.21.15.111:0/1789507249 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbe80069de0 0x7fbe8006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:58:59.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.911+0000 7fbe97fff700 1 --2- 172.21.15.111:0/1789507249 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbe80069de0 0x7fbe8006c290 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009be0 tx=0x7fbe88008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:58:59.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:58:59.914+0000 7fbe977fe700 1 -- 172.21.15.111:0/1789507249 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fbe90054370 con 0x7fbe8c003a00 2023-03-29T10:59:00.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.150+0000 7fbea51ff700 1 -- 172.21.15.111:0/1789507249 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbea01f6600 con 0x7fbe8c003a00 2023-03-29T10:59:00.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.691+0000 7fbe977fe700 1 -- 172.21.15.111:0/1789507249 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbe90057990 con 0x7fbe8c003a00 2023-03-29T10:59:00.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.694+0000 7fbea51ff700 1 -- 172.21.15.111:0/1789507249 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbe80069de0 msgr2=0x7fbe8006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:00.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.694+0000 7fbea51ff700 1 --2- 172.21.15.111:0/1789507249 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbe80069de0 0x7fbe8006c290 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009be0 tx=0x7fbe88008040 comp rx=0 tx=0).stop 2023-03-29T10:59:00.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.694+0000 7fbea51ff700 1 -- 172.21.15.111:0/1789507249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe8c003a00 msgr2=0x7fbea00c6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:00.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.694+0000 7fbea51ff700 1 --2- 172.21.15.111:0/1789507249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe8c003a00 0x7fbea00c6450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000e4b0 tx=0x7fbe900047c0 comp rx=0 tx=0).stop 2023-03-29T10:59:00.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.694+0000 7fbea51ff700 1 -- 172.21.15.111:0/1789507249 shutdown_connections 2023-03-29T10:59:00.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.694+0000 7fbea51ff700 1 --2- 172.21.15.111:0/1789507249 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbe80069de0 0x7fbe8006c290 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:00.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.694+0000 7fbea51ff700 1 --2- 172.21.15.111:0/1789507249 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbe8c003a00 0x7fbea00c6450 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:00.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.694+0000 7fbea51ff700 1 --2- 172.21.15.111:0/1789507249 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbea01f5270 0x7fbea0159840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:00.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.694+0000 7fbea51ff700 1 --2- 172.21.15.111:0/1789507249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbea01f4e00 0x7fbea01f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:00.694 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.694+0000 7fbea51ff700 1 -- 172.21.15.111:0/1789507249 >> 172.21.15.111:0/1789507249 conn(0x7fbea00cf720 msgr2=0x7fbea015b330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:00.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.694+0000 7fbea51ff700 1 -- 172.21.15.111:0/1789507249 shutdown_connections 2023-03-29T10:59:00.695 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.695+0000 7fbea51ff700 1 -- 172.21.15.111:0/1789507249 wait complete. 2023-03-29T10:59:00.712 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-03-29T10:59:00.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.900+0000 7f469b1ff700 1 Processor -- start 2023-03-29T10:59:00.900 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.900+0000 7f469b1ff700 1 -- start start 2023-03-29T10:59:00.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.901+0000 7f469b1ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f46940cfbf0 0x7f46941608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:00.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.901+0000 7f469b1ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f46940d1100 0x7f46940d1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:00.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.901+0000 7f469b1ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46940d0020 con 0x7f46940b87a0 2023-03-29T10:59:00.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.901+0000 7f469b1ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46940d02d0 con 0x7f46940d1100 2023-03-29T10:59:00.901 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.901+0000 7f469b1ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46940d0cb0 con 0x7f46940cfbf0 2023-03-29T10:59:00.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.901+0000 7f469979c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f46940d1100 0x7f46940d1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:00.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.901+0000 7f469979c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f46940d1100 0x7f46940d1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52036/0 (socket says 172.21.15.111:52036) 2023-03-29T10:59:00.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.901+0000 7f4698f9b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f46940b87a0 0x7f46941601e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:40996/0 (socket says 172.21.15.111:40996) 2023-03-29T10:59:00.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.901+0000 7f469979c700 1 -- 172.21.15.111:0/3563547150 learned_addr learned my addr 172.21.15.111:0/3563547150 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:00.902 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.902+0000 7f469979c700 1 -- 172.21.15.111:0/3563547150 >> v2:172.21.15.119:3300/0 conn(0x7f46940cfbf0 msgr2=0x7f46941608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:00.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.902+0000 7f4693fff700 1 --2- 172.21.15.111:0/3563547150 >> v2:172.21.15.119:3300/0 conn(0x7f46940cfbf0 0x7f46941608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:00.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.902+0000 7f469979c700 1 --2- 172.21.15.111:0/3563547150 >> v2:172.21.15.119:3300/0 conn(0x7f46940cfbf0 0x7f46941608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:00.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.902+0000 7f46937fe700 1 -- 172.21.15.111:0/3563547150 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070763456 0 0) 0x7f46940d0020 con 0x7f46940b87a0 2023-03-29T10:59:00.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.902+0000 7f469979c700 1 -- 172.21.15.111:0/3563547150 >> v1:172.21.15.111:6789/0 conn(0x7f46940b87a0 legacy=0x7f46941601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:00.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.902+0000 7f469979c700 1 -- 172.21.15.111:0/3563547150 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46940cdc60 con 0x7f46940d1100 2023-03-29T10:59:00.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.902+0000 7f4693fff700 1 --2- 172.21.15.111:0/3563547150 >> v2:172.21.15.119:3300/0 conn(0x7f46940cfbf0 0x7f46941608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:59:00.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.902+0000 7f469979c700 1 --2- 172.21.15.111:0/3563547150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46940d1100 0x7f46940d1520 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f46940b8f60 tx=0x7f468c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a7c0878205c94b server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:00.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.902+0000 7f46937fe700 1 -- 172.21.15.111:0/3563547150 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c00e070 con 0x7f46940d1100 2023-03-29T10:59:00.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.902+0000 7f46937fe700 1 -- 172.21.15.111:0/3563547150 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468c00b980 con 0x7f46940d1100 2023-03-29T10:59:00.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.902+0000 7f46937fe700 1 -- 172.21.15.111:0/3563547150 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c00bd30 con 0x7f46940d1100 2023-03-29T10:59:00.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.903+0000 7f469b1ff700 1 -- 172.21.15.111:0/3563547150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46940d1100 msgr2=0x7f46940d1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:00.903 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.903+0000 7f469b1ff700 1 --2- 172.21.15.111:0/3563547150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46940d1100 0x7f46940d1520 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f46940b8f60 tx=0x7f468c00b330 comp rx=0 tx=0).stop 2023-03-29T10:59:00.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.903+0000 7f469b1ff700 1 -- 172.21.15.111:0/3563547150 shutdown_connections 2023-03-29T10:59:00.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.903+0000 7f469b1ff700 1 --2- 172.21.15.111:0/3563547150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46940d1100 0x7f46940d1520 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:00.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.903+0000 7f469b1ff700 1 --2- 172.21.15.111:0/3563547150 >> v2:172.21.15.119:3300/0 conn(0x7f46940cfbf0 0x7f46941608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:00.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.903+0000 7f469b1ff700 1 -- 172.21.15.111:0/3563547150 >> 172.21.15.111:0/3563547150 conn(0x7f4694071b10 msgr2=0x7f4694071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:00.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.903+0000 7f469b1ff700 1 -- 172.21.15.111:0/3563547150 shutdown_connections 2023-03-29T10:59:00.904 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.903+0000 7f469b1ff700 1 -- 172.21.15.111:0/3563547150 wait complete. 2023-03-29T10:59:00.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.905+0000 7f469b1ff700 1 Processor -- start 2023-03-29T10:59:00.905 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.905+0000 7f469b1ff700 1 -- start start 2023-03-29T10:59:00.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.906+0000 7f469b1ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46940b87a0 0x7f46940c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:00.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.906+0000 7f469b1ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46940cfbf0 0x7f469415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:00.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.906+0000 7f469b1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46940d1100 0x7f469415b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:00.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.906+0000 7f469b1ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469415eca0 con 0x7f46940b87a0 2023-03-29T10:59:00.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.906+0000 7f469b1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469415ef80 con 0x7f46940d1100 2023-03-29T10:59:00.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.906+0000 7f469b1ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469415f260 con 0x7f46940cfbf0 2023-03-29T10:59:00.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.906+0000 7f4698f9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46940b87a0 0x7f46940c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:00.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.906+0000 7f4698f9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46940b87a0 0x7f46940c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52038/0 (socket says 172.21.15.111:52038) 2023-03-29T10:59:00.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.906+0000 7f4698f9b700 1 -- 172.21.15.111:0/1979403880 learned_addr learned my addr 172.21.15.111:0/1979403880 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:00.906 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.906+0000 7f4693fff700 1 --2- 172.21.15.111:0/1979403880 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46940cfbf0 0x7f469415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:00.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.906+0000 7f469979c700 1 --2- 172.21.15.111:0/1979403880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46940d1100 0x7f469415b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:00.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.906+0000 7f4698f9b700 1 -- 172.21.15.111:0/1979403880 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46940cfbf0 msgr2=0x7f469415aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:00.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.906+0000 7f4698f9b700 1 --2- 172.21.15.111:0/1979403880 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46940cfbf0 0x7f469415aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:00.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.906+0000 7f4698f9b700 1 -- 172.21.15.111:0/1979403880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46940d1100 msgr2=0x7f469415b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:00.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.906+0000 7f4698f9b700 1 --2- 172.21.15.111:0/1979403880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46940d1100 0x7f469415b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:00.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.906+0000 7f4698f9b700 1 -- 172.21.15.111:0/1979403880 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c009b30 con 0x7f46940b87a0 2023-03-29T10:59:00.907 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.906+0000 7f469979c700 1 --2- 172.21.15.111:0/1979403880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46940d1100 0x7f469415b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:59:00.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.907+0000 7f4698f9b700 1 --2- 172.21.15.111:0/1979403880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46940b87a0 0x7f46940c8e00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f468800c550 tx=0x7f468800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:00.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.907+0000 7f46917fa700 1 -- 172.21.15.111:0/1979403880 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688004c80 con 0x7f46940b87a0 2023-03-29T10:59:00.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.907+0000 7f46917fa700 1 -- 172.21.15.111:0/1979403880 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4688004e00 con 0x7f46940b87a0 2023-03-29T10:59:00.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.907+0000 7f469b1ff700 1 -- 172.21.15.111:0/1979403880 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46940cd980 con 0x7f46940b87a0 2023-03-29T10:59:00.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.907+0000 7f46917fa700 1 -- 172.21.15.111:0/1979403880 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688007310 con 0x7f46940b87a0 2023-03-29T10:59:00.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.907+0000 7f469b1ff700 1 -- 172.21.15.111:0/1979403880 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46940cd6a0 con 0x7f46940b87a0 2023-03-29T10:59:00.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.908+0000 7f469b1ff700 1 -- 172.21.15.111:0/1979403880 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4694160e30 con 0x7f46940b87a0 2023-03-29T10:59:00.908 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.909+0000 7f46917fa700 1 -- 172.21.15.111:0/1979403880 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f4688007470 con 0x7f46940b87a0 2023-03-29T10:59:00.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.910+0000 7f46917fa700 1 --2- 172.21.15.111:0/1979403880 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4678069de0 0x7f467806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:00.909 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.910+0000 7f46917fa700 1 -- 172.21.15.111:0/1979403880 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f4688087f70 con 0x7f46940b87a0 2023-03-29T10:59:00.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.910+0000 7f4693fff700 1 --2- 172.21.15.111:0/1979403880 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4678069de0 0x7f467806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:00.910 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.911+0000 7f4693fff700 1 --2- 172.21.15.111:0/1979403880 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4678069de0 0x7f467806c290 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f46940b77f0 tx=0x7f4680008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:00.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:00.914+0000 7f46917fa700 1 -- 172.21.15.111:0/1979403880 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f4688053490 con 0x7f46940b87a0 2023-03-29T10:59:01.106 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.107+0000 7f469b1ff700 1 -- 172.21.15.111:0/1979403880 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4694074430 con 0x7f46940b87a0 2023-03-29T10:59:01.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.767+0000 7f46917fa700 1 -- 172.21.15.111:0/1979403880 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v32) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4688056ab0 con 0x7f46940b87a0 2023-03-29T10:59:01.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.770+0000 7f469b1ff700 1 -- 172.21.15.111:0/1979403880 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4678069de0 msgr2=0x7f467806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:01.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.770+0000 7f469b1ff700 1 --2- 172.21.15.111:0/1979403880 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4678069de0 0x7f467806c290 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f46940b77f0 tx=0x7f4680008040 comp rx=0 tx=0).stop 2023-03-29T10:59:01.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.770+0000 7f469b1ff700 1 -- 172.21.15.111:0/1979403880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46940b87a0 msgr2=0x7f46940c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:01.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.770+0000 7f469b1ff700 1 --2- 172.21.15.111:0/1979403880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46940b87a0 0x7f46940c8e00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f468800c550 tx=0x7f468800c860 comp rx=0 tx=0).stop 2023-03-29T10:59:01.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.771+0000 7f469b1ff700 1 -- 172.21.15.111:0/1979403880 shutdown_connections 2023-03-29T10:59:01.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.771+0000 7f469b1ff700 1 --2- 172.21.15.111:0/1979403880 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4678069de0 0x7f467806c290 secure :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f46940b77f0 tx=0x7f4680008040 comp rx=0 tx=0).stop 2023-03-29T10:59:01.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.771+0000 7f469b1ff700 1 --2- 172.21.15.111:0/1979403880 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f46940b87a0 0x7f46940c8e00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:01.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.771+0000 7f469b1ff700 1 --2- 172.21.15.111:0/1979403880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f46940d1100 0x7f469415b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:01.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.771+0000 7f469b1ff700 1 --2- 172.21.15.111:0/1979403880 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f46940cfbf0 0x7f469415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:01.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.771+0000 7f469b1ff700 1 -- 172.21.15.111:0/1979403880 >> 172.21.15.111:0/1979403880 conn(0x7f4694071b10 msgr2=0x7f46940d36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:01.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.771+0000 7f469b1ff700 1 -- 172.21.15.111:0/1979403880 shutdown_connections 2023-03-29T10:59:01.771 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.771+0000 7f469b1ff700 1 -- 172.21.15.111:0/1979403880 wait complete. 2023-03-29T10:59:01.772 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_metadata' removed 2023-03-29T10:59:01.791 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-03-29T10:59:01.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.980+0000 7fdc9afff700 1 Processor -- start 2023-03-29T10:59:01.980 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.981+0000 7fdc9afff700 1 -- start start 2023-03-29T10:59:01.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.982+0000 7fdc9afff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc940cf010 0x7fdc94160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:01.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.982+0000 7fdc9afff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdc940cff50 0x7fdc940d2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:01.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.982+0000 7fdc9afff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc940cd520 con 0x7fdc9415c590 2023-03-29T10:59:01.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.982+0000 7fdc9afff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc940d2640 con 0x7fdc940cff50 2023-03-29T10:59:01.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.982+0000 7fdc9afff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc941610d0 con 0x7fdc940cf010 2023-03-29T10:59:01.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.982+0000 7fdc9959c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdc940cff50 0x7fdc940d2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:01.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.982+0000 7fdc9959c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fdc940cff50 0x7fdc940d2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52044/0 (socket says 172.21.15.111:52044) 2023-03-29T10:59:01.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.982+0000 7fdc9959c700 1 -- 172.21.15.111:0/4066755212 learned_addr learned my addr 172.21.15.111:0/4066755212 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:01.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.982+0000 7fdc93fff700 1 --2- 172.21.15.111:0/4066755212 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc940cf010 0x7fdc94160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:01.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.982+0000 7fdc9959c700 1 -- 172.21.15.111:0/4066755212 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc940cf010 msgr2=0x7fdc94160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:01.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.982+0000 7fdc9959c700 1 --2- 172.21.15.111:0/4066755212 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc940cf010 0x7fdc94160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:01.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.982+0000 7fdc9959c700 1 -- 172.21.15.111:0/4066755212 >> v1:172.21.15.119:6789/0 conn(0x7fdc9415c590 legacy=0x7fdc940cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:01.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.982+0000 7fdc9959c700 1 -- 172.21.15.111:0/4066755212 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc940b23c0 con 0x7fdc940cff50 2023-03-29T10:59:01.983 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.983+0000 7fdc9959c700 1 --2- 172.21.15.111:0/4066755212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc940cff50 0x7fdc940d2ad0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009e50 tx=0x7fdc8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f98dc00dac68a8a server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:01.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.983+0000 7fdc937fe700 1 -- 172.21.15.111:0/4066755212 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c00e070 con 0x7fdc940cff50 2023-03-29T10:59:01.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.983+0000 7fdc937fe700 1 -- 172.21.15.111:0/4066755212 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c00b9c0 con 0x7fdc940cff50 2023-03-29T10:59:01.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.983+0000 7fdc937fe700 1 -- 172.21.15.111:0/4066755212 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c00bd70 con 0x7fdc940cff50 2023-03-29T10:59:01.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.983+0000 7fdc9afff700 1 -- 172.21.15.111:0/4066755212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc940cff50 msgr2=0x7fdc940d2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:01.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.983+0000 7fdc9afff700 1 --2- 172.21.15.111:0/4066755212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc940cff50 0x7fdc940d2ad0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009e50 tx=0x7fdc8c00b330 comp rx=0 tx=0).stop 2023-03-29T10:59:01.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.984+0000 7fdc9afff700 1 -- 172.21.15.111:0/4066755212 shutdown_connections 2023-03-29T10:59:01.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.984+0000 7fdc9afff700 1 --2- 172.21.15.111:0/4066755212 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc940cff50 0x7fdc940d2ad0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:01.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.984+0000 7fdc9afff700 1 --2- 172.21.15.111:0/4066755212 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc940cf010 0x7fdc94160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:01.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.984+0000 7fdc9afff700 1 -- 172.21.15.111:0/4066755212 >> 172.21.15.111:0/4066755212 conn(0x7fdc94071b10 msgr2=0x7fdc94071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:01.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.984+0000 7fdc9afff700 1 -- 172.21.15.111:0/4066755212 shutdown_connections 2023-03-29T10:59:01.984 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.984+0000 7fdc9afff700 1 -- 172.21.15.111:0/4066755212 wait complete. 2023-03-29T10:59:01.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.985+0000 7fdc9afff700 1 Processor -- start 2023-03-29T10:59:01.985 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.985+0000 7fdc9afff700 1 -- start start 2023-03-29T10:59:01.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.986+0000 7fdc9afff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc940cf010 0x7fdc941f5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:01.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.986+0000 7fdc9afff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc940cff50 0x7fdc940b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:01.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.986+0000 7fdc9afff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc9415c590 0x7fdc940b7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:01.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.986+0000 7fdc9afff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc941f64d0 con 0x7fdc9415c590 2023-03-29T10:59:01.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.986+0000 7fdc9afff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9415ee30 con 0x7fdc940cf010 2023-03-29T10:59:01.986 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.986+0000 7fdc9afff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9415f110 con 0x7fdc940cff50 2023-03-29T10:59:01.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.986+0000 7fdc9959c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc9415c590 0x7fdc940b7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:01.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.987+0000 7fdc9959c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc9415c590 0x7fdc940b7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52050/0 (socket says 172.21.15.111:52050) 2023-03-29T10:59:01.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.987+0000 7fdc9959c700 1 -- 172.21.15.111:0/1970339506 learned_addr learned my addr 172.21.15.111:0/1970339506 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:01.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.987+0000 7fdc98d9b700 1 --2- 172.21.15.111:0/1970339506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc940cf010 0x7fdc941f5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:01.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.987+0000 7fdc93fff700 1 --2- 172.21.15.111:0/1970339506 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc940cff50 0x7fdc940b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:01.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.987+0000 7fdc9959c700 1 -- 172.21.15.111:0/1970339506 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc940cff50 msgr2=0x7fdc940b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:01.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.987+0000 7fdc9959c700 1 --2- 172.21.15.111:0/1970339506 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc940cff50 0x7fdc940b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:01.987 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.987+0000 7fdc9959c700 1 -- 172.21.15.111:0/1970339506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc940cf010 msgr2=0x7fdc941f5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:01.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.987+0000 7fdc9959c700 1 --2- 172.21.15.111:0/1970339506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc940cf010 0x7fdc941f5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:01.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.987+0000 7fdc9959c700 1 -- 172.21.15.111:0/1970339506 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c009b30 con 0x7fdc9415c590 2023-03-29T10:59:01.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.987+0000 7fdc93fff700 1 --2- 172.21.15.111:0/1970339506 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc940cff50 0x7fdc940b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:59:01.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.987+0000 7fdc98d9b700 1 --2- 172.21.15.111:0/1970339506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc940cf010 0x7fdc941f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:59:01.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.987+0000 7fdc9959c700 1 --2- 172.21.15.111:0/1970339506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc9415c590 0x7fdc940b7c60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdc94072710 tx=0x7fdc8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:01.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.987+0000 7fdc917fa700 1 -- 172.21.15.111:0/1970339506 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c004920 con 0x7fdc9415c590 2023-03-29T10:59:01.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.987+0000 7fdc917fa700 1 -- 172.21.15.111:0/1970339506 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8c0099b0 con 0x7fdc9415c590 2023-03-29T10:59:01.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.987+0000 7fdc917fa700 1 -- 172.21.15.111:0/1970339506 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c01c680 con 0x7fdc9415c590 2023-03-29T10:59:01.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.987+0000 7fdc9afff700 1 -- 172.21.15.111:0/1970339506 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc940cd770 con 0x7fdc9415c590 2023-03-29T10:59:01.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.988+0000 7fdc9afff700 1 -- 172.21.15.111:0/1970339506 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc940cf560 con 0x7fdc9415c590 2023-03-29T10:59:01.988 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.989+0000 7fdc9afff700 1 -- 172.21.15.111:0/1970339506 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9415c960 con 0x7fdc9415c590 2023-03-29T10:59:01.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.989+0000 7fdc917fa700 1 -- 172.21.15.111:0/1970339506 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fdc8c01c7e0 con 0x7fdc9415c590 2023-03-29T10:59:01.990 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.991+0000 7fdc917fa700 1 --2- 172.21.15.111:0/1970339506 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdc78069de0 0x7fdc7806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:01.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.991+0000 7fdc917fa700 1 -- 172.21.15.111:0/1970339506 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6519+0+0 (secure 0 0 0) 0x7fdc8c089fc0 con 0x7fdc9415c590 2023-03-29T10:59:01.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.991+0000 7fdc98d9b700 1 --2- 172.21.15.111:0/1970339506 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdc78069de0 0x7fdc7806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:01.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.992+0000 7fdc98d9b700 1 --2- 172.21.15.111:0/1970339506 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdc78069de0 0x7fdc7806c290 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdc88000f60 tx=0x7fdc88009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:01.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:01.996+0000 7fdc917fa700 1 -- 172.21.15.111:0/1970339506 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fdc8c055610 con 0x7fdc9415c590 2023-03-29T10:59:02.200 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.200+0000 7fdc9afff700 1 -- 172.21.15.111:0/1970339506 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdc94074430 con 0x7fdc9415c590 2023-03-29T10:59:02.774 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.770+0000 7fdc917fa700 1 -- 172.21.15.111:0/1970339506 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v33) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdc8c058c30 con 0x7fdc9415c590 2023-03-29T10:59:02.774 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.775+0000 7fdc9afff700 1 -- 172.21.15.111:0/1970339506 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdc78069de0 msgr2=0x7fdc7806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:02.774 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.775+0000 7fdc9afff700 1 --2- 172.21.15.111:0/1970339506 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdc78069de0 0x7fdc7806c290 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdc88000f60 tx=0x7fdc88009040 comp rx=0 tx=0).stop 2023-03-29T10:59:02.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.775+0000 7fdc9afff700 1 -- 172.21.15.111:0/1970339506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc9415c590 msgr2=0x7fdc940b7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:02.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.775+0000 7fdc9afff700 1 --2- 172.21.15.111:0/1970339506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc9415c590 0x7fdc940b7c60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdc94072710 tx=0x7fdc8c004ad0 comp rx=0 tx=0).stop 2023-03-29T10:59:02.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.775+0000 7fdc9afff700 1 -- 172.21.15.111:0/1970339506 shutdown_connections 2023-03-29T10:59:02.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.775+0000 7fdc9afff700 1 --2- 172.21.15.111:0/1970339506 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fdc78069de0 0x7fdc7806c290 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:02.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.775+0000 7fdc9afff700 1 --2- 172.21.15.111:0/1970339506 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fdc9415c590 0x7fdc940b7c60 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:02.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.775+0000 7fdc9afff700 1 --2- 172.21.15.111:0/1970339506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc940cf010 0x7fdc941f5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:02.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.775+0000 7fdc9afff700 1 --2- 172.21.15.111:0/1970339506 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fdc940cff50 0x7fdc940b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:02.775 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.775+0000 7fdc9afff700 1 -- 172.21.15.111:0/1970339506 >> 172.21.15.111:0/1970339506 conn(0x7fdc94071b10 msgr2=0x7fdc940b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:02.776 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.776+0000 7fdc9afff700 1 -- 172.21.15.111:0/1970339506 shutdown_connections 2023-03-29T10:59:02.777 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.777+0000 7fdc9afff700 1 -- 172.21.15.111:0/1970339506 wait complete. 2023-03-29T10:59:02.780 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data' removed 2023-03-29T10:59:02.799 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-03-29T10:59:02.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.989+0000 7fb51adff700 1 Processor -- start 2023-03-29T10:59:02.989 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.989+0000 7fb51adff700 1 -- start start 2023-03-29T10:59:02.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.990+0000 7fb51adff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5140cfbf0 0x7fb5141608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:02.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.990+0000 7fb51adff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb5140d1100 0x7fb5140d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:02.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.990+0000 7fb51adff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5140d0020 con 0x7fb5140b87a0 2023-03-29T10:59:02.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.990+0000 7fb51adff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5140d02d0 con 0x7fb5140d1100 2023-03-29T10:59:02.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.991+0000 7fb51adff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5140d0cb0 con 0x7fb5140cfbf0 2023-03-29T10:59:02.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.991+0000 7fb51939c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb5140d1100 0x7fb5140d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:02.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.991+0000 7fb51939c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fb5140d1100 0x7fb5140d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52066/0 (socket says 172.21.15.111:52066) 2023-03-29T10:59:02.991 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.991+0000 7fb51939c700 1 -- 172.21.15.111:0/575849617 learned_addr learned my addr 172.21.15.111:0/575849617 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:02.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.991+0000 7fb518b9b700 1 --1- 172.21.15.111:0/575849617 >> v1:172.21.15.111:6789/0 conn(0x7fb5140b87a0 0x7fb5141601e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41004/0 (socket says 172.21.15.111:41004) 2023-03-29T10:59:02.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.991+0000 7fb51939c700 1 -- 172.21.15.111:0/575849617 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5140cfbf0 msgr2=0x7fb5141608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:02.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.991+0000 7fb513fff700 1 --2- 172.21.15.111:0/575849617 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5140cfbf0 0x7fb5141608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:02.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.991+0000 7fb51939c700 1 --2- 172.21.15.111:0/575849617 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5140cfbf0 0x7fb5141608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:02.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.991+0000 7fb51939c700 1 -- 172.21.15.111:0/575849617 >> v1:172.21.15.111:6789/0 conn(0x7fb5140b87a0 legacy=0x7fb5141601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:02.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.991+0000 7fb5137fe700 1 -- 172.21.15.111:0/575849617 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4275740774 0 0) 0x7fb5140d0020 con 0x7fb5140b87a0 2023-03-29T10:59:02.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.991+0000 7fb51939c700 1 -- 172.21.15.111:0/575849617 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5140cdc60 con 0x7fb5140d1100 2023-03-29T10:59:02.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.991+0000 7fb513fff700 1 --2- 172.21.15.111:0/575849617 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5140cfbf0 0x7fb5141608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:59:02.992 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.992+0000 7fb51939c700 1 --2- 172.21.15.111:0/575849617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5140d1100 0x7fb5140d1580 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009e50 tx=0x7fb50c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad8a54c3f365b37 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:02.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.992+0000 7fb5137fe700 1 -- 172.21.15.111:0/575849617 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c00e070 con 0x7fb5140d1100 2023-03-29T10:59:02.993 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.992+0000 7fb5137fe700 1 -- 172.21.15.111:0/575849617 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c00b9c0 con 0x7fb5140d1100 2023-03-29T10:59:02.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.992+0000 7fb5137fe700 1 -- 172.21.15.111:0/575849617 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c00bd70 con 0x7fb5140d1100 2023-03-29T10:59:02.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.992+0000 7fb51adff700 1 -- 172.21.15.111:0/575849617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5140d1100 msgr2=0x7fb5140d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:02.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.992+0000 7fb51adff700 1 --2- 172.21.15.111:0/575849617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5140d1100 0x7fb5140d1580 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb50c009e50 tx=0x7fb50c00b330 comp rx=0 tx=0).stop 2023-03-29T10:59:02.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.992+0000 7fb51adff700 1 -- 172.21.15.111:0/575849617 shutdown_connections 2023-03-29T10:59:02.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.992+0000 7fb51adff700 1 --2- 172.21.15.111:0/575849617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5140d1100 0x7fb5140d1580 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:02.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.992+0000 7fb51adff700 1 --2- 172.21.15.111:0/575849617 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5140cfbf0 0x7fb5141608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:02.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.992+0000 7fb51adff700 1 -- 172.21.15.111:0/575849617 >> 172.21.15.111:0/575849617 conn(0x7fb514071b10 msgr2=0x7fb514071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:02.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.992+0000 7fb51adff700 1 -- 172.21.15.111:0/575849617 shutdown_connections 2023-03-29T10:59:02.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.992+0000 7fb51adff700 1 -- 172.21.15.111:0/575849617 wait complete. 2023-03-29T10:59:02.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.994+0000 7fb51adff700 1 Processor -- start 2023-03-29T10:59:02.994 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.994+0000 7fb51adff700 1 -- start start 2023-03-29T10:59:02.995 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.995+0000 7fb51adff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5140b87a0 0x7fb514150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:02.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.995+0000 7fb51adff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5140cfbf0 0x7fb51415d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:02.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.995+0000 7fb51adff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5140d1100 0x7fb5140d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:02.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.995+0000 7fb51adff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb514150f80 con 0x7fb5140d1100 2023-03-29T10:59:02.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.995+0000 7fb51adff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51415dfb0 con 0x7fb5140b87a0 2023-03-29T10:59:02.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.995+0000 7fb51adff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb51415e290 con 0x7fb5140cfbf0 2023-03-29T10:59:02.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.995+0000 7fb51939c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5140d1100 0x7fb5140d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:02.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.995+0000 7fb51939c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5140d1100 0x7fb5140d2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52078/0 (socket says 172.21.15.111:52078) 2023-03-29T10:59:02.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.995+0000 7fb51939c700 1 -- 172.21.15.111:0/1837257200 learned_addr learned my addr 172.21.15.111:0/1837257200 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:02.996 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.995+0000 7fb518b9b700 1 --2- 172.21.15.111:0/1837257200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5140b87a0 0x7fb514150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:02.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.995+0000 7fb513fff700 1 --2- 172.21.15.111:0/1837257200 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5140cfbf0 0x7fb51415d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:02.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.995+0000 7fb51939c700 1 -- 172.21.15.111:0/1837257200 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5140cfbf0 msgr2=0x7fb51415d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:02.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.995+0000 7fb51939c700 1 --2- 172.21.15.111:0/1837257200 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5140cfbf0 0x7fb51415d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:02.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.995+0000 7fb51939c700 1 -- 172.21.15.111:0/1837257200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5140b87a0 msgr2=0x7fb514150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:02.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.995+0000 7fb51939c700 1 --2- 172.21.15.111:0/1837257200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5140b87a0 0x7fb514150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:02.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.996+0000 7fb51939c700 1 -- 172.21.15.111:0/1837257200 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c009b30 con 0x7fb5140d1100 2023-03-29T10:59:02.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.996+0000 7fb518b9b700 1 --2- 172.21.15.111:0/1837257200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5140b87a0 0x7fb514150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:59:02.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.996+0000 7fb51939c700 1 --2- 172.21.15.111:0/1837257200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5140d1100 0x7fb5140d2ae0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00b730 tx=0x7fb50c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:02.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.996+0000 7fb5117fa700 1 -- 172.21.15.111:0/1837257200 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c003ea0 con 0x7fb5140d1100 2023-03-29T10:59:02.997 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.996+0000 7fb5117fa700 1 -- 172.21.15.111:0/1837257200 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c0099b0 con 0x7fb5140d1100 2023-03-29T10:59:02.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.996+0000 7fb51adff700 1 -- 172.21.15.111:0/1837257200 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5140cd980 con 0x7fb5140d1100 2023-03-29T10:59:02.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.996+0000 7fb5117fa700 1 -- 172.21.15.111:0/1837257200 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c01c680 con 0x7fb5140d1100 2023-03-29T10:59:02.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.996+0000 7fb51adff700 1 -- 172.21.15.111:0/1837257200 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5140cd6a0 con 0x7fb5140d1100 2023-03-29T10:59:02.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.997+0000 7fb51adff700 1 -- 172.21.15.111:0/1837257200 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb514160e30 con 0x7fb5140d1100 2023-03-29T10:59:02.998 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.998+0000 7fb5117fa700 1 -- 172.21.15.111:0/1837257200 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fb50c01c7e0 con 0x7fb5140d1100 2023-03-29T10:59:02.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:02.999+0000 7fb5117fa700 1 --2- 172.21.15.111:0/1837257200 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb4f8069de0 0x7fb4f806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:02.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:03.000+0000 7fb5117fa700 1 -- 172.21.15.111:0/1837257200 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6047+0+0 (secure 0 0 0) 0x7fb50c089aa0 con 0x7fb5140d1100 2023-03-29T10:59:02.999 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:03.000+0000 7fb518b9b700 1 --2- 172.21.15.111:0/1837257200 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb4f8069de0 0x7fb4f806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:03.000 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:03.000+0000 7fb518b9b700 1 --2- 172.21.15.111:0/1837257200 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb4f8069de0 0x7fb4f806c290 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb508000f60 tx=0x7fb508007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:03.003 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:03.004+0000 7fb5117fa700 1 -- 172.21.15.111:0/1837257200 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fb50c055350 con 0x7fb5140d1100 2023-03-29T10:59:03.197 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:03.198+0000 7fb51adff700 1 -- 172.21.15.111:0/1837257200 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb51406c9e0 con 0x7fb5140d1100 2023-03-29T10:59:03.787 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:03.787+0000 7fb5117fa700 1 -- 172.21.15.111:0/1837257200 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v34) v1 ==== 178+0+0 (secure 0 0 0) 0x7fb50c058970 con 0x7fb5140d1100 2023-03-29T10:59:03.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:03.791+0000 7fb51adff700 1 -- 172.21.15.111:0/1837257200 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb4f8069de0 msgr2=0x7fb4f806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:03.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:03.791+0000 7fb51adff700 1 --2- 172.21.15.111:0/1837257200 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb4f8069de0 0x7fb4f806c290 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb508000f60 tx=0x7fb508007d30 comp rx=0 tx=0).stop 2023-03-29T10:59:03.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:03.791+0000 7fb51adff700 1 -- 172.21.15.111:0/1837257200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5140d1100 msgr2=0x7fb5140d2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:03.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:03.791+0000 7fb51adff700 1 --2- 172.21.15.111:0/1837257200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5140d1100 0x7fb5140d2ae0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb50c00b730 tx=0x7fb50c004ad0 comp rx=0 tx=0).stop 2023-03-29T10:59:03.791 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:03.792+0000 7fb51adff700 1 -- 172.21.15.111:0/1837257200 shutdown_connections 2023-03-29T10:59:03.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:03.792+0000 7fb51adff700 1 --2- 172.21.15.111:0/1837257200 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fb4f8069de0 0x7fb4f806c290 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:03.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:03.792+0000 7fb51adff700 1 --2- 172.21.15.111:0/1837257200 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fb5140d1100 0x7fb5140d2ae0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:03.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:03.792+0000 7fb51adff700 1 --2- 172.21.15.111:0/1837257200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5140b87a0 0x7fb514150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:03.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:03.792+0000 7fb51adff700 1 --2- 172.21.15.111:0/1837257200 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fb5140cfbf0 0x7fb51415d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:03.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:03.792+0000 7fb51adff700 1 -- 172.21.15.111:0/1837257200 >> 172.21.15.111:0/1837257200 conn(0x7fb514071b10 msgr2=0x7fb51415c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:03.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:03.792+0000 7fb51adff700 1 -- 172.21.15.111:0/1837257200 shutdown_connections 2023-03-29T10:59:03.792 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:03.792+0000 7fb51adff700 1 -- 172.21.15.111:0/1837257200 wait complete. 2023-03-29T10:59:03.793 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data_ec' removed 2023-03-29T10:59:03.813 INFO:tasks.ceph.mds.a:Restarting daemon 2023-03-29T10:59:03.813 INFO:tasks.ceph.mds.a:Stopping old one... 2023-03-29T10:59:03.816 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-03-29T10:59:03.816 INFO:teuthology.orchestra.run:waiting for 300 2023-03-29T10:59:03.816 INFO:tasks.ceph.mds.c:Restarting daemon 2023-03-29T10:59:03.816 INFO:tasks.ceph.mds.c:Stopping old one... 2023-03-29T10:59:03.817 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-03-29T10:59:03.817 INFO:teuthology.orchestra.run:waiting for 300 2023-03-29T10:59:03.817 INFO:tasks.ceph.mds.b:Restarting daemon 2023-03-29T10:59:03.817 INFO:tasks.ceph.mds.b:Stopping old one... 2023-03-29T10:59:03.817 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-03-29T10:59:03.817 INFO:teuthology.orchestra.run:waiting for 300 2023-03-29T10:59:03.817 INFO:tasks.ceph.mds.d:Restarting daemon 2023-03-29T10:59:03.817 INFO:tasks.ceph.mds.d:Stopping old one... 2023-03-29T10:59:03.818 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-03-29T10:59:03.818 INFO:teuthology.orchestra.run:waiting for 300 2023-03-29T10:59:03.831 INFO:tasks.ceph.mds.a:Stopped 2023-03-29T10:59:03.831 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-03-29T10:59:03.832 INFO:tasks.ceph.mds.c:Stopped 2023-03-29T10:59:03.833 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-03-29T10:59:03.833 INFO:tasks.ceph.mds.b:Stopped 2023-03-29T10:59:03.833 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-03-29T10:59:03.834 INFO:tasks.ceph.mds.d:Stopped 2023-03-29T10:59:03.834 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-03-29T10:59:03.876 INFO:tasks.ceph.mds.a:Started 2023-03-29T10:59:03.876 DEBUG:teuthology.parallel:result is None 2023-03-29T10:59:03.878 INFO:tasks.ceph.mds.b:Started 2023-03-29T10:59:03.878 DEBUG:teuthology.parallel:result is None 2023-03-29T10:59:03.879 INFO:tasks.ceph.mds.c:Started 2023-03-29T10:59:03.879 DEBUG:teuthology.parallel:result is None 2023-03-29T10:59:03.880 INFO:tasks.ceph.mds.d:Started 2023-03-29T10:59:03.880 DEBUG:teuthology.parallel:result is None 2023-03-29T10:59:03.881 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-03-29T10:59:04.013 INFO:tasks.ceph.mds.a.smithi111.stdout:starting mds.a at 2023-03-29T10:59:04.014 INFO:tasks.ceph.mds.c.smithi111.stdout:starting mds.c at 2023-03-29T10:59:04.016 INFO:tasks.ceph.mds.d.smithi119.stdout:starting mds.d at 2023-03-29T10:59:04.016 INFO:tasks.ceph.mds.b.smithi119.stdout:starting mds.b at 2023-03-29T10:59:04.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.082+0000 7f14827ff700 1 Processor -- start 2023-03-29T10:59:04.082 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.082+0000 7f14827ff700 1 -- start start 2023-03-29T10:59:04.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.083+0000 7f14827ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f147c158da0 0x7f147c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:04.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.083+0000 7f14827ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f147c06cc00 0x7f147c06a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:04.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.083+0000 7f14827ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f147c0ca890 con 0x7f147c1589d0 2023-03-29T10:59:04.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.083+0000 7f14827ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f147c0cd860 con 0x7f147c06cc00 2023-03-29T10:59:04.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.083+0000 7f14827ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f147c0ca5b0 con 0x7f147c158da0 2023-03-29T10:59:04.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.083+0000 7f1480d9c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f147c06cc00 0x7f147c06a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:04.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.083+0000 7f147bfff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f147c1589d0 0x7f147c0cd150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41016/0 (socket says 172.21.15.111:41016) 2023-03-29T10:59:04.083 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.083+0000 7f147bfff700 1 -- 172.21.15.111:0/2706509828 learned_addr learned my addr 172.21.15.111:0/2706509828 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:59:04.084 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.083+0000 7f1473fff700 1 --2- 172.21.15.111:0/2706509828 >> v2:172.21.15.119:3300/0 conn(0x7f147c158da0 0x7f147c06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:04.084 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.084+0000 7f1480d9c700 1 -- 172.21.15.111:0/2706509828 >> v2:172.21.15.119:3300/0 conn(0x7f147c158da0 msgr2=0x7f147c06c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:04.084 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.084+0000 7f147b7fe700 1 -- 172.21.15.111:0/2706509828 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3184673176 0 0) 0x7f147c0ca890 con 0x7f147c1589d0 2023-03-29T10:59:04.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.084+0000 7f1480d9c700 1 --2- 172.21.15.111:0/2706509828 >> v2:172.21.15.119:3300/0 conn(0x7f147c158da0 0x7f147c06c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:04.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.084+0000 7f1480d9c700 1 -- 172.21.15.111:0/2706509828 >> v1:172.21.15.111:6789/0 conn(0x7f147c1589d0 legacy=0x7f147c0cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:04.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.084+0000 7f1480d9c700 1 -- 172.21.15.111:0/2706509828 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147c0cc670 con 0x7f147c06cc00 2023-03-29T10:59:04.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.084+0000 7f1480d9c700 1 --2- 172.21.15.111:0/2706509828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f147c06cc00 0x7f147c06a3e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f147c070550 tx=0x7f147400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d076ba3903c23f7 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:04.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.084+0000 7f147b7fe700 1 -- 172.21.15.111:0/2706509828 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147400bd20 con 0x7f147c06cc00 2023-03-29T10:59:04.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.084+0000 7f147b7fe700 1 -- 172.21.15.111:0/2706509828 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147400be80 con 0x7f147c06cc00 2023-03-29T10:59:04.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.084+0000 7f147b7fe700 1 -- 172.21.15.111:0/2706509828 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14740116a0 con 0x7f147c06cc00 2023-03-29T10:59:04.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.085+0000 7f14827ff700 1 -- 172.21.15.111:0/2706509828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f147c06cc00 msgr2=0x7f147c06a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:04.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.085+0000 7f14827ff700 1 --2- 172.21.15.111:0/2706509828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f147c06cc00 0x7f147c06a3e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f147c070550 tx=0x7f147400b330 comp rx=0 tx=0).stop 2023-03-29T10:59:04.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.085+0000 7f14827ff700 1 -- 172.21.15.111:0/2706509828 shutdown_connections 2023-03-29T10:59:04.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.085+0000 7f14827ff700 1 --2- 172.21.15.111:0/2706509828 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f147c06cc00 0x7f147c06a3e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:04.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.085+0000 7f14827ff700 1 --2- 172.21.15.111:0/2706509828 >> v2:172.21.15.119:3300/0 conn(0x7f147c158da0 0x7f147c06c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:04.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.085+0000 7f14827ff700 1 -- 172.21.15.111:0/2706509828 >> 172.21.15.111:0/2706509828 conn(0x7f147c0cf720 msgr2=0x7f147c0cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:04.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.085+0000 7f14827ff700 1 -- 172.21.15.111:0/2706509828 shutdown_connections 2023-03-29T10:59:04.085 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.085+0000 7f14827ff700 1 -- 172.21.15.111:0/2706509828 wait complete. 2023-03-29T10:59:04.086 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.086+0000 7f14827ff700 1 Processor -- start 2023-03-29T10:59:04.086 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.087+0000 7f14827ff700 1 -- start start 2023-03-29T10:59:04.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.087+0000 7f14827ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f147c06cc00 0x7f147c151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:04.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.088+0000 7f14827ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f147c1589d0 0x7f147c0c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:04.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.088+0000 7f14827ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f147c158da0 0x7f147c0aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:04.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.088+0000 7f14827ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147c15a200 con 0x7f147c158da0 2023-03-29T10:59:04.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.088+0000 7f14827ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147c0c83c0 con 0x7f147c1589d0 2023-03-29T10:59:04.087 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.088+0000 7f14827ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f147c0c86a0 con 0x7f147c06cc00 2023-03-29T10:59:04.088 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.088+0000 7f1480d9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f147c158da0 0x7f147c0aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:04.088 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.088+0000 7f1473fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f147c1589d0 0x7f147c0c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:04.088 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.088+0000 7f147bfff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f147c06cc00 0x7f147c151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:04.088 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.088+0000 7f1480d9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f147c158da0 0x7f147c0aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52112/0 (socket says 172.21.15.111:52112) 2023-03-29T10:59:04.088 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.088+0000 7f1480d9c700 1 -- 172.21.15.111:0/3264773888 learned_addr learned my addr 172.21.15.111:0/3264773888 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:04.088 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.088+0000 7f1480d9c700 1 -- 172.21.15.111:0/3264773888 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f147c06cc00 msgr2=0x7f147c151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:04.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.088+0000 7f1480d9c700 1 --2- 172.21.15.111:0/3264773888 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f147c06cc00 0x7f147c151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:04.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.088+0000 7f1480d9c700 1 -- 172.21.15.111:0/3264773888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f147c1589d0 msgr2=0x7f147c0c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:04.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.088+0000 7f1480d9c700 1 --2- 172.21.15.111:0/3264773888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f147c1589d0 0x7f147c0c7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:04.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.088+0000 7f1480d9c700 1 -- 172.21.15.111:0/3264773888 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1474009a70 con 0x7f147c158da0 2023-03-29T10:59:04.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.088+0000 7f1480d9c700 1 --2- 172.21.15.111:0/3264773888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f147c158da0 0x7f147c0aeb60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f147400b680 tx=0x7f1474004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:04.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.089+0000 7f14797fa700 1 -- 172.21.15.111:0/3264773888 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14740048b0 con 0x7f147c158da0 2023-03-29T10:59:04.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.089+0000 7f14797fa700 1 -- 172.21.15.111:0/3264773888 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1474011460 con 0x7f147c158da0 2023-03-29T10:59:04.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.089+0000 7f14797fa700 1 -- 172.21.15.111:0/3264773888 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1474011810 con 0x7f147c158da0 2023-03-29T10:59:04.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.089+0000 7f14827ff700 1 -- 172.21.15.111:0/3264773888 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f147c0ccc30 con 0x7f147c158da0 2023-03-29T10:59:04.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.089+0000 7f14827ff700 1 -- 172.21.15.111:0/3264773888 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147c0cc950 con 0x7f147c158da0 2023-03-29T10:59:04.089 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.090+0000 7f14827ff700 1 -- 172.21.15.111:0/3264773888 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f147c06a920 con 0x7f147c158da0 2023-03-29T10:59:04.090 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.091+0000 7f14797fa700 1 -- 172.21.15.111:0/3264773888 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f1474011970 con 0x7f147c158da0 2023-03-29T10:59:04.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.092+0000 7f14797fa700 1 --2- 172.21.15.111:0/3264773888 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f145c069de0 0x7f145c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:04.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.092+0000 7f14797fa700 1 -- 172.21.15.111:0/3264773888 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f14740897e0 con 0x7f147c158da0 2023-03-29T10:59:04.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.093+0000 7f147bfff700 1 --2- 172.21.15.111:0/3264773888 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f145c069de0 0x7f145c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:04.092 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.093+0000 7f147bfff700 1 --2- 172.21.15.111:0/3264773888 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f145c069de0 0x7f145c06c290 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f146c007fd0 tx=0x7f146c007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:04.096 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.097+0000 7f14797fa700 1 -- 172.21.15.111:0/3264773888 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f1474055260 con 0x7f147c158da0 2023-03-29T10:59:04.314 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:04.314+0000 7f14827ff700 1 -- 172.21.15.111:0/3264773888 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f147c06acf0 con 0x7f147c158da0 2023-03-29T10:59:05.063 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.063+0000 7f14797fa700 1 -- 172.21.15.111:0/3264773888 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v39) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1474058880 con 0x7f147c158da0 2023-03-29T10:59:05.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.066+0000 7f14827ff700 1 -- 172.21.15.111:0/3264773888 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f145c069de0 msgr2=0x7f145c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:05.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.066+0000 7f14827ff700 1 --2- 172.21.15.111:0/3264773888 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f145c069de0 0x7f145c06c290 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f146c007fd0 tx=0x7f146c007de0 comp rx=0 tx=0).stop 2023-03-29T10:59:05.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.066+0000 7f14827ff700 1 -- 172.21.15.111:0/3264773888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f147c158da0 msgr2=0x7f147c0aeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:05.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.066+0000 7f14827ff700 1 --2- 172.21.15.111:0/3264773888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f147c158da0 0x7f147c0aeb60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f147400b680 tx=0x7f1474004b40 comp rx=0 tx=0).stop 2023-03-29T10:59:05.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.066+0000 7f14827ff700 1 -- 172.21.15.111:0/3264773888 shutdown_connections 2023-03-29T10:59:05.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.066+0000 7f14827ff700 1 --2- 172.21.15.111:0/3264773888 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f145c069de0 0x7f145c06c290 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:05.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.066+0000 7f14827ff700 1 --2- 172.21.15.111:0/3264773888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f147c158da0 0x7f147c0aeb60 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:05.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.066+0000 7f14827ff700 1 --2- 172.21.15.111:0/3264773888 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f147c06cc00 0x7f147c151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:05.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.066+0000 7f14827ff700 1 --2- 172.21.15.111:0/3264773888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f147c1589d0 0x7f147c0c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:05.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.066+0000 7f14827ff700 1 -- 172.21.15.111:0/3264773888 >> 172.21.15.111:0/3264773888 conn(0x7f147c0cf720 msgr2=0x7f147c0c9840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:05.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.066+0000 7f14827ff700 1 -- 172.21.15.111:0/3264773888 shutdown_connections 2023-03-29T10:59:05.066 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.066+0000 7f14827ff700 1 -- 172.21.15.111:0/3264773888 wait complete. 2023-03-29T10:59:05.067 INFO:teuthology.orchestra.run.smithi111.stderr: removed all blocklist entries 2023-03-29T10:59:05.085 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-03-29T10:59:05.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.268+0000 7f9bf9dff700 1 Processor -- start 2023-03-29T10:59:05.268 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.269+0000 7f9bf9dff700 1 -- start start 2023-03-29T10:59:05.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.270+0000 7f9bf9dff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f9bf415c950 0x7f9bf40cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:05.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.270+0000 7f9bf9dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9bf40d3a30 0x7f9bf40b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:05.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.270+0000 7f9bf9dff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf40cdc80 con 0x7f9bf40cf950 2023-03-29T10:59:05.269 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.270+0000 7f9bf9dff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf40cd6c0 con 0x7f9bf415c950 2023-03-29T10:59:05.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.270+0000 7f9bf9dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf40cd9a0 con 0x7f9bf40d3a30 2023-03-29T10:59:05.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.270+0000 7f9bf37fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f9bf40cf950 0x7f9bf40cfd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41020/0 (socket says 172.21.15.111:41020) 2023-03-29T10:59:05.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.270+0000 7f9bf37fe700 1 -- 172.21.15.111:0/4270212100 learned_addr learned my addr 172.21.15.111:0/4270212100 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:59:05.270 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.270+0000 7f9bebfff700 1 --2- 172.21.15.111:0/4270212100 >> v2:172.21.15.111:3300/0 conn(0x7f9bf415c950 0x7f9bf40cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:05.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.270+0000 7f9bf3fff700 1 --2- 172.21.15.111:0/4270212100 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9bf40d3a30 0x7f9bf40b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:05.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.270+0000 7f9bf2ffd700 1 -- 172.21.15.111:0/4270212100 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722592742 0 0) 0x7f9bf40cdc80 con 0x7f9bf40cf950 2023-03-29T10:59:05.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.270+0000 7f9bf2ffd700 1 -- 172.21.15.111:0/4270212100 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9be00035b0 con 0x7f9bf40cf950 2023-03-29T10:59:05.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.270+0000 7f9bebfff700 1 -- 172.21.15.111:0/4270212100 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9bf40d3a30 msgr2=0x7f9bf40b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:05.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.270+0000 7f9bebfff700 1 --2- 172.21.15.111:0/4270212100 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9bf40d3a30 0x7f9bf40b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:05.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.270+0000 7f9bebfff700 1 -- 172.21.15.111:0/4270212100 >> v1:172.21.15.111:6789/0 conn(0x7f9bf40cf950 legacy=0x7f9bf40cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:05.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.270+0000 7f9bebfff700 1 -- 172.21.15.111:0/4270212100 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf40d16c0 con 0x7f9bf415c950 2023-03-29T10:59:05.271 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.271+0000 7f9bebfff700 1 --2- 172.21.15.111:0/4270212100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bf415c950 0x7f9bf40cf0f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009840 tx=0x7f9bdc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e081a3a6bc3b3a0d server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:05.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.271+0000 7f9bf2ffd700 1 -- 172.21.15.111:0/4270212100 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc011830 con 0x7f9bf415c950 2023-03-29T10:59:05.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.271+0000 7f9bf2ffd700 1 -- 172.21.15.111:0/4270212100 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bdc011990 con 0x7f9bf415c950 2023-03-29T10:59:05.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.271+0000 7f9bf2ffd700 1 -- 172.21.15.111:0/4270212100 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc011d20 con 0x7f9bf415c950 2023-03-29T10:59:05.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.271+0000 7f9bf9dff700 1 -- 172.21.15.111:0/4270212100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bf415c950 msgr2=0x7f9bf40cf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:05.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.271+0000 7f9bf9dff700 1 --2- 172.21.15.111:0/4270212100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bf415c950 0x7f9bf40cf0f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009840 tx=0x7f9bdc009b50 comp rx=0 tx=0).stop 2023-03-29T10:59:05.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.271+0000 7f9bf9dff700 1 -- 172.21.15.111:0/4270212100 shutdown_connections 2023-03-29T10:59:05.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.272+0000 7f9bf9dff700 1 --2- 172.21.15.111:0/4270212100 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9bf40d3a30 0x7f9bf40b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:05.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.272+0000 7f9bf9dff700 1 --2- 172.21.15.111:0/4270212100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bf415c950 0x7f9bf40cf0f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:05.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.272+0000 7f9bf9dff700 1 -- 172.21.15.111:0/4270212100 >> 172.21.15.111:0/4270212100 conn(0x7f9bf4071b10 msgr2=0x7f9bf4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:05.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.272+0000 7f9bf9dff700 1 -- 172.21.15.111:0/4270212100 shutdown_connections 2023-03-29T10:59:05.272 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.272+0000 7f9bf9dff700 1 -- 172.21.15.111:0/4270212100 wait complete. 2023-03-29T10:59:05.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.273+0000 7f9bf9dff700 1 Processor -- start 2023-03-29T10:59:05.273 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.273+0000 7f9bf9dff700 1 -- start start 2023-03-29T10:59:05.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.274+0000 7f9bf9dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bf40cf950 0x7f9bf4157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:05.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.274+0000 7f9bf9dff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bf40d3a30 0x7f9bf4157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:05.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.274+0000 7f9bf9dff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9bf415c950 0x7f9bf40d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:05.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.274+0000 7f9bf9dff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf40c7c10 con 0x7f9bf40d3a30 2023-03-29T10:59:05.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.274+0000 7f9bf9dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf4157e40 con 0x7f9bf40cf950 2023-03-29T10:59:05.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.274+0000 7f9bf9dff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf415e270 con 0x7f9bf415c950 2023-03-29T10:59:05.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.275+0000 7f9bebfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bf40d3a30 0x7f9bf4157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:05.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.275+0000 7f9bebfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bf40d3a30 0x7f9bf4157900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52136/0 (socket says 172.21.15.111:52136) 2023-03-29T10:59:05.274 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.275+0000 7f9bebfff700 1 -- 172.21.15.111:0/799706143 learned_addr learned my addr 172.21.15.111:0/799706143 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:05.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.275+0000 7f9bebfff700 1 -- 172.21.15.111:0/799706143 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9bf415c950 msgr2=0x7f9bf40d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:05.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.275+0000 7f9bebfff700 1 --2- 172.21.15.111:0/799706143 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9bf415c950 0x7f9bf40d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:05.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.275+0000 7f9bebfff700 1 -- 172.21.15.111:0/799706143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bf40cf950 msgr2=0x7f9bf4157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:05.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.275+0000 7f9bebfff700 1 --2- 172.21.15.111:0/799706143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bf40cf950 0x7f9bf4157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:05.275 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.275+0000 7f9bebfff700 1 -- 172.21.15.111:0/799706143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bdc0094a0 con 0x7f9bf40d3a30 2023-03-29T10:59:05.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.275+0000 7f9bebfff700 1 --2- 172.21.15.111:0/799706143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bf40d3a30 0x7f9bf4157900 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009fd0 tx=0x7f9bdc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:05.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.275+0000 7f9bf0ff9700 1 -- 172.21.15.111:0/799706143 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc0036a0 con 0x7f9bf40d3a30 2023-03-29T10:59:05.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.275+0000 7f9bf0ff9700 1 -- 172.21.15.111:0/799706143 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bdc01e070 con 0x7f9bf40d3a30 2023-03-29T10:59:05.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.275+0000 7f9bf0ff9700 1 -- 172.21.15.111:0/799706143 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc003a60 con 0x7f9bf40d3a30 2023-03-29T10:59:05.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.275+0000 7f9bf9dff700 1 -- 172.21.15.111:0/799706143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf40cf630 con 0x7f9bf40d3a30 2023-03-29T10:59:05.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.276+0000 7f9bf9dff700 1 -- 172.21.15.111:0/799706143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf40d19a0 con 0x7f9bf40d3a30 2023-03-29T10:59:05.276 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.277+0000 7f9bf9dff700 1 -- 172.21.15.111:0/799706143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf40b62d0 con 0x7f9bf40d3a30 2023-03-29T10:59:05.277 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.277+0000 7f9bf0ff9700 1 -- 172.21.15.111:0/799706143 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f9bdc003bc0 con 0x7f9bf40d3a30 2023-03-29T10:59:05.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.278+0000 7f9bf0ff9700 1 --2- 172.21.15.111:0/799706143 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9bd4069de0 0x7f9bd406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:05.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.278+0000 7f9bf0ff9700 1 -- 172.21.15.111:0/799706143 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f9bdc088d00 con 0x7f9bf40d3a30 2023-03-29T10:59:05.278 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.279+0000 7f9bf37fe700 1 --2- 172.21.15.111:0/799706143 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9bd4069de0 0x7f9bd406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:05.279 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.279+0000 7f9bf37fe700 1 --2- 172.21.15.111:0/799706143 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9bd4069de0 0x7f9bd406c290 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40cbaa0 tx=0x7f9be40079c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:05.282 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.283+0000 7f9bf0ff9700 1 -- 172.21.15.111:0/799706143 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f9bdc0547b0 con 0x7f9bf40d3a30 2023-03-29T10:59:05.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.550+0000 7f9bf9dff700 1 -- 172.21.15.111:0/799706143 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9bf4074430 con 0x7f9bf40d3a30 2023-03-29T10:59:05.550 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.551+0000 7f9bf0ff9700 1 -- 172.21.15.111:0/799706143 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f9bdc015090 con 0x7f9bf40d3a30 2023-03-29T10:59:05.551 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:59:05.551 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-03-29T10:59:05.551 INFO:teuthology.orchestra.run.smithi111.stdout: "auth_dump": [ 2023-03-29T10:59:05.551 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.551 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.a", 2023-03-29T10:59:05.551 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQBYGSRkbCL9NxAAPlUk/kxFQWaRb0KWzZgdzQ==", 2023-03-29T10:59:05.551 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.551 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-03-29T10:59:05.551 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2023-03-29T10:59:05.551 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2023-03-29T10:59:05.551 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-03-29T10:59:05.551 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.551 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.551 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.551 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.b", 2023-03-29T10:59:05.551 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQBZGSRkS+5iERAAMiKMA0iWZgOLY8o7YY5HBg==", 2023-03-29T10:59:05.552 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.552 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-03-29T10:59:05.552 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2023-03-29T10:59:05.552 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2023-03-29T10:59:05.552 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-03-29T10:59:05.552 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.552 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.552 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.552 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.c", 2023-03-29T10:59:05.552 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQBZGSRkOlnjBhAAYuibQZYXCSfiZ+5g2HMMhg==", 2023-03-29T10:59:05.552 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.552 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-03-29T10:59:05.552 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2023-03-29T10:59:05.552 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2023-03-29T10:59:05.552 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-03-29T10:59:05.552 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.552 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.552 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.553 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mds.d", 2023-03-29T10:59:05.553 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQBZGSRkpWrNGxAA6QPfNxtu8SOwefAkYTq7NQ==", 2023-03-29T10:59:05.553 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.553 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-03-29T10:59:05.553 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2023-03-29T10:59:05.553 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2023-03-29T10:59:05.553 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-03-29T10:59:05.553 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.553 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.553 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.553 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.0", 2023-03-29T10:59:05.553 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQBdGSRkfthGDxAAu0H6zYyBtIiGcZhhgfA5tg==", 2023-03-29T10:59:05.553 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.553 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-03-29T10:59:05.553 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-03-29T10:59:05.553 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-03-29T10:59:05.553 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.553 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.554 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.554 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.1", 2023-03-29T10:59:05.554 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQBgGSRkTA6hERAAzWcslS2JXfC3Veik8854Kw==", 2023-03-29T10:59:05.554 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.554 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-03-29T10:59:05.554 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-03-29T10:59:05.554 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-03-29T10:59:05.554 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.554 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.554 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.554 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.2", 2023-03-29T10:59:05.554 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQBjGSRkIXt/ExAAKv910aQ0DXwlapS9BaLW5Q==", 2023-03-29T10:59:05.554 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.554 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-03-29T10:59:05.554 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-03-29T10:59:05.554 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-03-29T10:59:05.554 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.554 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.555 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.555 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.3", 2023-03-29T10:59:05.555 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQBmGSRkVSmZFhAAAOK61H8Drx0cx8YEsMP9Fg==", 2023-03-29T10:59:05.555 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.555 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-03-29T10:59:05.555 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-03-29T10:59:05.555 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-03-29T10:59:05.555 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.555 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.555 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.555 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.4", 2023-03-29T10:59:05.555 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQBsGSRkV31DJxAAiWiGcWrjbX5Bh38gDLG2Og==", 2023-03-29T10:59:05.555 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.555 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-03-29T10:59:05.555 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-03-29T10:59:05.555 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-03-29T10:59:05.555 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.555 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.555 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.556 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.5", 2023-03-29T10:59:05.556 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQBvGSRkP5JQKhAAaoAfEw3cCrBEcQhRNmwLrA==", 2023-03-29T10:59:05.556 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.556 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-03-29T10:59:05.556 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-03-29T10:59:05.556 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-03-29T10:59:05.556 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.556 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.556 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.556 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.6", 2023-03-29T10:59:05.556 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQByGSRkhbiiLBAARXXU1Bwk7M3fLEDGnO+1Lw==", 2023-03-29T10:59:05.556 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.556 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-03-29T10:59:05.556 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-03-29T10:59:05.556 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-03-29T10:59:05.556 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.556 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.556 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.557 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "osd.7", 2023-03-29T10:59:05.557 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQB1GSRkShfBLxAA/FxkMObHsaXgDF3CIv354A==", 2023-03-29T10:59:05.557 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.557 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow profile osd", 2023-03-29T10:59:05.557 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile osd", 2023-03-29T10:59:05.557 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-03-29T10:59:05.557 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.557 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.557 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.557 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.0", 2023-03-29T10:59:05.557 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQBZGSRkTCPmJBAAoBPWEMDsr6r1DUWLIYeCHw==", 2023-03-29T10:59:05.557 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.557 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-03-29T10:59:05.557 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow r", 2023-03-29T10:59:05.557 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow rw", 2023-03-29T10:59:05.557 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rwx" 2023-03-29T10:59:05.557 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.558 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.558 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.558 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.1", 2023-03-29T10:59:05.558 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQBZGSRkhqZmMhAASfrTwS+BZGF/E6S34W+z6Q==", 2023-03-29T10:59:05.558 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.558 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-03-29T10:59:05.558 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow r", 2023-03-29T10:59:05.558 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow rw", 2023-03-29T10:59:05.558 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rwx" 2023-03-29T10:59:05.558 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.558 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.558 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.558 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.2", 2023-03-29T10:59:05.558 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQBZGSRkuR6dKxAAn3ZN4MVSbqXTTQQlwUVLHA==", 2023-03-29T10:59:05.558 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.558 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-03-29T10:59:05.558 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow r", 2023-03-29T10:59:05.558 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow rw", 2023-03-29T10:59:05.558 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rwx" 2023-03-29T10:59:05.559 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.559 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.559 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.559 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.3", 2023-03-29T10:59:05.559 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQBZGSRkQfseORAAGOTyPhNRfqi0NxE+EvVRbw==", 2023-03-29T10:59:05.559 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.559 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow", 2023-03-29T10:59:05.559 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow r", 2023-03-29T10:59:05.559 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow rw", 2023-03-29T10:59:05.559 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow rwx" 2023-03-29T10:59:05.559 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.559 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.559 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.559 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.admin", 2023-03-29T10:59:05.559 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQBYGSRkuxcuAxAAXotRkav5CccVRCV8v8dLNA==", 2023-03-29T10:59:05.559 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.559 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow *", 2023-03-29T10:59:05.559 INFO:teuthology.orchestra.run.smithi111.stdout: "mgr": "allow *", 2023-03-29T10:59:05.559 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow *", 2023-03-29T10:59:05.560 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-03-29T10:59:05.560 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.560 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.560 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.560 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-mds", 2023-03-29T10:59:05.560 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQB9GSRkzGW7HRAAk1K5s77DkQdvWDs4qYUonA==", 2023-03-29T10:59:05.560 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.560 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-mds" 2023-03-29T10:59:05.560 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.560 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.560 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.560 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-mgr", 2023-03-29T10:59:05.560 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQB9GSRkTG+7HRAAnS5zGsMlp0fb8jtscWjWAQ==", 2023-03-29T10:59:05.560 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.560 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-mgr" 2023-03-29T10:59:05.560 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.560 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.560 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.561 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-osd", 2023-03-29T10:59:05.561 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQB9GSRkCnm7HRAAaTFJy8QtqrNDAnT+6cnjVw==", 2023-03-29T10:59:05.561 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.561 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-osd" 2023-03-29T10:59:05.561 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.561 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.561 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.561 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-rbd", 2023-03-29T10:59:05.561 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQB9GSRkZYK7HRAAOGRQNH2kjZCmTvvHqUOZMQ==", 2023-03-29T10:59:05.561 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.561 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-rbd" 2023-03-29T10:59:05.561 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.561 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.561 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.561 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-03-29T10:59:05.561 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQB9GSRkDYu7HRAA7/ILFlZdLhm/1HnhEZS6zQ==", 2023-03-29T10:59:05.561 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-03-29T10:59:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "client.bootstrap-rgw", 2023-03-29T10:59:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQB9GSRkbJO7HRAA60dA3bSmhZ0FaJ3iPUcABQ==", 2023-03-29T10:59:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile bootstrap-rgw" 2023-03-29T10:59:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mgr.x", 2023-03-29T10:59:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQBYGSRkG0YRMRAAvfVgZ0GOj/NHocPhqdfBtA==", 2023-03-29T10:59:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow *", 2023-03-29T10:59:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile mgr", 2023-03-29T10:59:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-03-29T10:59:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.562 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:05.563 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:05.563 INFO:teuthology.orchestra.run.smithi111.stdout: "entity": "mgr.y", 2023-03-29T10:59:05.563 INFO:teuthology.orchestra.run.smithi111.stdout: "key": "AQBYGSRkJ2+PKRAAGoCnOjmVtYroJKEzZHd84A==", 2023-03-29T10:59:05.563 INFO:teuthology.orchestra.run.smithi111.stdout: "caps": { 2023-03-29T10:59:05.563 INFO:teuthology.orchestra.run.smithi111.stdout: "mds": "allow *", 2023-03-29T10:59:05.563 INFO:teuthology.orchestra.run.smithi111.stdout: "mon": "allow profile mgr", 2023-03-29T10:59:05.563 INFO:teuthology.orchestra.run.smithi111.stdout: "osd": "allow *" 2023-03-29T10:59:05.563 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.563 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:05.563 INFO:teuthology.orchestra.run.smithi111.stdout: ] 2023-03-29T10:59:05.563 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-03-29T10:59:05.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.553+0000 7f9bf9dff700 1 -- 172.21.15.111:0/799706143 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9bd4069de0 msgr2=0x7f9bd406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:05.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.553+0000 7f9bf9dff700 1 --2- 172.21.15.111:0/799706143 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9bd4069de0 0x7f9bd406c290 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9bf40cbaa0 tx=0x7f9be40079c0 comp rx=0 tx=0).stop 2023-03-29T10:59:05.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.553+0000 7f9bf9dff700 1 -- 172.21.15.111:0/799706143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bf40d3a30 msgr2=0x7f9bf4157900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:05.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.553+0000 7f9bf9dff700 1 --2- 172.21.15.111:0/799706143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bf40d3a30 0x7f9bf4157900 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009fd0 tx=0x7f9bdc004750 comp rx=0 tx=0).stop 2023-03-29T10:59:05.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.553+0000 7f9bf9dff700 1 -- 172.21.15.111:0/799706143 shutdown_connections 2023-03-29T10:59:05.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.553+0000 7f9bf9dff700 1 --2- 172.21.15.111:0/799706143 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9bd4069de0 0x7f9bd406c290 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:05.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.553+0000 7f9bf9dff700 1 --2- 172.21.15.111:0/799706143 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f9bf415c950 0x7f9bf40d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:05.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.553+0000 7f9bf9dff700 1 --2- 172.21.15.111:0/799706143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bf40cf950 0x7f9bf4157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:05.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.553+0000 7f9bf9dff700 1 --2- 172.21.15.111:0/799706143 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f9bf40d3a30 0x7f9bf4157900 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:05.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.553+0000 7f9bf9dff700 1 -- 172.21.15.111:0/799706143 >> 172.21.15.111:0/799706143 conn(0x7f9bf4071b10 msgr2=0x7f9bf4071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:05.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.553+0000 7f9bf9dff700 1 -- 172.21.15.111:0/799706143 shutdown_connections 2023-03-29T10:59:05.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.553+0000 7f9bf9dff700 1 -- 172.21.15.111:0/799706143 wait complete. 2023-03-29T10:59:05.569 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-03-29T10:59:05.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.755+0000 7f08be7ff700 1 Processor -- start 2023-03-29T10:59:05.755 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.755+0000 7f08be7ff700 1 -- start start 2023-03-29T10:59:05.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.757+0000 7f08be7ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f08b80d2110 0x7f08b8167550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:05.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.757+0000 7f08be7ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b8161000 con 0x7f08b80cfbf0 2023-03-29T10:59:05.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.757+0000 7f08be7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b80d0020 con 0x7f08b80b87a0 2023-03-29T10:59:05.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.757+0000 7f08be7ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b80d02d0 con 0x7f08b80d2110 2023-03-29T10:59:05.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.757+0000 7f08b77fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f08b80cfbf0 0x7f08b81608f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41036/0 (socket says 172.21.15.111:41036) 2023-03-29T10:59:05.757 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.757+0000 7f08b77fe700 1 -- 172.21.15.111:0/245184034 learned_addr learned my addr 172.21.15.111:0/245184034 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:59:05.757 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.757+0000 7f08bcd9c700 1 --2- 172.21.15.111:0/245184034 >> v2:172.21.15.119:3300/0 conn(0x7f08b80d2110 0x7f08b8167550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:05.757 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.757+0000 7f08b6ffd700 1 -- 172.21.15.111:0/245184034 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424002412 0 0) 0x7f08b8161000 con 0x7f08b80cfbf0 2023-03-29T10:59:05.757 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.757+0000 7f08b6ffd700 1 -- 172.21.15.111:0/245184034 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08a00035b0 con 0x7f08b80cfbf0 2023-03-29T10:59:05.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.757+0000 7f08bcd9c700 1 -- 172.21.15.111:0/245184034 >> v1:172.21.15.119:6789/0 conn(0x7f08b80b87a0 legacy=0x7f08b81601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:05.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.757+0000 7f08bcd9c700 1 -- 172.21.15.111:0/245184034 >> v1:172.21.15.111:6789/0 conn(0x7f08b80cfbf0 legacy=0x7f08b81608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:05.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.757+0000 7f08bcd9c700 1 -- 172.21.15.111:0/245184034 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b80cdc60 con 0x7f08b80d2110 2023-03-29T10:59:05.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.758+0000 7f08bcd9c700 1 --2- 172.21.15.111:0/245184034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08b80d2110 0x7f08b8167550 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009d90 tx=0x7f08b000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36af244fee362a84 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:05.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.758+0000 7f08b6ffd700 1 -- 172.21.15.111:0/245184034 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b000e070 con 0x7f08b80d2110 2023-03-29T10:59:05.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.758+0000 7f08b6ffd700 1 -- 172.21.15.111:0/245184034 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b000bb20 con 0x7f08b80d2110 2023-03-29T10:59:05.758 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.758+0000 7f08b6ffd700 1 -- 172.21.15.111:0/245184034 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b000be90 con 0x7f08b80d2110 2023-03-29T10:59:05.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.759+0000 7f08be7ff700 1 -- 172.21.15.111:0/245184034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08b80d2110 msgr2=0x7f08b8167550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:05.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.759+0000 7f08be7ff700 1 --2- 172.21.15.111:0/245184034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08b80d2110 0x7f08b8167550 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f08b0009d90 tx=0x7f08b000b330 comp rx=0 tx=0).stop 2023-03-29T10:59:05.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.759+0000 7f08be7ff700 1 -- 172.21.15.111:0/245184034 shutdown_connections 2023-03-29T10:59:05.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.759+0000 7f08be7ff700 1 --2- 172.21.15.111:0/245184034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08b80d2110 0x7f08b8167550 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:05.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.759+0000 7f08be7ff700 1 -- 172.21.15.111:0/245184034 >> 172.21.15.111:0/245184034 conn(0x7f08b8071b10 msgr2=0x7f08b8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:05.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.759+0000 7f08be7ff700 1 -- 172.21.15.111:0/245184034 shutdown_connections 2023-03-29T10:59:05.759 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.759+0000 7f08be7ff700 1 -- 172.21.15.111:0/245184034 wait complete. 2023-03-29T10:59:05.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.761+0000 7f08be7ff700 1 Processor -- start 2023-03-29T10:59:05.760 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.761+0000 7f08be7ff700 1 -- start start 2023-03-29T10:59:05.762 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.762+0000 7f08be7ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b80b87a0 0x7f08b815c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:05.762 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.762+0000 7f08be7ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08b80cfbf0 0x7f08b8153470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:05.762 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.762+0000 7f08be7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08b80d2110 0x7f08b81539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:05.762 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.762+0000 7f08be7ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b8155460 con 0x7f08b80b87a0 2023-03-29T10:59:05.762 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.762+0000 7f08be7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80c6dd0 con 0x7f08b80d2110 2023-03-29T10:59:05.762 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.762+0000 7f08be7ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b8154240 con 0x7f08b80cfbf0 2023-03-29T10:59:05.763 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.763+0000 7f08b7fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b80b87a0 0x7f08b815c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:05.763 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.763+0000 7f08b7fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b80b87a0 0x7f08b815c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52148/0 (socket says 172.21.15.111:52148) 2023-03-29T10:59:05.763 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.763+0000 7f08b7fff700 1 -- 172.21.15.111:0/2640591797 learned_addr learned my addr 172.21.15.111:0/2640591797 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:05.763 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.763+0000 7f08bcd9c700 1 --2- 172.21.15.111:0/2640591797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08b80d2110 0x7f08b81539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:05.763 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.763+0000 7f08b77fe700 1 --2- 172.21.15.111:0/2640591797 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08b80cfbf0 0x7f08b8153470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:05.763 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.763+0000 7f08b7fff700 1 -- 172.21.15.111:0/2640591797 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08b80cfbf0 msgr2=0x7f08b8153470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:05.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.763+0000 7f08b7fff700 1 --2- 172.21.15.111:0/2640591797 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08b80cfbf0 0x7f08b8153470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:05.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.763+0000 7f08b7fff700 1 -- 172.21.15.111:0/2640591797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08b80d2110 msgr2=0x7f08b81539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:05.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.763+0000 7f08b7fff700 1 --2- 172.21.15.111:0/2640591797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08b80d2110 0x7f08b81539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:05.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.763+0000 7f08b7fff700 1 -- 172.21.15.111:0/2640591797 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b0009a70 con 0x7f08b80b87a0 2023-03-29T10:59:05.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.763+0000 7f08b7fff700 1 --2- 172.21.15.111:0/2640591797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b80b87a0 0x7f08b815c370 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f08a800c430 tx=0x7f08a800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:05.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.763+0000 7f08b4ff9700 1 -- 172.21.15.111:0/2640591797 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a80052d0 con 0x7f08b80b87a0 2023-03-29T10:59:05.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.764+0000 7f08b4ff9700 1 -- 172.21.15.111:0/2640591797 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a8005450 con 0x7f08b80b87a0 2023-03-29T10:59:05.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.764+0000 7f08be7ff700 1 -- 172.21.15.111:0/2640591797 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b80cd980 con 0x7f08b80b87a0 2023-03-29T10:59:05.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.764+0000 7f08b4ff9700 1 -- 172.21.15.111:0/2640591797 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a8006aa0 con 0x7f08b80b87a0 2023-03-29T10:59:05.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.764+0000 7f08be7ff700 1 -- 172.21.15.111:0/2640591797 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b80cd6a0 con 0x7f08b80b87a0 2023-03-29T10:59:05.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.765+0000 7f08be7ff700 1 -- 172.21.15.111:0/2640591797 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b8167a90 con 0x7f08b80b87a0 2023-03-29T10:59:05.765 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.766+0000 7f08b4ff9700 1 -- 172.21.15.111:0/2640591797 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f08a8003680 con 0x7f08b80b87a0 2023-03-29T10:59:05.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.766+0000 7f08b4ff9700 1 --2- 172.21.15.111:0/2640591797 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f08a4069de0 0x7f08a406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:05.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.766+0000 7f08b4ff9700 1 -- 172.21.15.111:0/2640591797 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f08a8086fa0 con 0x7f08b80b87a0 2023-03-29T10:59:05.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.767+0000 7f08b77fe700 1 --2- 172.21.15.111:0/2640591797 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f08a4069de0 0x7f08a406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:05.766 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.767+0000 7f08b77fe700 1 --2- 172.21.15.111:0/2640591797 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f08a4069de0 0x7f08a406c290 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f08b81f5e60 tx=0x7f08ac009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:05.770 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:05.771+0000 7f08b4ff9700 1 -- 172.21.15.111:0/2640591797 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f08a804b160 con 0x7f08b80b87a0 2023-03-29T10:59:06.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.040+0000 7f08be7ff700 1 -- 172.21.15.111:0/2640591797 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f08b8071ee0 con 0x7f08b80b87a0 2023-03-29T10:59:06.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.045+0000 7f08b4ff9700 1 -- 172.21.15.111:0/2640591797 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f08a8052a00 con 0x7f08b80b87a0 2023-03-29T10:59:06.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.047+0000 7f08be7ff700 1 -- 172.21.15.111:0/2640591797 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f08a4069de0 msgr2=0x7f08a406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:06.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.047+0000 7f08be7ff700 1 --2- 172.21.15.111:0/2640591797 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f08a4069de0 0x7f08a406c290 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f08b81f5e60 tx=0x7f08ac009040 comp rx=0 tx=0).stop 2023-03-29T10:59:06.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.047+0000 7f08be7ff700 1 -- 172.21.15.111:0/2640591797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b80b87a0 msgr2=0x7f08b815c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:06.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.047+0000 7f08be7ff700 1 --2- 172.21.15.111:0/2640591797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b80b87a0 0x7f08b815c370 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f08a800c430 tx=0x7f08a800c7f0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.047+0000 7f08be7ff700 1 -- 172.21.15.111:0/2640591797 shutdown_connections 2023-03-29T10:59:06.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.047+0000 7f08be7ff700 1 --2- 172.21.15.111:0/2640591797 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f08a4069de0 0x7f08a406c290 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.047+0000 7f08be7ff700 1 --2- 172.21.15.111:0/2640591797 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08b80b87a0 0x7f08b815c370 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.047+0000 7f08be7ff700 1 --2- 172.21.15.111:0/2640591797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08b80d2110 0x7f08b81539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.047+0000 7f08be7ff700 1 --2- 172.21.15.111:0/2640591797 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08b80cfbf0 0x7f08b8153470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.047+0000 7f08be7ff700 1 -- 172.21.15.111:0/2640591797 >> 172.21.15.111:0/2640591797 conn(0x7f08b8071b10 msgr2=0x7f08b80d33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:06.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.047+0000 7f08be7ff700 1 -- 172.21.15.111:0/2640591797 shutdown_connections 2023-03-29T10:59:06.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.047+0000 7f08be7ff700 1 -- 172.21.15.111:0/2640591797 wait complete. 2023-03-29T10:59:06.063 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-03-29T10:59:06.247 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.247+0000 7f4bf7bff700 1 Processor -- start 2023-03-29T10:59:06.247 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.247+0000 7f4bf7bff700 1 -- start start 2023-03-29T10:59:06.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.248+0000 7f4bf7bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bf00b87a0 0x7f4bf00cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:06.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.248+0000 7f4bf7bff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4bf0160d40 0x7f4bf00d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:06.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.248+0000 7f4bf7bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf00cdc60 con 0x7f4bf01601e0 2023-03-29T10:59:06.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.248+0000 7f4bf7bff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf00d02d0 con 0x7f4bf0160d40 2023-03-29T10:59:06.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.248+0000 7f4bf7bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf00d0cb0 con 0x7f4bf00b87a0 2023-03-29T10:59:06.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.248+0000 7f4bf619c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4bf0160d40 0x7f4bf00d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:06.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.248+0000 7f4bf619c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4bf0160d40 0x7f4bf00d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52156/0 (socket says 172.21.15.111:52156) 2023-03-29T10:59:06.249 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.248+0000 7f4bf619c700 1 -- 172.21.15.111:0/3160334854 learned_addr learned my addr 172.21.15.111:0/3160334854 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:06.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.249+0000 7f4bf599b700 1 --2- 172.21.15.111:0/3160334854 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bf00b87a0 0x7f4bf00cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:06.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.249+0000 7f4bf619c700 1 -- 172.21.15.111:0/3160334854 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bf00b87a0 msgr2=0x7f4bf00cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:06.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.249+0000 7f4bf619c700 1 --2- 172.21.15.111:0/3160334854 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bf00b87a0 0x7f4bf00cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.249+0000 7f4bf619c700 1 -- 172.21.15.111:0/3160334854 >> v1:172.21.15.119:6789/0 conn(0x7f4bf01601e0 legacy=0x7f4bf0160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:06.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.249+0000 7f4bf619c700 1 -- 172.21.15.111:0/3160334854 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf00cd6a0 con 0x7f4bf0160d40 2023-03-29T10:59:06.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.249+0000 7f4bf619c700 1 --2- 172.21.15.111:0/3160334854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bf0160d40 0x7f4bf00d1100 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4be8009e50 tx=0x7f4be800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e80cedd3b44e94 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:06.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.249+0000 7f4bf4999700 1 -- 172.21.15.111:0/3160334854 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be800e070 con 0x7f4bf0160d40 2023-03-29T10:59:06.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.249+0000 7f4bf4999700 1 -- 172.21.15.111:0/3160334854 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be8003a20 con 0x7f4bf0160d40 2023-03-29T10:59:06.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.249+0000 7f4bf4999700 1 -- 172.21.15.111:0/3160334854 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be8003dd0 con 0x7f4bf0160d40 2023-03-29T10:59:06.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.250+0000 7f4bf7bff700 1 -- 172.21.15.111:0/3160334854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bf0160d40 msgr2=0x7f4bf00d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:06.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.250+0000 7f4bf7bff700 1 --2- 172.21.15.111:0/3160334854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bf0160d40 0x7f4bf00d1100 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4be8009e50 tx=0x7f4be800b330 comp rx=0 tx=0).stop 2023-03-29T10:59:06.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.250+0000 7f4bf7bff700 1 -- 172.21.15.111:0/3160334854 shutdown_connections 2023-03-29T10:59:06.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.250+0000 7f4bf7bff700 1 --2- 172.21.15.111:0/3160334854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bf0160d40 0x7f4bf00d1100 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.250+0000 7f4bf7bff700 1 --2- 172.21.15.111:0/3160334854 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bf00b87a0 0x7f4bf00cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.250+0000 7f4bf7bff700 1 -- 172.21.15.111:0/3160334854 >> 172.21.15.111:0/3160334854 conn(0x7f4bf0071b10 msgr2=0x7f4bf0071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:06.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.250+0000 7f4bf7bff700 1 -- 172.21.15.111:0/3160334854 shutdown_connections 2023-03-29T10:59:06.251 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.250+0000 7f4bf7bff700 1 -- 172.21.15.111:0/3160334854 wait complete. 2023-03-29T10:59:06.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.252+0000 7f4bf7bff700 1 Processor -- start 2023-03-29T10:59:06.252 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.252+0000 7f4bf7bff700 1 -- start start 2023-03-29T10:59:06.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.252+0000 7f4bf7bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bf00b87a0 0x7f4bf00d2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:06.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.252+0000 7f4bf7bff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bf01601e0 0x7f4bf0150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:06.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.253+0000 7f4bf7bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bf0160d40 0x7f4bf01514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:06.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.253+0000 7f4bf7bff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf01f2f30 con 0x7f4bf01601e0 2023-03-29T10:59:06.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.253+0000 7f4bf7bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf00c8220 con 0x7f4bf00b87a0 2023-03-29T10:59:06.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.253+0000 7f4bf7bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf00c7c60 con 0x7f4bf0160d40 2023-03-29T10:59:06.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.253+0000 7f4bf519a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bf01601e0 0x7f4bf0150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:06.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.253+0000 7f4bf519a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bf01601e0 0x7f4bf0150f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52168/0 (socket says 172.21.15.111:52168) 2023-03-29T10:59:06.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.253+0000 7f4bf519a700 1 -- 172.21.15.111:0/192527607 learned_addr learned my addr 172.21.15.111:0/192527607 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:06.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.253+0000 7f4bf599b700 1 --2- 172.21.15.111:0/192527607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bf00b87a0 0x7f4bf00d2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:06.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.253+0000 7f4bf619c700 1 --2- 172.21.15.111:0/192527607 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bf0160d40 0x7f4bf01514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:06.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.253+0000 7f4bf519a700 1 -- 172.21.15.111:0/192527607 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bf0160d40 msgr2=0x7f4bf01514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:06.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.253+0000 7f4bf519a700 1 --2- 172.21.15.111:0/192527607 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bf0160d40 0x7f4bf01514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.253+0000 7f4bf519a700 1 -- 172.21.15.111:0/192527607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bf00b87a0 msgr2=0x7f4bf00d2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:06.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.253+0000 7f4bf519a700 1 --2- 172.21.15.111:0/192527607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bf00b87a0 0x7f4bf00d2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.254 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.253+0000 7f4bf519a700 1 -- 172.21.15.111:0/192527607 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be8009b30 con 0x7f4bf01601e0 2023-03-29T10:59:06.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.253+0000 7f4bf619c700 1 --2- 172.21.15.111:0/192527607 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bf0160d40 0x7f4bf01514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:59:06.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.253+0000 7f4bf519a700 1 --2- 172.21.15.111:0/192527607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bf01601e0 0x7f4bf0150f80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4be000c4d0 tx=0x7f4be000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:06.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.254+0000 7f4be67fc700 1 -- 172.21.15.111:0/192527607 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0011790 con 0x7f4bf01601e0 2023-03-29T10:59:06.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.254+0000 7f4be67fc700 1 -- 172.21.15.111:0/192527607 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be00118f0 con 0x7f4bf01601e0 2023-03-29T10:59:06.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.254+0000 7f4be67fc700 1 -- 172.21.15.111:0/192527607 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be0011ca0 con 0x7f4bf01601e0 2023-03-29T10:59:06.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.254+0000 7f4bf7bff700 1 -- 172.21.15.111:0/192527607 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf00d1640 con 0x7f4bf01601e0 2023-03-29T10:59:06.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.254+0000 7f4bf7bff700 1 -- 172.21.15.111:0/192527607 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf00cd980 con 0x7f4bf01601e0 2023-03-29T10:59:06.255 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.255+0000 7f4bf7bff700 1 -- 172.21.15.111:0/192527607 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf00d17d0 con 0x7f4bf01601e0 2023-03-29T10:59:06.256 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.256+0000 7f4be67fc700 1 -- 172.21.15.111:0/192527607 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f4be001c460 con 0x7f4bf01601e0 2023-03-29T10:59:06.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.256+0000 7f4be67fc700 1 --2- 172.21.15.111:0/192527607 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4bd4069de0 0x7f4bd406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:06.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.257+0000 7f4be67fc700 1 -- 172.21.15.111:0/192527607 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f4be0087bb0 con 0x7f4bf01601e0 2023-03-29T10:59:06.257 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.257+0000 7f4bf599b700 1 --2- 172.21.15.111:0/192527607 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4bd4069de0 0x7f4bd406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:06.258 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.257+0000 7f4bf599b700 1 --2- 172.21.15.111:0/192527607 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4bd4069de0 0x7f4bd406c290 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4bec006b20 tx=0x7f4bec008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:06.261 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.261+0000 7f4be67fc700 1 -- 172.21.15.111:0/192527607 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f4be0053600 con 0x7f4bf01601e0 2023-03-29T10:59:06.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.534+0000 7f4bf7bff700 1 -- 172.21.15.111:0/192527607 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f4bf0074430 con 0x7f4bf01601e0 2023-03-29T10:59:06.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.538+0000 7f4be67fc700 1 -- 172.21.15.111:0/192527607 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4be0056c20 con 0x7f4bf01601e0 2023-03-29T10:59:06.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.540+0000 7f4bf7bff700 1 -- 172.21.15.111:0/192527607 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4bd4069de0 msgr2=0x7f4bd406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:06.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.540+0000 7f4bf7bff700 1 --2- 172.21.15.111:0/192527607 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4bd4069de0 0x7f4bd406c290 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4bec006b20 tx=0x7f4bec008040 comp rx=0 tx=0).stop 2023-03-29T10:59:06.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.540+0000 7f4bf7bff700 1 -- 172.21.15.111:0/192527607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bf01601e0 msgr2=0x7f4bf0150f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:06.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.540+0000 7f4bf7bff700 1 --2- 172.21.15.111:0/192527607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bf01601e0 0x7f4bf0150f80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4be000c4d0 tx=0x7f4be000c7e0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.540+0000 7f4bf7bff700 1 -- 172.21.15.111:0/192527607 shutdown_connections 2023-03-29T10:59:06.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.540+0000 7f4bf7bff700 1 --2- 172.21.15.111:0/192527607 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4bd4069de0 0x7f4bd406c290 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.540+0000 7f4bf7bff700 1 --2- 172.21.15.111:0/192527607 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4bf0160d40 0x7f4bf01514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.540+0000 7f4bf7bff700 1 --2- 172.21.15.111:0/192527607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bf00b87a0 0x7f4bf00d2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.540+0000 7f4bf7bff700 1 --2- 172.21.15.111:0/192527607 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4bf01601e0 0x7f4bf0150f80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.541 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.540+0000 7f4bf7bff700 1 -- 172.21.15.111:0/192527607 >> 172.21.15.111:0/192527607 conn(0x7f4bf0071b10 msgr2=0x7f4bf00cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:06.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.540+0000 7f4bf7bff700 1 -- 172.21.15.111:0/192527607 shutdown_connections 2023-03-29T10:59:06.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.541+0000 7f4bf7bff700 1 -- 172.21.15.111:0/192527607 wait complete. 2023-03-29T10:59:06.558 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-03-29T10:59:06.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.740+0000 7f65225ff700 1 Processor -- start 2023-03-29T10:59:06.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.740+0000 7f65225ff700 1 -- start start 2023-03-29T10:59:06.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.741+0000 7f65225ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f651c0d0930 0x7f651c0ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:06.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.741+0000 7f65225ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f651c06bc00 0x7f651c0ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:06.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.741+0000 7f65225ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651c0aeed0 con 0x7f651c0ae8e0 2023-03-29T10:59:06.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.741+0000 7f65225ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f651c0af1b0 con 0x7f651c06bc00 2023-03-29T10:59:06.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.741+0000 7f65225ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0af490 con 0x7f651c0d0930 2023-03-29T10:59:06.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.741+0000 7f6520b9c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f651c06bc00 0x7f651c0ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:06.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.742+0000 7f6520b9c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f651c06bc00 0x7f651c0ceb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52170/0 (socket says 172.21.15.111:52170) 2023-03-29T10:59:06.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.742+0000 7f6520b9c700 1 -- 172.21.15.111:0/1069831078 learned_addr learned my addr 172.21.15.111:0/1069831078 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:06.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.742+0000 7f651bfff700 1 --2- 172.21.15.111:0/1069831078 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f651c0d0930 0x7f651c0ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:06.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.742+0000 7f6520b9c700 1 -- 172.21.15.111:0/1069831078 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f651c0d0930 msgr2=0x7f651c0ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:06.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.742+0000 7f6520b9c700 1 --2- 172.21.15.111:0/1069831078 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f651c0d0930 0x7f651c0ae3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.742+0000 7f6520b9c700 1 -- 172.21.15.111:0/1069831078 >> v1:172.21.15.119:6789/0 conn(0x7f651c0ae8e0 legacy=0x7f651c06b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:06.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.742+0000 7f6520b9c700 1 -- 172.21.15.111:0/1069831078 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c0af770 con 0x7f651c06bc00 2023-03-29T10:59:06.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.742+0000 7f6520b9c700 1 --2- 172.21.15.111:0/1069831078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f651c06bc00 0x7f651c0ceb50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6514009e50 tx=0x7f651400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3776fcb5efaeee server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:06.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.742+0000 7f651affd700 1 -- 172.21.15.111:0/1069831078 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400e070 con 0x7f651c06bc00 2023-03-29T10:59:06.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.742+0000 7f651affd700 1 -- 172.21.15.111:0/1069831078 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6514003a20 con 0x7f651c06bc00 2023-03-29T10:59:06.743 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.743+0000 7f651affd700 1 -- 172.21.15.111:0/1069831078 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6514003dd0 con 0x7f651c06bc00 2023-03-29T10:59:06.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.743+0000 7f65225ff700 1 -- 172.21.15.111:0/1069831078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f651c06bc00 msgr2=0x7f651c0ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:06.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.743+0000 7f65225ff700 1 --2- 172.21.15.111:0/1069831078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f651c06bc00 0x7f651c0ceb50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6514009e50 tx=0x7f651400b330 comp rx=0 tx=0).stop 2023-03-29T10:59:06.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.743+0000 7f65225ff700 1 -- 172.21.15.111:0/1069831078 shutdown_connections 2023-03-29T10:59:06.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.743+0000 7f65225ff700 1 --2- 172.21.15.111:0/1069831078 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f651c06bc00 0x7f651c0ceb50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.743+0000 7f65225ff700 1 --2- 172.21.15.111:0/1069831078 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f651c0d0930 0x7f651c0ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.743+0000 7f65225ff700 1 -- 172.21.15.111:0/1069831078 >> 172.21.15.111:0/1069831078 conn(0x7f651c09a0a0 msgr2=0x7f651c0d0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:06.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.743+0000 7f65225ff700 1 -- 172.21.15.111:0/1069831078 shutdown_connections 2023-03-29T10:59:06.744 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.743+0000 7f65225ff700 1 -- 172.21.15.111:0/1069831078 wait complete. 2023-03-29T10:59:06.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.745+0000 7f65225ff700 1 Processor -- start 2023-03-29T10:59:06.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.746+0000 7f65225ff700 1 -- start start 2023-03-29T10:59:06.747 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.747+0000 7f65225ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f651c06bc00 0x7f651c0b2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:06.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.747+0000 7f65225ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c0ae8e0 0x7f651c0c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:06.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.747+0000 7f65225ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f651c0d0930 0x7f651c15ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:06.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.747+0000 7f65225ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0b3070 con 0x7f651c06bc00 2023-03-29T10:59:06.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.747+0000 7f65225ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c141930 con 0x7f651c0ae8e0 2023-03-29T10:59:06.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.747+0000 7f65225ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651c141000 con 0x7f651c0d0930 2023-03-29T10:59:06.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.747+0000 7f651bfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f651c06bc00 0x7f651c0b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:06.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.747+0000 7f651bfff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f651c06bc00 0x7f651c0b2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52176/0 (socket says 172.21.15.111:52176) 2023-03-29T10:59:06.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.747+0000 7f651bfff700 1 -- 172.21.15.111:0/2987097555 learned_addr learned my addr 172.21.15.111:0/2987097555 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:06.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.747+0000 7f651b7fe700 1 --2- 172.21.15.111:0/2987097555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c0ae8e0 0x7f651c0c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:06.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.747+0000 7f651bfff700 1 -- 172.21.15.111:0/2987097555 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f651c0d0930 msgr2=0x7f651c15ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:06.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.747+0000 7f651bfff700 1 --2- 172.21.15.111:0/2987097555 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f651c0d0930 0x7f651c15ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.747+0000 7f651bfff700 1 -- 172.21.15.111:0/2987097555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c0ae8e0 msgr2=0x7f651c0c6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:06.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.748+0000 7f651bfff700 1 --2- 172.21.15.111:0/2987097555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c0ae8e0 0x7f651c0c6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:06.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.748+0000 7f651bfff700 1 -- 172.21.15.111:0/2987097555 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6514009b30 con 0x7f651c06bc00 2023-03-29T10:59:06.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.748+0000 7f651b7fe700 1 --2- 172.21.15.111:0/2987097555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c0ae8e0 0x7f651c0c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:59:06.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.748+0000 7f651bfff700 1 --2- 172.21.15.111:0/2987097555 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f651c06bc00 0x7f651c0b2840 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f650c00e810 tx=0x7f650c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:06.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.748+0000 7f6518ff9700 1 -- 172.21.15.111:0/2987097555 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c013610 con 0x7f651c06bc00 2023-03-29T10:59:06.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.748+0000 7f6518ff9700 1 -- 172.21.15.111:0/2987097555 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650c013770 con 0x7f651c06bc00 2023-03-29T10:59:06.749 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.748+0000 7f6518ff9700 1 -- 172.21.15.111:0/2987097555 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c013b20 con 0x7f651c06bc00 2023-03-29T10:59:06.750 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.748+0000 7f65225ff700 1 -- 172.21.15.111:0/2987097555 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651c0cf090 con 0x7f651c06bc00 2023-03-29T10:59:06.750 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.748+0000 7f65225ff700 1 -- 172.21.15.111:0/2987097555 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c0b0060 con 0x7f651c06bc00 2023-03-29T10:59:06.750 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.749+0000 7f65225ff700 1 -- 172.21.15.111:0/2987097555 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f651c0cf220 con 0x7f651c06bc00 2023-03-29T10:59:06.750 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.750+0000 7f6518ff9700 1 -- 172.21.15.111:0/2987097555 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f650c013c80 con 0x7f651c06bc00 2023-03-29T10:59:06.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.751+0000 7f6518ff9700 1 --2- 172.21.15.111:0/2987097555 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6508069de0 0x7f650806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:06.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.752+0000 7f6518ff9700 1 -- 172.21.15.111:0/2987097555 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f650c087ca0 con 0x7f651c06bc00 2023-03-29T10:59:06.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.752+0000 7f651b7fe700 1 --2- 172.21.15.111:0/2987097555 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6508069de0 0x7f650806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:06.753 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.752+0000 7f651b7fe700 1 --2- 172.21.15.111:0/2987097555 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6508069de0 0x7f650806c290 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f651c0c7e00 tx=0x7f6510009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:06.756 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:06.756+0000 7f6518ff9700 1 -- 172.21.15.111:0/2987097555 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f650c0570f0 con 0x7f651c06bc00 2023-03-29T10:59:07.028 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.027+0000 7f65225ff700 1 -- 172.21.15.111:0/2987097555 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f651c0afd90 con 0x7f651c06bc00 2023-03-29T10:59:07.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.031+0000 7f6518ff9700 1 -- 172.21.15.111:0/2987097555 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f650c056ce0 con 0x7f651c06bc00 2023-03-29T10:59:07.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.033+0000 7f65225ff700 1 -- 172.21.15.111:0/2987097555 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6508069de0 msgr2=0x7f650806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:07.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.033+0000 7f65225ff700 1 --2- 172.21.15.111:0/2987097555 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6508069de0 0x7f650806c290 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f651c0c7e00 tx=0x7f6510009040 comp rx=0 tx=0).stop 2023-03-29T10:59:07.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.033+0000 7f65225ff700 1 -- 172.21.15.111:0/2987097555 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f651c06bc00 msgr2=0x7f651c0b2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:07.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.033+0000 7f65225ff700 1 --2- 172.21.15.111:0/2987097555 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f651c06bc00 0x7f651c0b2840 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f650c00e810 tx=0x7f650c00ebd0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.033+0000 7f65225ff700 1 -- 172.21.15.111:0/2987097555 shutdown_connections 2023-03-29T10:59:07.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.033+0000 7f65225ff700 1 --2- 172.21.15.111:0/2987097555 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6508069de0 0x7f650806c290 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.033+0000 7f65225ff700 1 --2- 172.21.15.111:0/2987097555 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f651c06bc00 0x7f651c0b2840 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.033+0000 7f65225ff700 1 --2- 172.21.15.111:0/2987097555 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f651c0d0930 0x7f651c15ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.033+0000 7f65225ff700 1 --2- 172.21.15.111:0/2987097555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c0ae8e0 0x7f651c0c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.033+0000 7f65225ff700 1 -- 172.21.15.111:0/2987097555 >> 172.21.15.111:0/2987097555 conn(0x7f651c09a0a0 msgr2=0x7f651c0b4480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:07.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.033+0000 7f65225ff700 1 -- 172.21.15.111:0/2987097555 shutdown_connections 2023-03-29T10:59:07.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.034+0000 7f65225ff700 1 -- 172.21.15.111:0/2987097555 wait complete. 2023-03-29T10:59:07.051 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-03-29T10:59:07.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.235+0000 7fa0dc7ff700 1 Processor -- start 2023-03-29T10:59:07.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.235+0000 7fa0dc7ff700 1 -- start start 2023-03-29T10:59:07.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.236+0000 7fa0dc7ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa0d4158da0 0x7fa0d406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:07.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.236+0000 7fa0dc7ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa0d406cc00 0x7fa0d406a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:07.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.236+0000 7fa0dc7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d40ca890 con 0x7fa0d41589d0 2023-03-29T10:59:07.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.237+0000 7fa0dc7ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0d40cd860 con 0x7fa0d406cc00 2023-03-29T10:59:07.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.237+0000 7fa0dc7ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d40ca5b0 con 0x7fa0d4158da0 2023-03-29T10:59:07.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.237+0000 7fa0dad9c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa0d406cc00 0x7fa0d406a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:07.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.237+0000 7fa0dad9c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fa0d406cc00 0x7fa0d406a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52182/0 (socket says 172.21.15.111:52182) 2023-03-29T10:59:07.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.237+0000 7fa0dad9c700 1 -- 172.21.15.111:0/3749854150 learned_addr learned my addr 172.21.15.111:0/3749854150 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:07.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.237+0000 7fa0d9d9a700 1 --2- 172.21.15.111:0/3749854150 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa0d4158da0 0x7fa0d406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:07.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.237+0000 7fa0dad9c700 1 -- 172.21.15.111:0/3749854150 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa0d4158da0 msgr2=0x7fa0d406c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:07.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.237+0000 7fa0dad9c700 1 --2- 172.21.15.111:0/3749854150 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa0d4158da0 0x7fa0d406c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.237+0000 7fa0dad9c700 1 -- 172.21.15.111:0/3749854150 >> v1:172.21.15.119:6789/0 conn(0x7fa0d41589d0 legacy=0x7fa0d40cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:07.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.237+0000 7fa0dad9c700 1 -- 172.21.15.111:0/3749854150 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d40cc670 con 0x7fa0d406cc00 2023-03-29T10:59:07.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.237+0000 7fa0dad9c700 1 --2- 172.21.15.111:0/3749854150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d406cc00 0x7fa0d406a3e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00d150 tx=0x7fa0cc00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282b2f7db5c99f47 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:07.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.238+0000 7fa0d9599700 1 -- 172.21.15.111:0/3749854150 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc010070 con 0x7fa0d406cc00 2023-03-29T10:59:07.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.238+0000 7fa0d9599700 1 -- 172.21.15.111:0/3749854150 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0cc00db00 con 0x7fa0d406cc00 2023-03-29T10:59:07.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.238+0000 7fa0d9599700 1 -- 172.21.15.111:0/3749854150 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc016400 con 0x7fa0d406cc00 2023-03-29T10:59:07.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.238+0000 7fa0dc7ff700 1 -- 172.21.15.111:0/3749854150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d406cc00 msgr2=0x7fa0d406a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:07.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.238+0000 7fa0dc7ff700 1 --2- 172.21.15.111:0/3749854150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d406cc00 0x7fa0d406a3e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00d150 tx=0x7fa0cc00d460 comp rx=0 tx=0).stop 2023-03-29T10:59:07.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.238+0000 7fa0dc7ff700 1 -- 172.21.15.111:0/3749854150 shutdown_connections 2023-03-29T10:59:07.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.238+0000 7fa0dc7ff700 1 --2- 172.21.15.111:0/3749854150 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d406cc00 0x7fa0d406a3e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.238+0000 7fa0dc7ff700 1 --2- 172.21.15.111:0/3749854150 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa0d4158da0 0x7fa0d406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.238+0000 7fa0dc7ff700 1 -- 172.21.15.111:0/3749854150 >> 172.21.15.111:0/3749854150 conn(0x7fa0d40cf720 msgr2=0x7fa0d40cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:07.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.238+0000 7fa0dc7ff700 1 -- 172.21.15.111:0/3749854150 shutdown_connections 2023-03-29T10:59:07.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.238+0000 7fa0dc7ff700 1 -- 172.21.15.111:0/3749854150 wait complete. 2023-03-29T10:59:07.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.240+0000 7fa0dc7ff700 1 Processor -- start 2023-03-29T10:59:07.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.240+0000 7fa0dc7ff700 1 -- start start 2023-03-29T10:59:07.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.241+0000 7fa0dc7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0d406cc00 0x7fa0d40ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:07.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.241+0000 7fa0dc7ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa0d41589d0 0x7fa0d415bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:07.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.241+0000 7fa0dc7ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d4158da0 0x7fa0d4153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:07.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.241+0000 7fa0dc7ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d414f870 con 0x7fa0d4158da0 2023-03-29T10:59:07.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.241+0000 7fa0dc7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d414fb50 con 0x7fa0d406cc00 2023-03-29T10:59:07.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.241+0000 7fa0dc7ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d414fe30 con 0x7fa0d41589d0 2023-03-29T10:59:07.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.241+0000 7fa0dad9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d4158da0 0x7fa0d4153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:07.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.241+0000 7fa0dad9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d4158da0 0x7fa0d4153a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52194/0 (socket says 172.21.15.111:52194) 2023-03-29T10:59:07.242 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.241+0000 7fa0dad9c700 1 -- 172.21.15.111:0/1993544488 learned_addr learned my addr 172.21.15.111:0/1993544488 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:07.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.242+0000 7fa0da59b700 1 --2- 172.21.15.111:0/1993544488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0d406cc00 0x7fa0d40ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:07.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.242+0000 7fa0d9d9a700 1 --2- 172.21.15.111:0/1993544488 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa0d41589d0 0x7fa0d415bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:07.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.242+0000 7fa0dad9c700 1 -- 172.21.15.111:0/1993544488 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa0d41589d0 msgr2=0x7fa0d415bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:07.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.242+0000 7fa0dad9c700 1 --2- 172.21.15.111:0/1993544488 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa0d41589d0 0x7fa0d415bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.242+0000 7fa0dad9c700 1 -- 172.21.15.111:0/1993544488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0d406cc00 msgr2=0x7fa0d40ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:07.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.242+0000 7fa0dad9c700 1 --2- 172.21.15.111:0/1993544488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0d406cc00 0x7fa0d40ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.242+0000 7fa0dad9c700 1 -- 172.21.15.111:0/1993544488 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc00bd40 con 0x7fa0d4158da0 2023-03-29T10:59:07.243 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.242+0000 7fa0da59b700 1 --2- 172.21.15.111:0/1993544488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0d406cc00 0x7fa0d40ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:59:07.244 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.242+0000 7fa0dad9c700 1 --2- 172.21.15.111:0/1993544488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d4158da0 0x7fa0d4153a90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40708b0 tx=0x7fa0cc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:07.244 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.242+0000 7fa0caffd700 1 -- 172.21.15.111:0/1993544488 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc0176c0 con 0x7fa0d4158da0 2023-03-29T10:59:07.244 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.242+0000 7fa0caffd700 1 -- 172.21.15.111:0/1993544488 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0cc017840 con 0x7fa0d4158da0 2023-03-29T10:59:07.244 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.242+0000 7fa0dc7ff700 1 -- 172.21.15.111:0/1993544488 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d40ccc30 con 0x7fa0d4158da0 2023-03-29T10:59:07.244 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.242+0000 7fa0caffd700 1 -- 172.21.15.111:0/1993544488 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc017bf0 con 0x7fa0d4158da0 2023-03-29T10:59:07.244 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.242+0000 7fa0dc7ff700 1 -- 172.21.15.111:0/1993544488 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d40cc950 con 0x7fa0d4158da0 2023-03-29T10:59:07.244 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.243+0000 7fa0dc7ff700 1 -- 172.21.15.111:0/1993544488 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d406a920 con 0x7fa0d4158da0 2023-03-29T10:59:07.245 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.244+0000 7fa0caffd700 1 -- 172.21.15.111:0/1993544488 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fa0cc017d50 con 0x7fa0d4158da0 2023-03-29T10:59:07.246 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.245+0000 7fa0caffd700 1 --2- 172.21.15.111:0/1993544488 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa0b8069de0 0x7fa0b806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:07.246 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.245+0000 7fa0caffd700 1 -- 172.21.15.111:0/1993544488 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fa0cc0897f0 con 0x7fa0d4158da0 2023-03-29T10:59:07.246 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.246+0000 7fa0da59b700 1 --2- 172.21.15.111:0/1993544488 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa0b8069de0 0x7fa0b806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:07.247 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.246+0000 7fa0da59b700 1 --2- 172.21.15.111:0/1993544488 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa0b8069de0 0x7fa0b806c290 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa0d414bda0 tx=0x7fa0d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:07.250 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.250+0000 7fa0caffd700 1 -- 172.21.15.111:0/1993544488 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fa0cc0552a0 con 0x7fa0d4158da0 2023-03-29T10:59:07.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.522+0000 7fa0dc7ff700 1 -- 172.21.15.111:0/1993544488 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fa0d40b4a10 con 0x7fa0d4158da0 2023-03-29T10:59:07.526 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.526+0000 7fa0caffd700 1 -- 172.21.15.111:0/1993544488 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa0cc0588c0 con 0x7fa0d4158da0 2023-03-29T10:59:07.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.528+0000 7fa0dc7ff700 1 -- 172.21.15.111:0/1993544488 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa0b8069de0 msgr2=0x7fa0b806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:07.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.529+0000 7fa0dc7ff700 1 --2- 172.21.15.111:0/1993544488 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa0b8069de0 0x7fa0b806c290 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa0d414bda0 tx=0x7fa0d0009040 comp rx=0 tx=0).stop 2023-03-29T10:59:07.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.529+0000 7fa0dc7ff700 1 -- 172.21.15.111:0/1993544488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d4158da0 msgr2=0x7fa0d4153a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:07.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.529+0000 7fa0dc7ff700 1 --2- 172.21.15.111:0/1993544488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d4158da0 0x7fa0d4153a90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40708b0 tx=0x7fa0cc004ad0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.529+0000 7fa0dc7ff700 1 -- 172.21.15.111:0/1993544488 shutdown_connections 2023-03-29T10:59:07.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.529+0000 7fa0dc7ff700 1 --2- 172.21.15.111:0/1993544488 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fa0b8069de0 0x7fa0b806c290 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.529+0000 7fa0dc7ff700 1 --2- 172.21.15.111:0/1993544488 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fa0d4158da0 0x7fa0d4153a90 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.529+0000 7fa0dc7ff700 1 --2- 172.21.15.111:0/1993544488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0d406cc00 0x7fa0d40ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.529+0000 7fa0dc7ff700 1 --2- 172.21.15.111:0/1993544488 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fa0d41589d0 0x7fa0d415bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.529+0000 7fa0dc7ff700 1 -- 172.21.15.111:0/1993544488 >> 172.21.15.111:0/1993544488 conn(0x7fa0d40cf720 msgr2=0x7fa0d41521b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:07.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.529+0000 7fa0dc7ff700 1 -- 172.21.15.111:0/1993544488 shutdown_connections 2023-03-29T10:59:07.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.529+0000 7fa0dc7ff700 1 -- 172.21.15.111:0/1993544488 wait complete. 2023-03-29T10:59:07.546 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-03-29T10:59:07.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.730+0000 7f9472fff700 1 Processor -- start 2023-03-29T10:59:07.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.730+0000 7f9472fff700 1 -- start start 2023-03-29T10:59:07.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.731+0000 7f9472fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f946c15c590 0x7f946c160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:07.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.731+0000 7f9472fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f946c0cf010 0x7f946c0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:07.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.731+0000 7f9472fff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946c0cd520 con 0x7f946c0cfd80 2023-03-29T10:59:07.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.731+0000 7f9472fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946c0d2640 con 0x7f946c15c590 2023-03-29T10:59:07.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.731+0000 7f9472fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c1610d0 con 0x7f946c0cf010 2023-03-29T10:59:07.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.732+0000 7f947159c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f946c0cfd80 0x7f946c0d0aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:41046/0 (socket says 172.21.15.111:41046) 2023-03-29T10:59:07.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.732+0000 7f947159c700 1 -- 172.21.15.111:0/1206556665 learned_addr learned my addr 172.21.15.111:0/1206556665 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:59:07.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.732+0000 7f9470d9b700 1 --2- 172.21.15.111:0/1206556665 >> v2:172.21.15.119:3300/0 conn(0x7f946c15c590 0x7f946c160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:07.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.732+0000 7f946bfff700 1 --2- 172.21.15.111:0/1206556665 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f946c0cf010 0x7f946c0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:07.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.732+0000 7f946b7fe700 1 -- 172.21.15.111:0/1206556665 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2860403692 0 0) 0x7f946c0cd520 con 0x7f946c0cfd80 2023-03-29T10:59:07.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.732+0000 7f946b7fe700 1 -- 172.21.15.111:0/1206556665 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f945c0035b0 con 0x7f946c0cfd80 2023-03-29T10:59:07.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.732+0000 7f9470d9b700 1 -- 172.21.15.111:0/1206556665 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f946c0cf010 msgr2=0x7f946c0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:07.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.732+0000 7f9470d9b700 1 --2- 172.21.15.111:0/1206556665 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f946c0cf010 0x7f946c0cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.732+0000 7f9470d9b700 1 -- 172.21.15.111:0/1206556665 >> v1:172.21.15.111:6789/0 conn(0x7f946c0cfd80 legacy=0x7f946c0d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:07.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.732+0000 7f9470d9b700 1 -- 172.21.15.111:0/1206556665 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c0d0200 con 0x7f946c15c590 2023-03-29T10:59:07.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.733+0000 7f9470d9b700 1 --2- 172.21.15.111:0/1206556665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f946c15c590 0x7f946c160b90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9460009840 tx=0x7f9460009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47130abfca02695b server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:07.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.733+0000 7f946b7fe700 1 -- 172.21.15.111:0/1206556665 <== mon.1 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9460011830 con 0x7f946c15c590 2023-03-29T10:59:07.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.733+0000 7f946b7fe700 1 -- 172.21.15.111:0/1206556665 <== mon.1 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9460011990 con 0x7f946c15c590 2023-03-29T10:59:07.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.733+0000 7f946b7fe700 1 -- 172.21.15.111:0/1206556665 <== mon.1 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9460011d20 con 0x7f946c15c590 2023-03-29T10:59:07.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.733+0000 7f9472fff700 1 -- 172.21.15.111:0/1206556665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f946c15c590 msgr2=0x7f946c160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:07.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.733+0000 7f9472fff700 1 --2- 172.21.15.111:0/1206556665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f946c15c590 0x7f946c160b90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9460009840 tx=0x7f9460009b50 comp rx=0 tx=0).stop 2023-03-29T10:59:07.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.733+0000 7f9472fff700 1 -- 172.21.15.111:0/1206556665 shutdown_connections 2023-03-29T10:59:07.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.733+0000 7f9472fff700 1 --2- 172.21.15.111:0/1206556665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f946c15c590 0x7f946c160b90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.733+0000 7f9472fff700 1 --2- 172.21.15.111:0/1206556665 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f946c0cf010 0x7f946c0cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.733+0000 7f9472fff700 1 -- 172.21.15.111:0/1206556665 >> 172.21.15.111:0/1206556665 conn(0x7f946c071b10 msgr2=0x7f946c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:07.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.734+0000 7f9472fff700 1 -- 172.21.15.111:0/1206556665 shutdown_connections 2023-03-29T10:59:07.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.734+0000 7f9472fff700 1 -- 172.21.15.111:0/1206556665 wait complete. 2023-03-29T10:59:07.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.735+0000 7f9472fff700 1 Processor -- start 2023-03-29T10:59:07.736 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.735+0000 7f9472fff700 1 -- start start 2023-03-29T10:59:07.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.736+0000 7f9472fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f946c0cf010 0x7f946c0c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:07.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.736+0000 7f9472fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f946c0cfd80 0x7f946c1f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:07.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.736+0000 7f9472fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f946c15c590 0x7f946c1f58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:07.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.736+0000 7f9472fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0b5d00 con 0x7f946c15c590 2023-03-29T10:59:07.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.736+0000 7f9472fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c0b5fe0 con 0x7f946c0cfd80 2023-03-29T10:59:07.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.736+0000 7f9472fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c1f6290 con 0x7f946c0cf010 2023-03-29T10:59:07.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.737+0000 7f947159c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f946c15c590 0x7f946c1f58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:07.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.737+0000 7f947159c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f946c15c590 0x7f946c1f58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:52204/0 (socket says 172.21.15.111:52204) 2023-03-29T10:59:07.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.737+0000 7f947159c700 1 -- 172.21.15.111:0/3969160630 learned_addr learned my addr 172.21.15.111:0/3969160630 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:07.738 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.737+0000 7f9470d9b700 1 --2- 172.21.15.111:0/3969160630 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f946c0cf010 0x7f946c0c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:07.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.737+0000 7f946bfff700 1 --2- 172.21.15.111:0/3969160630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f946c0cfd80 0x7f946c1f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:07.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.737+0000 7f947159c700 1 -- 172.21.15.111:0/3969160630 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f946c0cf010 msgr2=0x7f946c0c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:07.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.737+0000 7f947159c700 1 --2- 172.21.15.111:0/3969160630 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f946c0cf010 0x7f946c0c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.737+0000 7f947159c700 1 -- 172.21.15.111:0/3969160630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f946c0cfd80 msgr2=0x7f946c1f5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:07.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.737+0000 7f947159c700 1 --2- 172.21.15.111:0/3969160630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f946c0cfd80 0x7f946c1f5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:07.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.737+0000 7f947159c700 1 -- 172.21.15.111:0/3969160630 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94600094a0 con 0x7f946c15c590 2023-03-29T10:59:07.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.737+0000 7f946bfff700 1 --2- 172.21.15.111:0/3969160630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f946c0cfd80 0x7f946c1f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:59:07.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.737+0000 7f947159c700 1 --2- 172.21.15.111:0/3969160630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f946c15c590 0x7f946c1f58c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f946400cb40 tx=0x7f946400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:07.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.737+0000 7f94697fa700 1 -- 172.21.15.111:0/3969160630 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946400ab50 con 0x7f946c15c590 2023-03-29T10:59:07.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.737+0000 7f94697fa700 1 -- 172.21.15.111:0/3969160630 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946400acb0 con 0x7f946c15c590 2023-03-29T10:59:07.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.737+0000 7f9472fff700 1 -- 172.21.15.111:0/3969160630 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946c0cf560 con 0x7f946c15c590 2023-03-29T10:59:07.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.737+0000 7f94697fa700 1 -- 172.21.15.111:0/3969160630 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464012620 con 0x7f946c15c590 2023-03-29T10:59:07.739 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.738+0000 7f9472fff700 1 -- 172.21.15.111:0/3969160630 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c0b23c0 con 0x7f946c15c590 2023-03-29T10:59:07.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.739+0000 7f9472fff700 1 -- 172.21.15.111:0/3969160630 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946c0cd770 con 0x7f946c15c590 2023-03-29T10:59:07.740 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.740+0000 7f94697fa700 1 -- 172.21.15.111:0/3969160630 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f946401b3b0 con 0x7f946c15c590 2023-03-29T10:59:07.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.740+0000 7f94697fa700 1 --2- 172.21.15.111:0/3969160630 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9450069de0 0x7f945006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:07.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.741+0000 7f94697fa700 1 -- 172.21.15.111:0/3969160630 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f9464087c70 con 0x7f946c15c590 2023-03-29T10:59:07.741 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.741+0000 7f9470d9b700 1 --2- 172.21.15.111:0/3969160630 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9450069de0 0x7f945006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:07.742 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.741+0000 7f9470d9b700 1 --2- 172.21.15.111:0/3969160630 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9450069de0 0x7f945006c290 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9460000c00 tx=0x7f946001c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:07.745 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:07.744+0000 7f94697fa700 1 -- 172.21.15.111:0/3969160630 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f9464053630 con 0x7f946c15c590 2023-03-29T10:59:08.019 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.018+0000 7f9472fff700 1 -- 172.21.15.111:0/3969160630 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f946c0b6150 con 0x7f946c15c590 2023-03-29T10:59:08.023 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.023+0000 7f94697fa700 1 -- 172.21.15.111:0/3969160630 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f94640170a0 con 0x7f946c15c590 2023-03-29T10:59:08.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.025+0000 7f9472fff700 1 -- 172.21.15.111:0/3969160630 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9450069de0 msgr2=0x7f945006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:08.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.025+0000 7f9472fff700 1 --2- 172.21.15.111:0/3969160630 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9450069de0 0x7f945006c290 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9460000c00 tx=0x7f946001c040 comp rx=0 tx=0).stop 2023-03-29T10:59:08.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.025+0000 7f9472fff700 1 -- 172.21.15.111:0/3969160630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f946c15c590 msgr2=0x7f946c1f58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:08.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.025+0000 7f9472fff700 1 --2- 172.21.15.111:0/3969160630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f946c15c590 0x7f946c1f58c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f946400cb40 tx=0x7f946400cf00 comp rx=0 tx=0).stop 2023-03-29T10:59:08.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.025+0000 7f9472fff700 1 -- 172.21.15.111:0/3969160630 shutdown_connections 2023-03-29T10:59:08.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.025+0000 7f9472fff700 1 --2- 172.21.15.111:0/3969160630 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f9450069de0 0x7f945006c290 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.025+0000 7f9472fff700 1 --2- 172.21.15.111:0/3969160630 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f946c15c590 0x7f946c1f58c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.025+0000 7f9472fff700 1 --2- 172.21.15.111:0/3969160630 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f946c0cf010 0x7f946c0c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.025+0000 7f9472fff700 1 --2- 172.21.15.111:0/3969160630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f946c0cfd80 0x7f946c1f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.025+0000 7f9472fff700 1 -- 172.21.15.111:0/3969160630 >> 172.21.15.111:0/3969160630 conn(0x7f946c071b10 msgr2=0x7f946c0d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:08.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.025+0000 7f9472fff700 1 -- 172.21.15.111:0/3969160630 shutdown_connections 2023-03-29T10:59:08.026 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.025+0000 7f9472fff700 1 -- 172.21.15.111:0/3969160630 wait complete. 2023-03-29T10:59:08.043 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-03-29T10:59:08.230 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.229+0000 7f64fbdff700 1 Processor -- start 2023-03-29T10:59:08.230 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.229+0000 7f64fbdff700 1 -- start start 2023-03-29T10:59:08.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.230+0000 7f64fbdff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f64f40cd790 0x7f64f40ca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:08.231 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.231+0000 7f64fbdff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f64f40cdb60 0x7f64f40b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:08.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.231+0000 7f64fbdff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64f40b1b80 0x7f64f406b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:08.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.231+0000 7f64fbdff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f406a5f0 con 0x7f64f40cdb60 2023-03-29T10:59:08.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.231+0000 7f64fbdff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f406a840 con 0x7f64f40cd790 2023-03-29T10:59:08.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.231+0000 7f64fbdff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f406ab20 con 0x7f64f40b1b80 2023-03-29T10:59:08.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.231+0000 7f64f939a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f64f40cdb60 0x7f64f40b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:08.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.231+0000 7f64f939a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f64f40cdb60 0x7f64f40b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40122/0 (socket says 172.21.15.111:40122) 2023-03-29T10:59:08.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.231+0000 7f64f939a700 1 -- 172.21.15.111:0/2193584184 learned_addr learned my addr 172.21.15.111:0/2193584184 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:08.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.231+0000 7f64fa39c700 1 --2- 172.21.15.111:0/2193584184 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64f40b1b80 0x7f64f406b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:08.232 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.231+0000 7f64f9b9b700 1 --2- 172.21.15.111:0/2193584184 >> v2:172.21.15.119:3300/0 conn(0x7f64f40cd790 0x7f64f40ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:08.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.231+0000 7f64f939a700 1 -- 172.21.15.111:0/2193584184 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64f40b1b80 msgr2=0x7f64f406b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:08.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.231+0000 7f64f939a700 1 --2- 172.21.15.111:0/2193584184 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64f40b1b80 0x7f64f406b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.231+0000 7f64f939a700 1 -- 172.21.15.111:0/2193584184 >> v2:172.21.15.119:3300/0 conn(0x7f64f40cd790 msgr2=0x7f64f40ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:08.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.231+0000 7f64f939a700 1 --2- 172.21.15.111:0/2193584184 >> v2:172.21.15.119:3300/0 conn(0x7f64f40cd790 0x7f64f40ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.231+0000 7f64f939a700 1 -- 172.21.15.111:0/2193584184 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f406ae00 con 0x7f64f40cdb60 2023-03-29T10:59:08.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.232+0000 7f64f939a700 1 --2- 172.21.15.111:0/2193584184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64f40cdb60 0x7f64f40b1640 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f64e40098f0 tx=0x7f64e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0798379916ae603 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:08.233 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.232+0000 7f64f8b99700 1 -- 172.21.15.111:0/2193584184 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4011830 con 0x7f64f40cdb60 2023-03-29T10:59:08.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.232+0000 7f64f8b99700 1 -- 172.21.15.111:0/2193584184 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e4011990 con 0x7f64f40cdb60 2023-03-29T10:59:08.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.232+0000 7f64f8b99700 1 -- 172.21.15.111:0/2193584184 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4011d20 con 0x7f64f40cdb60 2023-03-29T10:59:08.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.232+0000 7f64fbdff700 1 -- 172.21.15.111:0/2193584184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64f40cdb60 msgr2=0x7f64f40b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:08.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.232+0000 7f64fbdff700 1 --2- 172.21.15.111:0/2193584184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64f40cdb60 0x7f64f40b1640 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f64e40098f0 tx=0x7f64e4009c00 comp rx=0 tx=0).stop 2023-03-29T10:59:08.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.233+0000 7f64fbdff700 1 -- 172.21.15.111:0/2193584184 shutdown_connections 2023-03-29T10:59:08.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.233+0000 7f64fbdff700 1 --2- 172.21.15.111:0/2193584184 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64f40b1b80 0x7f64f406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.233+0000 7f64fbdff700 1 --2- 172.21.15.111:0/2193584184 >> v2:172.21.15.119:3300/0 conn(0x7f64f40cd790 0x7f64f40ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.233+0000 7f64fbdff700 1 --2- 172.21.15.111:0/2193584184 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64f40cdb60 0x7f64f40b1640 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.233+0000 7f64fbdff700 1 -- 172.21.15.111:0/2193584184 >> 172.21.15.111:0/2193584184 conn(0x7f64f406e980 msgr2=0x7f64f40cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:08.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.233+0000 7f64fbdff700 1 -- 172.21.15.111:0/2193584184 shutdown_connections 2023-03-29T10:59:08.234 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.233+0000 7f64fbdff700 1 -- 172.21.15.111:0/2193584184 wait complete. 2023-03-29T10:59:08.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.234+0000 7f64fbdff700 1 Processor -- start 2023-03-29T10:59:08.235 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.234+0000 7f64fbdff700 1 -- start start 2023-03-29T10:59:08.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.235+0000 7f64fbdff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64f40b1b80 0x7f64f415e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:08.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.235+0000 7f64fbdff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64f40cd790 0x7f64f4154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:08.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.235+0000 7f64fbdff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64f40cdb60 0x7f64f415aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:08.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.235+0000 7f64fbdff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f415bf40 con 0x7f64f40cd790 2023-03-29T10:59:08.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.235+0000 7f64fbdff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f415f340 con 0x7f64f40cdb60 2023-03-29T10:59:08.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.235+0000 7f64fbdff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f415e730 con 0x7f64f40b1b80 2023-03-29T10:59:08.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.235+0000 7f64f939a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64f40cd790 0x7f64f4154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:08.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.236+0000 7f64f939a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64f40cd790 0x7f64f4154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40134/0 (socket says 172.21.15.111:40134) 2023-03-29T10:59:08.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.236+0000 7f64f939a700 1 -- 172.21.15.111:0/3965184864 learned_addr learned my addr 172.21.15.111:0/3965184864 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:08.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.236+0000 7f64f9b9b700 1 --2- 172.21.15.111:0/3965184864 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64f40b1b80 0x7f64f415e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:08.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.236+0000 7f64fa39c700 1 --2- 172.21.15.111:0/3965184864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64f40cdb60 0x7f64f415aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:08.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.236+0000 7f64f939a700 1 -- 172.21.15.111:0/3965184864 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64f40b1b80 msgr2=0x7f64f415e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:08.237 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.236+0000 7f64f939a700 1 --2- 172.21.15.111:0/3965184864 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64f40b1b80 0x7f64f415e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.236+0000 7f64f939a700 1 -- 172.21.15.111:0/3965184864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64f40cdb60 msgr2=0x7f64f415aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:08.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.236+0000 7f64f939a700 1 --2- 172.21.15.111:0/3965184864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64f40cdb60 0x7f64f415aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.236+0000 7f64f939a700 1 -- 172.21.15.111:0/3965184864 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e4009550 con 0x7f64f40cd790 2023-03-29T10:59:08.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.236+0000 7f64f9b9b700 1 --2- 172.21.15.111:0/3965184864 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64f40b1b80 0x7f64f415e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:59:08.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.236+0000 7f64fa39c700 1 --2- 172.21.15.111:0/3965184864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64f40cdb60 0x7f64f415aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:59:08.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.236+0000 7f64f939a700 1 --2- 172.21.15.111:0/3965184864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64f40cd790 0x7f64f4154660 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009c40 tx=0x7f64e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:08.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.236+0000 7f64ea7fc700 1 -- 172.21.15.111:0/3965184864 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4004510 con 0x7f64f40cd790 2023-03-29T10:59:08.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.236+0000 7f64ea7fc700 1 -- 172.21.15.111:0/3965184864 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e40036a0 con 0x7f64f40cd790 2023-03-29T10:59:08.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.236+0000 7f64fbdff700 1 -- 172.21.15.111:0/3965184864 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f40b2000 con 0x7f64f40cd790 2023-03-29T10:59:08.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.237+0000 7f64ea7fc700 1 -- 172.21.15.111:0/3965184864 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4003a10 con 0x7f64f40cd790 2023-03-29T10:59:08.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.237+0000 7f64fbdff700 1 -- 172.21.15.111:0/3965184864 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f406b0e0 con 0x7f64f40cd790 2023-03-29T10:59:08.238 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.238+0000 7f64fbdff700 1 -- 172.21.15.111:0/3965184864 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f40ce000 con 0x7f64f40cd790 2023-03-29T10:59:08.239 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.238+0000 7f64ea7fc700 1 -- 172.21.15.111:0/3965184864 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f64e4003b70 con 0x7f64f40cd790 2023-03-29T10:59:08.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.239+0000 7f64ea7fc700 1 --2- 172.21.15.111:0/3965184864 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f64d8069de0 0x7f64d806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:08.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.239+0000 7f64ea7fc700 1 -- 172.21.15.111:0/3965184864 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f64e4089430 con 0x7f64f40cd790 2023-03-29T10:59:08.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.239+0000 7f64f9b9b700 1 --2- 172.21.15.111:0/3965184864 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f64d8069de0 0x7f64d806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:08.240 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.240+0000 7f64f9b9b700 1 --2- 172.21.15.111:0/3965184864 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f64d8069de0 0x7f64d806c290 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f64f40aea20 tx=0x7f64f0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:08.244 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.244+0000 7f64ea7fc700 1 -- 172.21.15.111:0/3965184864 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f64e4054ee0 con 0x7f64f40cd790 2023-03-29T10:59:08.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.515+0000 7f64fbdff700 1 -- 172.21.15.111:0/3965184864 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f64f415f6d0 con 0x7f64f40cd790 2023-03-29T10:59:08.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.519+0000 7f64ea7fc700 1 -- 172.21.15.111:0/3965184864 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f64e402ee60 con 0x7f64f40cd790 2023-03-29T10:59:08.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.521+0000 7f64fbdff700 1 -- 172.21.15.111:0/3965184864 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f64d8069de0 msgr2=0x7f64d806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:08.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.521+0000 7f64fbdff700 1 --2- 172.21.15.111:0/3965184864 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f64d8069de0 0x7f64d806c290 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f64f40aea20 tx=0x7f64f0008040 comp rx=0 tx=0).stop 2023-03-29T10:59:08.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.521+0000 7f64fbdff700 1 -- 172.21.15.111:0/3965184864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64f40cd790 msgr2=0x7f64f4154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:08.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.521+0000 7f64fbdff700 1 --2- 172.21.15.111:0/3965184864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64f40cd790 0x7f64f4154660 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009c40 tx=0x7f64e4004750 comp rx=0 tx=0).stop 2023-03-29T10:59:08.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.521+0000 7f64fbdff700 1 -- 172.21.15.111:0/3965184864 shutdown_connections 2023-03-29T10:59:08.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.521+0000 7f64fbdff700 1 --2- 172.21.15.111:0/3965184864 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f64d8069de0 0x7f64d806c290 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.521+0000 7f64fbdff700 1 --2- 172.21.15.111:0/3965184864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f64f40cdb60 0x7f64f415aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.521+0000 7f64fbdff700 1 --2- 172.21.15.111:0/3965184864 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f64f40b1b80 0x7f64f415e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.521+0000 7f64fbdff700 1 --2- 172.21.15.111:0/3965184864 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f64f40cd790 0x7f64f4154660 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.521+0000 7f64fbdff700 1 -- 172.21.15.111:0/3965184864 >> 172.21.15.111:0/3965184864 conn(0x7f64f406e980 msgr2=0x7f64f40ca7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:08.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.522+0000 7f64fbdff700 1 -- 172.21.15.111:0/3965184864 shutdown_connections 2023-03-29T10:59:08.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.522+0000 7f64fbdff700 1 -- 172.21.15.111:0/3965184864 wait complete. 2023-03-29T10:59:08.539 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:59:08.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.723+0000 7f22dd9ff700 1 Processor -- start 2023-03-29T10:59:08.724 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.723+0000 7f22dd9ff700 1 -- start start 2023-03-29T10:59:08.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.724+0000 7f22dd9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f22d80b2440 0x7f22d80cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:08.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.724+0000 7f22dd9ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f22d80d2ea0 0x7f22d80d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:08.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.724+0000 7f22dd9ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d80d3860 con 0x7f22d815db70 2023-03-29T10:59:08.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.724+0000 7f22dd9ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d80b2810 con 0x7f22d80d2ea0 2023-03-29T10:59:08.725 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.724+0000 7f22dd9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d815ce30 con 0x7f22d80b2440 2023-03-29T10:59:08.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.725+0000 7f22d77fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f22d80d2ea0 0x7f22d80d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:08.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.725+0000 7f22d67fc700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f22d815db70 0x7f22d8160e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38154/0 (socket says 172.21.15.111:38154) 2023-03-29T10:59:08.727 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.725+0000 7f22d67fc700 1 -- 172.21.15.111:0/1636186011 learned_addr learned my addr 172.21.15.111:0/1636186011 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:59:08.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.725+0000 7f22d6ffd700 1 --2- 172.21.15.111:0/1636186011 >> v2:172.21.15.119:3300/0 conn(0x7f22d80b2440 0x7f22d80cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:08.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.725+0000 7f22d77fe700 1 -- 172.21.15.111:0/1636186011 >> v2:172.21.15.119:3300/0 conn(0x7f22d80b2440 msgr2=0x7f22d80cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:08.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.725+0000 7f22d77fe700 1 --2- 172.21.15.111:0/1636186011 >> v2:172.21.15.119:3300/0 conn(0x7f22d80b2440 0x7f22d80cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.725+0000 7f22d77fe700 1 -- 172.21.15.111:0/1636186011 >> v1:172.21.15.111:6789/0 conn(0x7f22d815db70 legacy=0x7f22d8160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:08.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.725+0000 7f22d77fe700 1 -- 172.21.15.111:0/1636186011 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d80cdee0 con 0x7f22d80d2ea0 2023-03-29T10:59:08.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.725+0000 7f22d6ffd700 1 --2- 172.21.15.111:0/1636186011 >> v2:172.21.15.119:3300/0 conn(0x7f22d80b2440 0x7f22d80cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:59:08.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.725+0000 7f22d5ffb700 1 -- 172.21.15.111:0/1636186011 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619401296 0 0) 0x7f22d80d3860 con 0x7f22d815db70 2023-03-29T10:59:08.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.725+0000 7f22d77fe700 1 --2- 172.21.15.111:0/1636186011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80d2ea0 0x7f22d80d3320 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f22d80d3a60 tx=0x7f22d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d166fd99bb8e83cd server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:08.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.725+0000 7f22d5ffb700 1 -- 172.21.15.111:0/1636186011 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d000e070 con 0x7f22d80d2ea0 2023-03-29T10:59:08.728 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.726+0000 7f22d5ffb700 1 -- 172.21.15.111:0/1636186011 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d000bbd0 con 0x7f22d80d2ea0 2023-03-29T10:59:08.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.726+0000 7f22d5ffb700 1 -- 172.21.15.111:0/1636186011 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0012430 con 0x7f22d80d2ea0 2023-03-29T10:59:08.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.726+0000 7f22dd9ff700 1 -- 172.21.15.111:0/1636186011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80d2ea0 msgr2=0x7f22d80d3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:08.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.726+0000 7f22dd9ff700 1 --2- 172.21.15.111:0/1636186011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80d2ea0 0x7f22d80d3320 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f22d80d3a60 tx=0x7f22d000b330 comp rx=0 tx=0).stop 2023-03-29T10:59:08.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.726+0000 7f22dd9ff700 1 -- 172.21.15.111:0/1636186011 shutdown_connections 2023-03-29T10:59:08.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.726+0000 7f22dd9ff700 1 --2- 172.21.15.111:0/1636186011 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80d2ea0 0x7f22d80d3320 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.726+0000 7f22dd9ff700 1 --2- 172.21.15.111:0/1636186011 >> v2:172.21.15.119:3300/0 conn(0x7f22d80b2440 0x7f22d80cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.726+0000 7f22dd9ff700 1 -- 172.21.15.111:0/1636186011 >> 172.21.15.111:0/1636186011 conn(0x7f22d8071b10 msgr2=0x7f22d8071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:08.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.726+0000 7f22dd9ff700 1 -- 172.21.15.111:0/1636186011 shutdown_connections 2023-03-29T10:59:08.729 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.726+0000 7f22dd9ff700 1 -- 172.21.15.111:0/1636186011 wait complete. 2023-03-29T10:59:08.731 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.728+0000 7f22dd9ff700 1 Processor -- start 2023-03-29T10:59:08.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.728+0000 7f22dd9ff700 1 -- start start 2023-03-29T10:59:08.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.729+0000 7f22dd9ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22d80b2440 0x7f22d81514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:08.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.729+0000 7f22dd9ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80d2ea0 0x7f22d80cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:08.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.729+0000 7f22dd9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d815db70 0x7f22d80cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:08.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.729+0000 7f22dd9ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80cb460 con 0x7f22d80d2ea0 2023-03-29T10:59:08.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.729+0000 7f22dd9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80cb740 con 0x7f22d815db70 2023-03-29T10:59:08.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.729+0000 7f22dd9ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80cba20 con 0x7f22d80b2440 2023-03-29T10:59:08.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.729+0000 7f22d67fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80d2ea0 0x7f22d80cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:08.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.729+0000 7f22d67fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80d2ea0 0x7f22d80cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40154/0 (socket says 172.21.15.111:40154) 2023-03-29T10:59:08.732 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.729+0000 7f22d67fc700 1 -- 172.21.15.111:0/1601690404 learned_addr learned my addr 172.21.15.111:0/1601690404 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:08.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.729+0000 7f22d6ffd700 1 --2- 172.21.15.111:0/1601690404 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22d80b2440 0x7f22d81514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:08.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.729+0000 7f22d77fe700 1 --2- 172.21.15.111:0/1601690404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d815db70 0x7f22d80cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:08.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.729+0000 7f22d67fc700 1 -- 172.21.15.111:0/1601690404 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22d80b2440 msgr2=0x7f22d81514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:08.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.729+0000 7f22d67fc700 1 --2- 172.21.15.111:0/1601690404 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22d80b2440 0x7f22d81514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.729+0000 7f22d67fc700 1 -- 172.21.15.111:0/1601690404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d815db70 msgr2=0x7f22d80cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:08.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.729+0000 7f22d67fc700 1 --2- 172.21.15.111:0/1601690404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d815db70 0x7f22d80cfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.729+0000 7f22d67fc700 1 -- 172.21.15.111:0/1601690404 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d0009a70 con 0x7f22d80d2ea0 2023-03-29T10:59:08.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.729+0000 7f22d77fe700 1 --2- 172.21.15.111:0/1601690404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d815db70 0x7f22d80cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:59:08.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.730+0000 7f22d67fc700 1 --2- 172.21.15.111:0/1601690404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80d2ea0 0x7f22d80cc900 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00c550 tx=0x7f22cc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:08.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.730+0000 7f22bffff700 1 -- 172.21.15.111:0/1601690404 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc004c80 con 0x7f22d80d2ea0 2023-03-29T10:59:08.733 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.730+0000 7f22bffff700 1 -- 172.21.15.111:0/1601690404 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22cc004e00 con 0x7f22d80d2ea0 2023-03-29T10:59:08.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.730+0000 7f22bffff700 1 -- 172.21.15.111:0/1601690404 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc007310 con 0x7f22d80d2ea0 2023-03-29T10:59:08.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.730+0000 7f22dd9ff700 1 -- 172.21.15.111:0/1601690404 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d815d090 con 0x7f22d80d2ea0 2023-03-29T10:59:08.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.730+0000 7f22dd9ff700 1 -- 172.21.15.111:0/1601690404 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d80cdc00 con 0x7f22d80d2ea0 2023-03-29T10:59:08.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.731+0000 7f22dd9ff700 1 -- 172.21.15.111:0/1601690404 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d815d370 con 0x7f22d80d2ea0 2023-03-29T10:59:08.734 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.732+0000 7f22bffff700 1 -- 172.21.15.111:0/1601690404 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f22cc007470 con 0x7f22d80d2ea0 2023-03-29T10:59:08.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.733+0000 7f22bffff700 1 --2- 172.21.15.111:0/1601690404 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f22c4069e10 0x7f22c406c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:08.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.733+0000 7f22bffff700 1 -- 172.21.15.111:0/1601690404 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f22cc087a60 con 0x7f22d80d2ea0 2023-03-29T10:59:08.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.733+0000 7f22d6ffd700 1 --2- 172.21.15.111:0/1601690404 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f22c4069e10 0x7f22c406c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:08.735 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.734+0000 7f22d6ffd700 1 --2- 172.21.15.111:0/1601690404 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f22c4069e10 0x7f22c406c2c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f22c80099d0 tx=0x7f22c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:08.737 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.737+0000 7f22bffff700 1 -- 172.21.15.111:0/1601690404 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f22cc053480 con 0x7f22d80d2ea0 2023-03-29T10:59:08.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.934+0000 7f22dd9ff700 1 -- 172.21.15.111:0/1601690404 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22d8074430 con 0x7f22d80d2ea0 2023-03-29T10:59:08.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.935+0000 7f22bffff700 1 -- 172.21.15.111:0/1601690404 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+14520 (secure 0 0 0) 0x7f22cc056aa0 con 0x7f22d80d2ea0 2023-03-29T10:59:08.935 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:59:08.935 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":39,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:59:05.055565+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:59:08.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.937+0000 7f22dd9ff700 1 -- 172.21.15.111:0/1601690404 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f22c4069e10 msgr2=0x7f22c406c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:08.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.937+0000 7f22dd9ff700 1 --2- 172.21.15.111:0/1601690404 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f22c4069e10 0x7f22c406c2c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f22c80099d0 tx=0x7f22c8008040 comp rx=0 tx=0).stop 2023-03-29T10:59:08.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.937+0000 7f22dd9ff700 1 -- 172.21.15.111:0/1601690404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80d2ea0 msgr2=0x7f22d80cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:08.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.937+0000 7f22dd9ff700 1 --2- 172.21.15.111:0/1601690404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80d2ea0 0x7f22d80cc900 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00c550 tx=0x7f22cc00c860 comp rx=0 tx=0).stop 2023-03-29T10:59:08.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.937+0000 7f22dd9ff700 1 -- 172.21.15.111:0/1601690404 shutdown_connections 2023-03-29T10:59:08.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.937+0000 7f22dd9ff700 1 --2- 172.21.15.111:0/1601690404 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f22c4069e10 0x7f22c406c2c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.937+0000 7f22dd9ff700 1 --2- 172.21.15.111:0/1601690404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d815db70 0x7f22d80cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.937+0000 7f22dd9ff700 1 --2- 172.21.15.111:0/1601690404 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f22d80b2440 0x7f22d81514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.937+0000 7f22dd9ff700 1 --2- 172.21.15.111:0/1601690404 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f22d80d2ea0 0x7f22d80cc900 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:08.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.937+0000 7f22dd9ff700 1 -- 172.21.15.111:0/1601690404 >> 172.21.15.111:0/1601690404 conn(0x7f22d8071b10 msgr2=0x7f22d815bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:08.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.937+0000 7f22dd9ff700 1 -- 172.21.15.111:0/1601690404 shutdown_connections 2023-03-29T10:59:08.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:08.937+0000 7f22dd9ff700 1 -- 172.21.15.111:0/1601690404 wait complete. 2023-03-29T10:59:08.955 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-03-29T10:57:16.183979+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-03-29T10:57:21.239809+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-03-29T10:59:08.955 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-03-29T10:59:09.141 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.140+0000 7f4b7b3ff700 1 Processor -- start 2023-03-29T10:59:09.141 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.141+0000 7f4b7b3ff700 1 -- start start 2023-03-29T10:59:09.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.142+0000 7f4b7b3ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4b740ca210 0x7f4b740b1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:09.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.142+0000 7f4b7b3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4b740b1b80 0x7f4b7406b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:09.142 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.142+0000 7f4b7b3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b7406a5f0 con 0x7f4b740cd790 2023-03-29T10:59:09.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.142+0000 7f4b7b3ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b740ca6b0 con 0x7f4b740ca210 2023-03-29T10:59:09.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.142+0000 7f4b7b3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7406a840 con 0x7f4b740b1b80 2023-03-29T10:59:09.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.142+0000 7f4b7899a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4b740ca210 0x7f4b740b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:09.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.142+0000 7f4b7899a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f4b740ca210 0x7f4b740b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40168/0 (socket says 172.21.15.111:40168) 2023-03-29T10:59:09.143 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.142+0000 7f4b7899a700 1 -- 172.21.15.111:0/1350291743 learned_addr learned my addr 172.21.15.111:0/1350291743 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:09.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.142+0000 7f4b7999c700 1 --2- 172.21.15.111:0/1350291743 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4b740b1b80 0x7f4b7406b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:09.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.142+0000 7f4b7899a700 1 -- 172.21.15.111:0/1350291743 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4b740b1b80 msgr2=0x7f4b7406b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:09.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.142+0000 7f4b7899a700 1 --2- 172.21.15.111:0/1350291743 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4b740b1b80 0x7f4b7406b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.142+0000 7f4b7899a700 1 -- 172.21.15.111:0/1350291743 >> v1:172.21.15.119:6789/0 conn(0x7f4b740cd790 legacy=0x7f4b740cdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:09.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.142+0000 7f4b7899a700 1 -- 172.21.15.111:0/1350291743 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7406ab20 con 0x7f4b740ca210 2023-03-29T10:59:09.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.143+0000 7f4b7899a700 1 --2- 172.21.15.111:0/1350291743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b740ca210 0x7f4b740b1640 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009920 tx=0x7f4b64009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd748ce367f2297 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:09.144 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.143+0000 7f4b6bfff700 1 -- 172.21.15.111:0/1350291743 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64004230 con 0x7f4b740ca210 2023-03-29T10:59:09.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.143+0000 7f4b6bfff700 1 -- 172.21.15.111:0/1350291743 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b64004390 con 0x7f4b740ca210 2023-03-29T10:59:09.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.143+0000 7f4b6bfff700 1 -- 172.21.15.111:0/1350291743 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64009e70 con 0x7f4b740ca210 2023-03-29T10:59:09.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.143+0000 7f4b7b3ff700 1 -- 172.21.15.111:0/1350291743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b740ca210 msgr2=0x7f4b740b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:09.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.143+0000 7f4b7b3ff700 1 --2- 172.21.15.111:0/1350291743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b740ca210 0x7f4b740b1640 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009920 tx=0x7f4b64009c30 comp rx=0 tx=0).stop 2023-03-29T10:59:09.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.143+0000 7f4b7b3ff700 1 -- 172.21.15.111:0/1350291743 shutdown_connections 2023-03-29T10:59:09.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.143+0000 7f4b7b3ff700 1 --2- 172.21.15.111:0/1350291743 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4b740b1b80 0x7f4b7406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.143+0000 7f4b7b3ff700 1 --2- 172.21.15.111:0/1350291743 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b740ca210 0x7f4b740b1640 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.143+0000 7f4b7b3ff700 1 -- 172.21.15.111:0/1350291743 >> 172.21.15.111:0/1350291743 conn(0x7f4b7406e980 msgr2=0x7f4b740cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:09.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.143+0000 7f4b7b3ff700 1 -- 172.21.15.111:0/1350291743 shutdown_connections 2023-03-29T10:59:09.145 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.143+0000 7f4b7b3ff700 1 -- 172.21.15.111:0/1350291743 wait complete. 2023-03-29T10:59:09.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.145+0000 7f4b7b3ff700 1 Processor -- start 2023-03-29T10:59:09.146 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.145+0000 7f4b7b3ff700 1 -- start start 2023-03-29T10:59:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.146+0000 7f4b7b3ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b740b1b80 0x7f4b7414b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.146+0000 7f4b7b3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b740ca210 0x7f4b740c36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.146+0000 7f4b7b3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4b740cd790 0x7f4b740c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.146+0000 7f4b7b3ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7415a8e0 con 0x7f4b740b1b80 2023-03-29T10:59:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.146+0000 7f4b7b3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7415a010 con 0x7f4b740ca210 2023-03-29T10:59:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.146+0000 7f4b7b3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7414b5c0 con 0x7f4b740cd790 2023-03-29T10:59:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.146+0000 7f4b7919b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b740b1b80 0x7f4b7414b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:09.147 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.146+0000 7f4b7919b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b740b1b80 0x7f4b7414b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40182/0 (socket says 172.21.15.111:40182) 2023-03-29T10:59:09.148 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.146+0000 7f4b7919b700 1 -- 172.21.15.111:0/1656145158 learned_addr learned my addr 172.21.15.111:0/1656145158 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:09.148 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.147+0000 7f4b7899a700 1 --2- 172.21.15.111:0/1656145158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b740ca210 0x7f4b740c36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:09.148 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.147+0000 7f4b7999c700 1 --2- 172.21.15.111:0/1656145158 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4b740cd790 0x7f4b740c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:09.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.147+0000 7f4b7919b700 1 -- 172.21.15.111:0/1656145158 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4b740cd790 msgr2=0x7f4b740c3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:09.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.147+0000 7f4b7919b700 1 --2- 172.21.15.111:0/1656145158 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4b740cd790 0x7f4b740c3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.147+0000 7f4b7919b700 1 -- 172.21.15.111:0/1656145158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b740ca210 msgr2=0x7f4b740c36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:09.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.147+0000 7f4b7919b700 1 --2- 172.21.15.111:0/1656145158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b740ca210 0x7f4b740c36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.147+0000 7f4b7919b700 1 -- 172.21.15.111:0/1656145158 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b64009550 con 0x7f4b740b1b80 2023-03-29T10:59:09.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.147+0000 7f4b7919b700 1 --2- 172.21.15.111:0/1656145158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b740b1b80 0x7f4b7414b080 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000c4d0 tx=0x7f4b7000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:09.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.147+0000 7f4b69ffb700 1 -- 172.21.15.111:0/1656145158 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b70011790 con 0x7f4b740b1b80 2023-03-29T10:59:09.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.147+0000 7f4b69ffb700 1 -- 172.21.15.111:0/1656145158 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b700118f0 con 0x7f4b740b1b80 2023-03-29T10:59:09.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.147+0000 7f4b69ffb700 1 -- 172.21.15.111:0/1656145158 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b70011ca0 con 0x7f4b740b1b80 2023-03-29T10:59:09.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.147+0000 7f4b7b3ff700 1 -- 172.21.15.111:0/1656145158 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b7406b0e0 con 0x7f4b740b1b80 2023-03-29T10:59:09.149 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.148+0000 7f4b7b3ff700 1 -- 172.21.15.111:0/1656145158 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7406ae00 con 0x7f4b740b1b80 2023-03-29T10:59:09.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.149+0000 7f4b7b3ff700 1 -- 172.21.15.111:0/1656145158 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b740b2000 con 0x7f4b740b1b80 2023-03-29T10:59:09.150 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.150+0000 7f4b69ffb700 1 -- 172.21.15.111:0/1656145158 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f4b7001c460 con 0x7f4b740b1b80 2023-03-29T10:59:09.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.150+0000 7f4b69ffb700 1 --2- 172.21.15.111:0/1656145158 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4b58069de0 0x7f4b5806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:09.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.150+0000 7f4b69ffb700 1 -- 172.21.15.111:0/1656145158 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f4b70087ac0 con 0x7f4b740b1b80 2023-03-29T10:59:09.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.151+0000 7f4b7899a700 1 --2- 172.21.15.111:0/1656145158 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4b58069de0 0x7f4b5806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:09.151 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.151+0000 7f4b7899a700 1 --2- 172.21.15.111:0/1656145158 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4b58069de0 0x7f4b5806c290 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009130 tx=0x7f4b64008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:09.155 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.154+0000 7f4b69ffb700 1 -- 172.21.15.111:0/1656145158 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f4b70053510 con 0x7f4b740b1b80 2023-03-29T10:59:09.349 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.349+0000 7f4b7b3ff700 1 -- 172.21.15.111:0/1656145158 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4b7406c510 con 0x7f4b740b1b80 2023-03-29T10:59:09.350 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.349+0000 7f4b69ffb700 1 -- 172.21.15.111:0/1656145158 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4b70056b30 con 0x7f4b740b1b80 2023-03-29T10:59:09.350 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 1 2023-03-29T10:59:09.352 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.352+0000 7f4b7b3ff700 1 -- 172.21.15.111:0/1656145158 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4b58069de0 msgr2=0x7f4b5806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:09.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.352+0000 7f4b7b3ff700 1 --2- 172.21.15.111:0/1656145158 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4b58069de0 0x7f4b5806c290 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009130 tx=0x7f4b64008030 comp rx=0 tx=0).stop 2023-03-29T10:59:09.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.352+0000 7f4b7b3ff700 1 -- 172.21.15.111:0/1656145158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b740b1b80 msgr2=0x7f4b7414b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:09.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.352+0000 7f4b7b3ff700 1 --2- 172.21.15.111:0/1656145158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b740b1b80 0x7f4b7414b080 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000c4d0 tx=0x7f4b7000c7e0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.352+0000 7f4b7b3ff700 1 -- 172.21.15.111:0/1656145158 shutdown_connections 2023-03-29T10:59:09.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.352+0000 7f4b7b3ff700 1 --2- 172.21.15.111:0/1656145158 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f4b58069de0 0x7f4b5806c290 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.352+0000 7f4b7b3ff700 1 --2- 172.21.15.111:0/1656145158 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f4b740b1b80 0x7f4b7414b080 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.352+0000 7f4b7b3ff700 1 --2- 172.21.15.111:0/1656145158 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f4b740cd790 0x7f4b740c3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.352+0000 7f4b7b3ff700 1 --2- 172.21.15.111:0/1656145158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b740ca210 0x7f4b740c36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.352+0000 7f4b7b3ff700 1 -- 172.21.15.111:0/1656145158 >> 172.21.15.111:0/1656145158 conn(0x7f4b7406e980 msgr2=0x7f4b740cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:09.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.352+0000 7f4b7b3ff700 1 -- 172.21.15.111:0/1656145158 shutdown_connections 2023-03-29T10:59:09.353 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.352+0000 7f4b7b3ff700 1 -- 172.21.15.111:0/1656145158 wait complete. 2023-03-29T10:59:09.369 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-03-29T10:59:09.554 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.554+0000 7f901afff700 1 Processor -- start 2023-03-29T10:59:09.555 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.554+0000 7f901afff700 1 -- start start 2023-03-29T10:59:09.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.555+0000 7f901afff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f90140adf70 0x7f90140cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:09.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.555+0000 7f901afff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f901415cf10 0x7f901406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:09.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.555+0000 7f901afff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90140cc9e0 con 0x7f90140cd760 2023-03-29T10:59:09.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.555+0000 7f901afff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f901415de20 con 0x7f90140adf70 2023-03-29T10:59:09.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.555+0000 7f901afff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901415d860 con 0x7f901415cf10 2023-03-29T10:59:09.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.555+0000 7f9018d9b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f90140cd760 0x7f90140cdb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38162/0 (socket says 172.21.15.111:38162) 2023-03-29T10:59:09.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.556+0000 7f9018d9b700 1 -- 172.21.15.111:0/973277510 learned_addr learned my addr 172.21.15.111:0/973277510 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:59:09.556 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.555+0000 7f9013fff700 1 --2- 172.21.15.111:0/973277510 >> v2:172.21.15.111:3300/0 conn(0x7f90140adf70 0x7f90140cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:09.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.556+0000 7f901959c700 1 --2- 172.21.15.111:0/973277510 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f901415cf10 0x7f901406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:09.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.556+0000 7f90137fe700 1 -- 172.21.15.111:0/973277510 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2444232693 0 0) 0x7f90140cc9e0 con 0x7f90140cd760 2023-03-29T10:59:09.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.556+0000 7f90137fe700 1 -- 172.21.15.111:0/973277510 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90040035b0 con 0x7f90140cd760 2023-03-29T10:59:09.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.556+0000 7f9013fff700 1 -- 172.21.15.111:0/973277510 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f901415cf10 msgr2=0x7f901406c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:09.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.556+0000 7f9013fff700 1 --2- 172.21.15.111:0/973277510 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f901415cf10 0x7f901406c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.556+0000 7f9013fff700 1 -- 172.21.15.111:0/973277510 >> v1:172.21.15.111:6789/0 conn(0x7f90140cd760 legacy=0x7f90140cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:09.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.556+0000 7f9013fff700 1 -- 172.21.15.111:0/973277510 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901415db40 con 0x7f90140adf70 2023-03-29T10:59:09.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.556+0000 7f9013fff700 1 --2- 172.21.15.111:0/973277510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90140adf70 0x7f90140cce70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9000009820 tx=0x7f9000009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87fc3af96151fda server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:09.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.556+0000 7f90137fe700 1 -- 172.21.15.111:0/973277510 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9000005e30 con 0x7f90140adf70 2023-03-29T10:59:09.558 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.557+0000 7f90137fe700 1 -- 172.21.15.111:0/973277510 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9000006030 con 0x7f90140adf70 2023-03-29T10:59:09.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.557+0000 7f90137fe700 1 -- 172.21.15.111:0/973277510 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90000063c0 con 0x7f90140adf70 2023-03-29T10:59:09.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.557+0000 7f901afff700 1 -- 172.21.15.111:0/973277510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90140adf70 msgr2=0x7f90140cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:09.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.557+0000 7f901afff700 1 --2- 172.21.15.111:0/973277510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90140adf70 0x7f90140cce70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9000009820 tx=0x7f9000009b30 comp rx=0 tx=0).stop 2023-03-29T10:59:09.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.557+0000 7f901afff700 1 -- 172.21.15.111:0/973277510 shutdown_connections 2023-03-29T10:59:09.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.557+0000 7f901afff700 1 --2- 172.21.15.111:0/973277510 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f901415cf10 0x7f901406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.557+0000 7f901afff700 1 --2- 172.21.15.111:0/973277510 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90140adf70 0x7f90140cce70 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.557+0000 7f901afff700 1 -- 172.21.15.111:0/973277510 >> 172.21.15.111:0/973277510 conn(0x7f90140cf720 msgr2=0x7f90140cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:09.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.557+0000 7f901afff700 1 -- 172.21.15.111:0/973277510 shutdown_connections 2023-03-29T10:59:09.559 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.557+0000 7f901afff700 1 -- 172.21.15.111:0/973277510 wait complete. 2023-03-29T10:59:09.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.559+0000 7f901afff700 1 Processor -- start 2023-03-29T10:59:09.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.559+0000 7f901afff700 1 -- start start 2023-03-29T10:59:09.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.560+0000 7f901afff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90140adf70 0x7f90140b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:09.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.560+0000 7f901afff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90140cd760 0x7f90140b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:09.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.560+0000 7f901afff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f901415cf10 0x7f901414eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:09.563 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.560+0000 7f901afff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90140c6ef0 con 0x7f90140adf70 2023-03-29T10:59:09.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.560+0000 7f901afff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90140afeb0 con 0x7f90140cd760 2023-03-29T10:59:09.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.560+0000 7f901afff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90140af8f0 con 0x7f901415cf10 2023-03-29T10:59:09.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.560+0000 7f9018d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90140adf70 0x7f90140b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:09.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.560+0000 7f9018d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90140adf70 0x7f90140b18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40212/0 (socket says 172.21.15.111:40212) 2023-03-29T10:59:09.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.560+0000 7f9018d9b700 1 -- 172.21.15.111:0/3759606214 learned_addr learned my addr 172.21.15.111:0/3759606214 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:09.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.560+0000 7f901959c700 1 --2- 172.21.15.111:0/3759606214 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f901415cf10 0x7f901414eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:09.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.560+0000 7f9013fff700 1 --2- 172.21.15.111:0/3759606214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90140cd760 0x7f90140b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:09.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.561+0000 7f9018d9b700 1 -- 172.21.15.111:0/3759606214 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f901415cf10 msgr2=0x7f901414eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:09.564 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.561+0000 7f9018d9b700 1 --2- 172.21.15.111:0/3759606214 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f901415cf10 0x7f901414eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.561+0000 7f9018d9b700 1 -- 172.21.15.111:0/3759606214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90140cd760 msgr2=0x7f90140b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:09.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.561+0000 7f9018d9b700 1 --2- 172.21.15.111:0/3759606214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90140cd760 0x7f90140b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.561+0000 7f9018d9b700 1 -- 172.21.15.111:0/3759606214 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9000009430 con 0x7f90140adf70 2023-03-29T10:59:09.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.561+0000 7f9013fff700 1 --2- 172.21.15.111:0/3759606214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90140cd760 0x7f90140b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:59:09.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.561+0000 7f9018d9b700 1 --2- 172.21.15.111:0/3759606214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90140adf70 0x7f90140b18c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f900800c700 tx=0x7f900800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:09.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.561+0000 7f90117fa700 1 -- 172.21.15.111:0/3759606214 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008005d70 con 0x7f90140adf70 2023-03-29T10:59:09.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.561+0000 7f90117fa700 1 -- 172.21.15.111:0/3759606214 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90080070d0 con 0x7f90140adf70 2023-03-29T10:59:09.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.561+0000 7f90117fa700 1 -- 172.21.15.111:0/3759606214 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008007460 con 0x7f90140adf70 2023-03-29T10:59:09.565 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.561+0000 7f901afff700 1 -- 172.21.15.111:0/3759606214 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9014071600 con 0x7f90140adf70 2023-03-29T10:59:09.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.561+0000 7f901afff700 1 -- 172.21.15.111:0/3759606214 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90140cc670 con 0x7f90140adf70 2023-03-29T10:59:09.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.562+0000 7f901afff700 1 -- 172.21.15.111:0/3759606214 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90140cd3b0 con 0x7f90140adf70 2023-03-29T10:59:09.566 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.563+0000 7f90117fa700 1 -- 172.21.15.111:0/3759606214 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f90080075c0 con 0x7f90140adf70 2023-03-29T10:59:09.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.564+0000 7f90117fa700 1 --2- 172.21.15.111:0/3759606214 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8ff8069de0 0x7f8ff806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:09.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.564+0000 7f90117fa700 1 -- 172.21.15.111:0/3759606214 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f9008087430 con 0x7f90140adf70 2023-03-29T10:59:09.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.564+0000 7f9013fff700 1 --2- 172.21.15.111:0/3759606214 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8ff8069de0 0x7f8ff806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:09.567 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.565+0000 7f9013fff700 1 --2- 172.21.15.111:0/3759606214 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8ff8069de0 0x7f8ff806c290 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9000013040 tx=0x7f9000008070 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:09.568 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.568+0000 7f90117fa700 1 -- 172.21.15.111:0/3759606214 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f9008052e60 con 0x7f90140adf70 2023-03-29T10:59:09.763 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.763+0000 7f901afff700 1 -- 172.21.15.111:0/3759606214 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f90140c8d60 con 0x7f90140adf70 2023-03-29T10:59:09.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.763+0000 7f90117fa700 1 -- 172.21.15.111:0/3759606214 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v39) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9008056480 con 0x7f90140adf70 2023-03-29T10:59:09.764 INFO:teuthology.orchestra.run.smithi111.stdout:pg_num: 8 2023-03-29T10:59:09.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.766+0000 7f901afff700 1 -- 172.21.15.111:0/3759606214 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8ff8069de0 msgr2=0x7f8ff806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:09.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.766+0000 7f901afff700 1 --2- 172.21.15.111:0/3759606214 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8ff8069de0 0x7f8ff806c290 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9000013040 tx=0x7f9000008070 comp rx=0 tx=0).stop 2023-03-29T10:59:09.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.766+0000 7f901afff700 1 -- 172.21.15.111:0/3759606214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90140adf70 msgr2=0x7f90140b18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:09.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.766+0000 7f901afff700 1 --2- 172.21.15.111:0/3759606214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90140adf70 0x7f90140b18c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f900800c700 tx=0x7f900800cac0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.766+0000 7f901afff700 1 -- 172.21.15.111:0/3759606214 shutdown_connections 2023-03-29T10:59:09.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.766+0000 7f901afff700 1 --2- 172.21.15.111:0/3759606214 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f8ff8069de0 0x7f8ff806c290 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.766+0000 7f901afff700 1 --2- 172.21.15.111:0/3759606214 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f90140adf70 0x7f90140b18c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.766+0000 7f901afff700 1 --2- 172.21.15.111:0/3759606214 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f901415cf10 0x7f901414eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.766+0000 7f901afff700 1 --2- 172.21.15.111:0/3759606214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90140cd760 0x7f90140b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.766+0000 7f901afff700 1 -- 172.21.15.111:0/3759606214 >> 172.21.15.111:0/3759606214 conn(0x7f90140cf720 msgr2=0x7f901406a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:09.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.766+0000 7f901afff700 1 -- 172.21.15.111:0/3759606214 shutdown_connections 2023-03-29T10:59:09.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.766+0000 7f901afff700 1 -- 172.21.15.111:0/3759606214 wait complete. 2023-03-29T10:59:09.784 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-03-29T10:59:09.967 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.967+0000 7f5259bff700 1 Processor -- start 2023-03-29T10:59:09.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.967+0000 7f5259bff700 1 -- start start 2023-03-29T10:59:09.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.968+0000 7f5259bff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f525415c950 0x7f52540cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:09.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.968+0000 7f5259bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52540d3a30 0x7f52540b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:09.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.968+0000 7f5259bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52540cdc80 con 0x7f52540cf950 2023-03-29T10:59:09.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.968+0000 7f5259bff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52540cd6c0 con 0x7f525415c950 2023-03-29T10:59:09.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.968+0000 7f5259bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52540cd9a0 con 0x7f52540d3a30 2023-03-29T10:59:09.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.968+0000 7f5252ffd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f525415c950 0x7f52540cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:09.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.969+0000 7f5252ffd700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f525415c950 0x7f52540cf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40218/0 (socket says 172.21.15.111:40218) 2023-03-29T10:59:09.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.969+0000 7f5252ffd700 1 -- 172.21.15.111:0/2887556854 learned_addr learned my addr 172.21.15.111:0/2887556854 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:09.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.969+0000 7f5253fff700 1 --2- 172.21.15.111:0/2887556854 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52540d3a30 0x7f52540b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:09.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.969+0000 7f5252ffd700 1 -- 172.21.15.111:0/2887556854 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52540d3a30 msgr2=0x7f52540b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:09.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.969+0000 7f5252ffd700 1 --2- 172.21.15.111:0/2887556854 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52540d3a30 0x7f52540b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.969+0000 7f5252ffd700 1 -- 172.21.15.111:0/2887556854 >> v1:172.21.15.119:6789/0 conn(0x7f52540cf950 legacy=0x7f52540cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:09.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.969+0000 7f5252ffd700 1 -- 172.21.15.111:0/2887556854 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52540d16c0 con 0x7f525415c950 2023-03-29T10:59:09.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.969+0000 7f5253fff700 1 --2- 172.21.15.111:0/2887556854 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52540d3a30 0x7f52540b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:59:09.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.969+0000 7f5252ffd700 1 --2- 172.21.15.111:0/2887556854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f525415c950 0x7f52540cf0f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f52480098f0 tx=0x7f5248009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11e90dbb6f9eb00 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:09.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.970+0000 7f52527fc700 1 -- 172.21.15.111:0/2887556854 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524800b8a0 con 0x7f525415c950 2023-03-29T10:59:09.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.970+0000 7f52527fc700 1 -- 172.21.15.111:0/2887556854 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524800ba00 con 0x7f525415c950 2023-03-29T10:59:09.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.970+0000 7f52527fc700 1 -- 172.21.15.111:0/2887556854 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524800bd90 con 0x7f525415c950 2023-03-29T10:59:09.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.970+0000 7f5259bff700 1 -- 172.21.15.111:0/2887556854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f525415c950 msgr2=0x7f52540cf0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:09.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.970+0000 7f5259bff700 1 --2- 172.21.15.111:0/2887556854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f525415c950 0x7f52540cf0f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f52480098f0 tx=0x7f5248009c00 comp rx=0 tx=0).stop 2023-03-29T10:59:09.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.970+0000 7f5259bff700 1 -- 172.21.15.111:0/2887556854 shutdown_connections 2023-03-29T10:59:09.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.970+0000 7f5259bff700 1 --2- 172.21.15.111:0/2887556854 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52540d3a30 0x7f52540b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.970+0000 7f5259bff700 1 --2- 172.21.15.111:0/2887556854 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f525415c950 0x7f52540cf0f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.970+0000 7f5259bff700 1 -- 172.21.15.111:0/2887556854 >> 172.21.15.111:0/2887556854 conn(0x7f5254071b10 msgr2=0x7f5254071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:09.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.970+0000 7f5259bff700 1 -- 172.21.15.111:0/2887556854 shutdown_connections 2023-03-29T10:59:09.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.970+0000 7f5259bff700 1 -- 172.21.15.111:0/2887556854 wait complete. 2023-03-29T10:59:09.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.972+0000 7f5259bff700 1 Processor -- start 2023-03-29T10:59:09.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.972+0000 7f5259bff700 1 -- start start 2023-03-29T10:59:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.973+0000 7f5259bff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52540cf950 0x7f525415e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.973+0000 7f5259bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52540d3a30 0x7f52541470b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.973+0000 7f5259bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525415c950 0x7f525415f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.973+0000 7f5259bff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525415e800 con 0x7f52540cf950 2023-03-29T10:59:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.973+0000 7f5259bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525415d220 con 0x7f525415c950 2023-03-29T10:59:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.973+0000 7f5259bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525415d500 con 0x7f52540d3a30 2023-03-29T10:59:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.973+0000 7f52537fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52540cf950 0x7f525415e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.973+0000 7f52537fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52540cf950 0x7f525415e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40224/0 (socket says 172.21.15.111:40224) 2023-03-29T10:59:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.973+0000 7f52537fe700 1 -- 172.21.15.111:0/1003075234 learned_addr learned my addr 172.21.15.111:0/1003075234 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:09.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.973+0000 7f5253fff700 1 --2- 172.21.15.111:0/1003075234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525415c950 0x7f525415f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:09.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.973+0000 7f5252ffd700 1 --2- 172.21.15.111:0/1003075234 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52540d3a30 0x7f52541470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:09.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.974+0000 7f52537fe700 1 -- 172.21.15.111:0/1003075234 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52540d3a30 msgr2=0x7f52541470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:09.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.974+0000 7f52537fe700 1 --2- 172.21.15.111:0/1003075234 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52540d3a30 0x7f52541470b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.974+0000 7f52537fe700 1 -- 172.21.15.111:0/1003075234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525415c950 msgr2=0x7f525415f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:09.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.974+0000 7f52537fe700 1 --2- 172.21.15.111:0/1003075234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525415c950 0x7f525415f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:09.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.974+0000 7f52537fe700 1 -- 172.21.15.111:0/1003075234 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5248009550 con 0x7f52540cf950 2023-03-29T10:59:09.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.974+0000 7f5253fff700 1 --2- 172.21.15.111:0/1003075234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525415c950 0x7f525415f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:59:09.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.974+0000 7f52537fe700 1 --2- 172.21.15.111:0/1003075234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52540cf950 0x7f525415e2c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f524400c4e0 tx=0x7f524400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:09.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.974+0000 7f523bfff700 1 -- 172.21.15.111:0/1003075234 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52440052d0 con 0x7f52540cf950 2023-03-29T10:59:09.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.974+0000 7f523bfff700 1 -- 172.21.15.111:0/1003075234 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5244005430 con 0x7f52540cf950 2023-03-29T10:59:09.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.974+0000 7f5259bff700 1 -- 172.21.15.111:0/1003075234 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52540cf630 con 0x7f52540cf950 2023-03-29T10:59:09.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.974+0000 7f523bfff700 1 -- 172.21.15.111:0/1003075234 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52440038c0 con 0x7f52540cf950 2023-03-29T10:59:09.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.974+0000 7f5259bff700 1 -- 172.21.15.111:0/1003075234 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52540d19a0 con 0x7f52540cf950 2023-03-29T10:59:09.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.975+0000 7f5259bff700 1 -- 172.21.15.111:0/1003075234 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52540b62d0 con 0x7f52540cf950 2023-03-29T10:59:09.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.976+0000 7f523bfff700 1 -- 172.21.15.111:0/1003075234 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f5244003a20 con 0x7f52540cf950 2023-03-29T10:59:09.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.978+0000 7f523bfff700 1 --2- 172.21.15.111:0/1003075234 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5240069de0 0x7f524006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:09.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.978+0000 7f523bfff700 1 -- 172.21.15.111:0/1003075234 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f5244087370 con 0x7f52540cf950 2023-03-29T10:59:09.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.978+0000 7f5252ffd700 1 --2- 172.21.15.111:0/1003075234 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5240069de0 0x7f524006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:09.979 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.978+0000 7f5252ffd700 1 --2- 172.21.15.111:0/1003075234 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5240069de0 0x7f524006c290 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5248014040 tx=0x7f5248008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:09.982 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:09.982+0000 7f523bfff700 1 -- 172.21.15.111:0/1003075234 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f5244052d90 con 0x7f52540cf950 2023-03-29T10:59:10.180 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.180+0000 7f5259bff700 1 -- 172.21.15.111:0/1003075234 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5254074430 con 0x7f52540cf950 2023-03-29T10:59:10.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.180+0000 7f523bfff700 1 -- 172.21.15.111:0/1003075234 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v39) v1 ==== 84+0+126 (secure 0 0 0) 0x7f52440563b0 con 0x7f52540cf950 2023-03-29T10:59:10.181 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:59:10.181 INFO:teuthology.orchestra.run.smithi111.stdout:[ 2023-03-29T10:59:10.181 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:10.181 INFO:teuthology.orchestra.run.smithi111.stdout: "poolnum": 1, 2023-03-29T10:59:10.181 INFO:teuthology.orchestra.run.smithi111.stdout: "poolname": ".mgr" 2023-03-29T10:59:10.181 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:10.181 INFO:teuthology.orchestra.run.smithi111.stdout: { 2023-03-29T10:59:10.181 INFO:teuthology.orchestra.run.smithi111.stdout: "poolnum": 2, 2023-03-29T10:59:10.181 INFO:teuthology.orchestra.run.smithi111.stdout: "poolname": "rbd" 2023-03-29T10:59:10.182 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:10.182 INFO:teuthology.orchestra.run.smithi111.stdout:] 2023-03-29T10:59:10.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.182+0000 7f5259bff700 1 -- 172.21.15.111:0/1003075234 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5240069de0 msgr2=0x7f524006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:10.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.182+0000 7f5259bff700 1 --2- 172.21.15.111:0/1003075234 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5240069de0 0x7f524006c290 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5248014040 tx=0x7f5248008030 comp rx=0 tx=0).stop 2023-03-29T10:59:10.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.182+0000 7f5259bff700 1 -- 172.21.15.111:0/1003075234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52540cf950 msgr2=0x7f525415e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:10.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.182+0000 7f5259bff700 1 --2- 172.21.15.111:0/1003075234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52540cf950 0x7f525415e2c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f524400c4e0 tx=0x7f524400c8a0 comp rx=0 tx=0).stop 2023-03-29T10:59:10.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.183+0000 7f5259bff700 1 -- 172.21.15.111:0/1003075234 shutdown_connections 2023-03-29T10:59:10.183 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.183+0000 7f5259bff700 1 --2- 172.21.15.111:0/1003075234 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5240069de0 0x7f524006c290 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:10.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.183+0000 7f5259bff700 1 --2- 172.21.15.111:0/1003075234 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f52540cf950 0x7f525415e2c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:10.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.183+0000 7f5259bff700 1 --2- 172.21.15.111:0/1003075234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f525415c950 0x7f525415f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:10.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.183+0000 7f5259bff700 1 --2- 172.21.15.111:0/1003075234 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f52540d3a30 0x7f52541470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:10.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.183+0000 7f5259bff700 1 -- 172.21.15.111:0/1003075234 >> 172.21.15.111:0/1003075234 conn(0x7f5254071b10 msgr2=0x7f52540b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:10.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.183+0000 7f5259bff700 1 -- 172.21.15.111:0/1003075234 shutdown_connections 2023-03-29T10:59:10.184 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.183+0000 7f5259bff700 1 -- 172.21.15.111:0/1003075234 wait complete. 2023-03-29T10:59:10.200 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-03-29T10:59:10.200 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-03-29T10:59:10.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.387+0000 7f3c7d3ff700 1 Processor -- start 2023-03-29T10:59:10.388 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.387+0000 7f3c7d3ff700 1 -- start start 2023-03-29T10:59:10.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.388+0000 7f3c7d3ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3c78158da0 0x7f3c7806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:10.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.388+0000 7f3c7d3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3c7806cc00 0x7f3c7806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:10.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.388+0000 7f3c7d3ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c780ca890 con 0x7f3c781589d0 2023-03-29T10:59:10.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.388+0000 7f3c7d3ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c780cd860 con 0x7f3c78158da0 2023-03-29T10:59:10.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.388+0000 7f3c7d3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c780ca5b0 con 0x7f3c7806cc00 2023-03-29T10:59:10.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.389+0000 7f3c767fc700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f3c78158da0 0x7f3c7806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:10.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.389+0000 7f3c76ffd700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f3c781589d0 0x7f3c780cd150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38172/0 (socket says 172.21.15.111:38172) 2023-03-29T10:59:10.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.389+0000 7f3c76ffd700 1 -- 172.21.15.111:0/291071617 learned_addr learned my addr 172.21.15.111:0/291071617 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:59:10.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.389+0000 7f3c767fc700 1 -- 172.21.15.111:0/291071617 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3c7806cc00 msgr2=0x7f3c7806a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:10.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.389+0000 7f3c767fc700 1 --2- 172.21.15.111:0/291071617 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3c7806cc00 0x7f3c7806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:10.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.389+0000 7f3c767fc700 1 -- 172.21.15.111:0/291071617 >> v1:172.21.15.111:6789/0 conn(0x7f3c781589d0 legacy=0x7f3c780cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:10.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.389+0000 7f3c767fc700 1 -- 172.21.15.111:0/291071617 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c780cc670 con 0x7f3c78158da0 2023-03-29T10:59:10.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.389+0000 7f3c75ffb700 1 -- 172.21.15.111:0/291071617 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660123098 0 0) 0x7f3c780ca890 con 0x7f3c781589d0 2023-03-29T10:59:10.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.389+0000 7f3c767fc700 1 --2- 172.21.15.111:0/291071617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c78158da0 0x7f3c7806c6c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009820 tx=0x7f3c6c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d673b0eb310d3af server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:10.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.390+0000 7f3c75ffb700 1 -- 172.21.15.111:0/291071617 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c005e30 con 0x7f3c78158da0 2023-03-29T10:59:10.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.390+0000 7f3c75ffb700 1 -- 172.21.15.111:0/291071617 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6c006030 con 0x7f3c78158da0 2023-03-29T10:59:10.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.390+0000 7f3c75ffb700 1 -- 172.21.15.111:0/291071617 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c0063e0 con 0x7f3c78158da0 2023-03-29T10:59:10.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.390+0000 7f3c7d3ff700 1 -- 172.21.15.111:0/291071617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c78158da0 msgr2=0x7f3c7806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:10.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.390+0000 7f3c7d3ff700 1 --2- 172.21.15.111:0/291071617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c78158da0 0x7f3c7806c6c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c009820 tx=0x7f3c6c009b30 comp rx=0 tx=0).stop 2023-03-29T10:59:10.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.390+0000 7f3c7d3ff700 1 -- 172.21.15.111:0/291071617 shutdown_connections 2023-03-29T10:59:10.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.390+0000 7f3c7d3ff700 1 --2- 172.21.15.111:0/291071617 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3c7806cc00 0x7f3c7806a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:10.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.390+0000 7f3c7d3ff700 1 --2- 172.21.15.111:0/291071617 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c78158da0 0x7f3c7806c6c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:10.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.390+0000 7f3c7d3ff700 1 -- 172.21.15.111:0/291071617 >> 172.21.15.111:0/291071617 conn(0x7f3c780cf720 msgr2=0x7f3c780cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:10.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.390+0000 7f3c7d3ff700 1 -- 172.21.15.111:0/291071617 shutdown_connections 2023-03-29T10:59:10.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.390+0000 7f3c7d3ff700 1 -- 172.21.15.111:0/291071617 wait complete. 2023-03-29T10:59:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.392+0000 7f3c7d3ff700 1 Processor -- start 2023-03-29T10:59:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.392+0000 7f3c7d3ff700 1 -- start start 2023-03-29T10:59:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.393+0000 7f3c7d3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c7806cc00 0x7f3c780ac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.393+0000 7f3c7d3ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c781589d0 0x7f3c7814bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.393+0000 7f3c7d3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3c78158da0 0x7f3c7815a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.393+0000 7f3c7d3ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7815a750 con 0x7f3c781589d0 2023-03-29T10:59:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.393+0000 7f3c7d3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7815b570 con 0x7f3c7806cc00 2023-03-29T10:59:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.393+0000 7f3c7d3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7815a960 con 0x7f3c78158da0 2023-03-29T10:59:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.393+0000 7f3c767fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c781589d0 0x7f3c7814bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:10.395 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.393+0000 7f3c767fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c781589d0 0x7f3c7814bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40240/0 (socket says 172.21.15.111:40240) 2023-03-29T10:59:10.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.393+0000 7f3c767fc700 1 -- 172.21.15.111:0/835502341 learned_addr learned my addr 172.21.15.111:0/835502341 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:10.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.393+0000 7f3c76ffd700 1 --2- 172.21.15.111:0/835502341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c7806cc00 0x7f3c780ac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:10.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.393+0000 7f3c777fe700 1 --2- 172.21.15.111:0/835502341 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3c78158da0 0x7f3c7815a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:10.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.394+0000 7f3c767fc700 1 -- 172.21.15.111:0/835502341 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3c78158da0 msgr2=0x7f3c7815a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:10.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.394+0000 7f3c767fc700 1 --2- 172.21.15.111:0/835502341 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3c78158da0 0x7f3c7815a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:10.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.394+0000 7f3c767fc700 1 -- 172.21.15.111:0/835502341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c7806cc00 msgr2=0x7f3c780ac2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:10.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.394+0000 7f3c767fc700 1 --2- 172.21.15.111:0/835502341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c7806cc00 0x7f3c780ac2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:10.396 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.394+0000 7f3c767fc700 1 -- 172.21.15.111:0/835502341 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c009430 con 0x7f3c781589d0 2023-03-29T10:59:10.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.394+0000 7f3c767fc700 1 --2- 172.21.15.111:0/835502341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c781589d0 0x7f3c7814bcd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3c780708b0 tx=0x7f3c6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:10.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.394+0000 7f3c5f7fe700 1 -- 172.21.15.111:0/835502341 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c006950 con 0x7f3c781589d0 2023-03-29T10:59:10.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.394+0000 7f3c5f7fe700 1 -- 172.21.15.111:0/835502341 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6c020070 con 0x7f3c781589d0 2023-03-29T10:59:10.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.394+0000 7f3c5f7fe700 1 -- 172.21.15.111:0/835502341 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6c025650 con 0x7f3c781589d0 2023-03-29T10:59:10.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.394+0000 7f3c7d3ff700 1 -- 172.21.15.111:0/835502341 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c780ccc30 con 0x7f3c781589d0 2023-03-29T10:59:10.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.394+0000 7f3c7d3ff700 1 -- 172.21.15.111:0/835502341 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c780cc950 con 0x7f3c781589d0 2023-03-29T10:59:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.395+0000 7f3c7d3ff700 1 -- 172.21.15.111:0/835502341 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7806a920 con 0x7f3c781589d0 2023-03-29T10:59:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.396+0000 7f3c5f7fe700 1 -- 172.21.15.111:0/835502341 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f3c6c022030 con 0x7f3c781589d0 2023-03-29T10:59:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.397+0000 7f3c5f7fe700 1 --2- 172.21.15.111:0/835502341 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3c64069de0 0x7f3c6406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.397+0000 7f3c5f7fe700 1 -- 172.21.15.111:0/835502341 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f3c6c0894b0 con 0x7f3c781589d0 2023-03-29T10:59:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.397+0000 7f3c76ffd700 1 --2- 172.21.15.111:0/835502341 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3c64069de0 0x7f3c6406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:10.398 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.398+0000 7f3c76ffd700 1 --2- 172.21.15.111:0/835502341 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3c64069de0 0x7f3c6406c290 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3c68007fd0 tx=0x7f3c68007de0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:10.401 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.401+0000 7f3c5f7fe700 1 -- 172.21.15.111:0/835502341 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f3c6c054f60 con 0x7f3c781589d0 2023-03-29T10:59:10.594 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.594+0000 7f3c7d3ff700 1 -- 172.21.15.111:0/835502341 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3c7815b720 con 0x7f3c781589d0 2023-03-29T10:59:10.814 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.813+0000 7f3c5f7fe700 1 -- 172.21.15.111:0/835502341 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v40) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3c7815b720 con 0x7f3c781589d0 2023-03-29T10:59:10.818 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.817+0000 7f3c7d3ff700 1 -- 172.21.15.111:0/835502341 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3c64069de0 msgr2=0x7f3c6406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:10.818 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.817+0000 7f3c7d3ff700 1 --2- 172.21.15.111:0/835502341 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3c64069de0 0x7f3c6406c290 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3c68007fd0 tx=0x7f3c68007de0 comp rx=0 tx=0).stop 2023-03-29T10:59:10.818 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.817+0000 7f3c7d3ff700 1 -- 172.21.15.111:0/835502341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c781589d0 msgr2=0x7f3c7814bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:10.818 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.817+0000 7f3c7d3ff700 1 --2- 172.21.15.111:0/835502341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c781589d0 0x7f3c7814bcd0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3c780708b0 tx=0x7f3c6c004750 comp rx=0 tx=0).stop 2023-03-29T10:59:10.818 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.818+0000 7f3c7d3ff700 1 -- 172.21.15.111:0/835502341 shutdown_connections 2023-03-29T10:59:10.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.818+0000 7f3c7d3ff700 1 --2- 172.21.15.111:0/835502341 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3c64069de0 0x7f3c6406c290 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:10.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.818+0000 7f3c7d3ff700 1 --2- 172.21.15.111:0/835502341 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f3c78158da0 0x7f3c7815a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:10.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.818+0000 7f3c7d3ff700 1 --2- 172.21.15.111:0/835502341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c7806cc00 0x7f3c780ac2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:10.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.818+0000 7f3c7d3ff700 1 --2- 172.21.15.111:0/835502341 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f3c781589d0 0x7f3c7814bcd0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:10.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.818+0000 7f3c7d3ff700 1 -- 172.21.15.111:0/835502341 >> 172.21.15.111:0/835502341 conn(0x7f3c780cf720 msgr2=0x7f3c781555e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:10.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.818+0000 7f3c7d3ff700 1 -- 172.21.15.111:0/835502341 shutdown_connections 2023-03-29T10:59:10.819 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:10.819+0000 7f3c7d3ff700 1 -- 172.21.15.111:0/835502341 wait complete. 2023-03-29T10:59:10.821 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_metadata' created 2023-03-29T10:59:10.838 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-03-29T10:59:11.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.026+0000 7f5973fff700 1 Processor -- start 2023-03-29T10:59:11.027 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.026+0000 7f5973fff700 1 -- start start 2023-03-29T10:59:11.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.027+0000 7f5973fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f596c15c590 0x7f596c160b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:11.029 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.027+0000 7f5973fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f596c0cf010 0x7f596c0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:11.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.027+0000 7f5973fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f596c0cfd80 0x7f596c0d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:11.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.027+0000 7f5973fff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596c0cd520 con 0x7f596c0cfd80 2023-03-29T10:59:11.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.027+0000 7f5973fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596c0d2640 con 0x7f596c0cf010 2023-03-29T10:59:11.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.027+0000 7f5973fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c1610d0 con 0x7f596c15c590 2023-03-29T10:59:11.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.028+0000 7f597259c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f596c0cfd80 0x7f596c0d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:11.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.028+0000 7f597259c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f596c0cfd80 0x7f596c0d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40252/0 (socket says 172.21.15.111:40252) 2023-03-29T10:59:11.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.028+0000 7f597259c700 1 -- 172.21.15.111:0/2685677996 learned_addr learned my addr 172.21.15.111:0/2685677996 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:11.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.028+0000 7f5971d9b700 1 --2- 172.21.15.111:0/2685677996 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f596c15c590 0x7f596c160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:11.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.028+0000 7f597159a700 1 --2- 172.21.15.111:0/2685677996 >> v2:172.21.15.119:3300/0 conn(0x7f596c0cf010 0x7f596c0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:11.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.028+0000 7f597259c700 1 -- 172.21.15.111:0/2685677996 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f596c15c590 msgr2=0x7f596c160b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:11.030 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.028+0000 7f597259c700 1 --2- 172.21.15.111:0/2685677996 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f596c15c590 0x7f596c160b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:11.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.028+0000 7f597259c700 1 -- 172.21.15.111:0/2685677996 >> v2:172.21.15.119:3300/0 conn(0x7f596c0cf010 msgr2=0x7f596c0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:11.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.028+0000 7f597259c700 1 --2- 172.21.15.111:0/2685677996 >> v2:172.21.15.119:3300/0 conn(0x7f596c0cf010 0x7f596c0cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:11.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.028+0000 7f597259c700 1 -- 172.21.15.111:0/2685677996 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c0d0200 con 0x7f596c0cfd80 2023-03-29T10:59:11.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.028+0000 7f5971d9b700 1 --2- 172.21.15.111:0/2685677996 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f596c15c590 0x7f596c160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:59:11.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.028+0000 7f597159a700 1 --2- 172.21.15.111:0/2685677996 >> v2:172.21.15.119:3300/0 conn(0x7f596c0cf010 0x7f596c0cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:59:11.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.028+0000 7f597259c700 1 --2- 172.21.15.111:0/2685677996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f596c0cfd80 0x7f596c0d0aa0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f596c0d3f90 tx=0x7f596400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86455120c88f6a2a server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:11.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.029+0000 7f5970d99700 1 -- 172.21.15.111:0/2685677996 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596400e070 con 0x7f596c0cfd80 2023-03-29T10:59:11.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.029+0000 7f5970d99700 1 -- 172.21.15.111:0/2685677996 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596400b9c0 con 0x7f596c0cfd80 2023-03-29T10:59:11.031 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.029+0000 7f5970d99700 1 -- 172.21.15.111:0/2685677996 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596400bd70 con 0x7f596c0cfd80 2023-03-29T10:59:11.034 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.029+0000 7f5973fff700 1 -- 172.21.15.111:0/2685677996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f596c0cfd80 msgr2=0x7f596c0d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:11.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.029+0000 7f5973fff700 1 --2- 172.21.15.111:0/2685677996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f596c0cfd80 0x7f596c0d0aa0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f596c0d3f90 tx=0x7f596400b330 comp rx=0 tx=0).stop 2023-03-29T10:59:11.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.029+0000 7f5973fff700 1 -- 172.21.15.111:0/2685677996 shutdown_connections 2023-03-29T10:59:11.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.029+0000 7f5973fff700 1 --2- 172.21.15.111:0/2685677996 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f596c15c590 0x7f596c160b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:11.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.029+0000 7f5973fff700 1 --2- 172.21.15.111:0/2685677996 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f596c0cfd80 0x7f596c0d0aa0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:11.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.029+0000 7f5973fff700 1 --2- 172.21.15.111:0/2685677996 >> v2:172.21.15.119:3300/0 conn(0x7f596c0cf010 0x7f596c0cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:11.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.029+0000 7f5973fff700 1 -- 172.21.15.111:0/2685677996 >> 172.21.15.111:0/2685677996 conn(0x7f596c071b10 msgr2=0x7f596c071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:11.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.029+0000 7f5973fff700 1 -- 172.21.15.111:0/2685677996 shutdown_connections 2023-03-29T10:59:11.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.030+0000 7f5973fff700 1 -- 172.21.15.111:0/2685677996 wait complete. 2023-03-29T10:59:11.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.031+0000 7f5973fff700 1 Processor -- start 2023-03-29T10:59:11.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.031+0000 7f5973fff700 1 -- start start 2023-03-29T10:59:11.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.032+0000 7f5973fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f596c0cf010 0x7f596c15e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:11.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.032+0000 7f5973fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f596c0cfd80 0x7f596c1f5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:11.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.032+0000 7f5973fff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f596c15c590 0x7f596c15d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:11.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.032+0000 7f5973fff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c15c960 con 0x7f596c0cf010 2023-03-29T10:59:11.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.032+0000 7f5973fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c1f60b0 con 0x7f596c0cfd80 2023-03-29T10:59:11.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.032+0000 7f5973fff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c1f6390 con 0x7f596c15c590 2023-03-29T10:59:11.035 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.032+0000 7f5971d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f596c0cf010 0x7f596c15e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:11.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.032+0000 7f5971d9b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f596c0cf010 0x7f596c15e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40258/0 (socket says 172.21.15.111:40258) 2023-03-29T10:59:11.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.032+0000 7f5971d9b700 1 -- 172.21.15.111:0/312697950 learned_addr learned my addr 172.21.15.111:0/312697950 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:11.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.033+0000 7f597159a700 1 --2- 172.21.15.111:0/312697950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f596c0cfd80 0x7f596c1f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:11.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.033+0000 7f597259c700 1 --2- 172.21.15.111:0/312697950 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f596c15c590 0x7f596c15d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:11.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.033+0000 7f5971d9b700 1 -- 172.21.15.111:0/312697950 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f596c15c590 msgr2=0x7f596c15d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:11.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.033+0000 7f5971d9b700 1 --2- 172.21.15.111:0/312697950 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f596c15c590 0x7f596c15d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:11.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.033+0000 7f5971d9b700 1 -- 172.21.15.111:0/312697950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f596c0cfd80 msgr2=0x7f596c1f5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:11.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.033+0000 7f5971d9b700 1 --2- 172.21.15.111:0/312697950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f596c0cfd80 0x7f596c1f5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:11.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.033+0000 7f5971d9b700 1 -- 172.21.15.111:0/312697950 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5964009b30 con 0x7f596c0cf010 2023-03-29T10:59:11.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.033+0000 7f597259c700 1 --2- 172.21.15.111:0/312697950 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f596c15c590 0x7f596c15d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:59:11.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.033+0000 7f5971d9b700 1 --2- 172.21.15.111:0/312697950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f596c0cf010 0x7f596c15e300 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f596800e760 tx=0x7f596800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:11.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.033+0000 7f59627fc700 1 -- 172.21.15.111:0/312697950 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968013610 con 0x7f596c0cf010 2023-03-29T10:59:11.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.033+0000 7f59627fc700 1 -- 172.21.15.111:0/312697950 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5968013770 con 0x7f596c0cf010 2023-03-29T10:59:11.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.033+0000 7f5973fff700 1 -- 172.21.15.111:0/312697950 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596c0cf560 con 0x7f596c0cf010 2023-03-29T10:59:11.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.033+0000 7f59627fc700 1 -- 172.21.15.111:0/312697950 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968013b20 con 0x7f596c0cf010 2023-03-29T10:59:11.036 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.033+0000 7f5973fff700 1 -- 172.21.15.111:0/312697950 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596c0b23c0 con 0x7f596c0cf010 2023-03-29T10:59:11.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.034+0000 7f5973fff700 1 -- 172.21.15.111:0/312697950 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596c0cd770 con 0x7f596c0cf010 2023-03-29T10:59:11.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.036+0000 7f59627fc700 1 -- 172.21.15.111:0/312697950 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f5968013c80 con 0x7f596c0cf010 2023-03-29T10:59:11.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.037+0000 7f59627fc700 1 --2- 172.21.15.111:0/312697950 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5950069de0 0x7f595006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:11.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.037+0000 7f59627fc700 1 -- 172.21.15.111:0/312697950 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f5968087fd0 con 0x7f596c0cf010 2023-03-29T10:59:11.037 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.037+0000 7f597159a700 1 --2- 172.21.15.111:0/312697950 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5950069de0 0x7f595006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:11.038 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.037+0000 7f597159a700 1 --2- 172.21.15.111:0/312697950 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5950069de0 0x7f595006c290 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f596c15fce0 tx=0x7f595c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:11.040 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.040+0000 7f59627fc700 1 -- 172.21.15.111:0/312697950 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f5968057430 con 0x7f596c0cf010 2023-03-29T10:59:11.236 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.235+0000 7f5973fff700 1 -- 172.21.15.111:0/312697950 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f596c074430 con 0x7f596c0cf010 2023-03-29T10:59:11.818 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.817+0000 7f59627fc700 1 -- 172.21.15.111:0/312697950 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v41) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5968057020 con 0x7f596c0cf010 2023-03-29T10:59:11.829 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.828+0000 7f5973fff700 1 -- 172.21.15.111:0/312697950 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5950069de0 msgr2=0x7f595006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:11.829 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.828+0000 7f5973fff700 1 --2- 172.21.15.111:0/312697950 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5950069de0 0x7f595006c290 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f596c15fce0 tx=0x7f595c008040 comp rx=0 tx=0).stop 2023-03-29T10:59:11.829 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.828+0000 7f5973fff700 1 -- 172.21.15.111:0/312697950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f596c0cf010 msgr2=0x7f596c15e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:11.829 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.829+0000 7f5973fff700 1 --2- 172.21.15.111:0/312697950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f596c0cf010 0x7f596c15e300 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f596800e760 tx=0x7f596800eb20 comp rx=0 tx=0).stop 2023-03-29T10:59:11.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.831+0000 7f5973fff700 1 -- 172.21.15.111:0/312697950 shutdown_connections 2023-03-29T10:59:11.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.831+0000 7f5973fff700 1 --2- 172.21.15.111:0/312697950 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f5950069de0 0x7f595006c290 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:11.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.831+0000 7f5973fff700 1 --2- 172.21.15.111:0/312697950 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f596c0cf010 0x7f596c15e300 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:11.831 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.831+0000 7f5973fff700 1 --2- 172.21.15.111:0/312697950 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f596c15c590 0x7f596c15d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:11.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.831+0000 7f5973fff700 1 --2- 172.21.15.111:0/312697950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f596c0cfd80 0x7f596c1f5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:11.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.831+0000 7f5973fff700 1 -- 172.21.15.111:0/312697950 >> 172.21.15.111:0/312697950 conn(0x7f596c071b10 msgr2=0x7f596c0b4580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:11.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.831+0000 7f5973fff700 1 -- 172.21.15.111:0/312697950 shutdown_connections 2023-03-29T10:59:11.832 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:11.832+0000 7f5973fff700 1 -- 172.21.15.111:0/312697950 wait complete. 2023-03-29T10:59:11.840 INFO:teuthology.orchestra.run.smithi111.stderr:pool 'cephfs_data' created 2023-03-29T10:59:11.857 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-03-29T10:59:12.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.044+0000 7f35debff700 1 Processor -- start 2023-03-29T10:59:12.044 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.044+0000 7f35debff700 1 -- start start 2023-03-29T10:59:12.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.045+0000 7f35debff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35d815a5a0 0x7f35d806a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.045 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.045+0000 7f35debff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f35d8071200 0x7f35d806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.045+0000 7f35debff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d80cb280 con 0x7f35d806a920 2023-03-29T10:59:12.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.045+0000 7f35debff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d815a970 con 0x7f35d8071200 2023-03-29T10:59:12.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.045+0000 7f35debff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d806ad90 con 0x7f35d815a5a0 2023-03-29T10:59:12.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.045+0000 7f35d7fff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f35d806a920 0x7f35d8070af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38186/0 (socket says 172.21.15.111:38186) 2023-03-29T10:59:12.046 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.045+0000 7f35d7fff700 1 -- 172.21.15.111:0/1162277688 learned_addr learned my addr 172.21.15.111:0/1162277688 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:59:12.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.045+0000 7f35dd19c700 1 --2- 172.21.15.111:0/1162277688 >> v2:172.21.15.119:3300/0 conn(0x7f35d8071200 0x7f35d806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.045+0000 7f35d77fe700 1 -- 172.21.15.111:0/1162277688 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 744850151 0 0) 0x7f35c4001290 con 0x7f35d806a920 2023-03-29T10:59:12.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.045+0000 7f35dc99b700 1 --2- 172.21.15.111:0/1162277688 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35d815a5a0 0x7f35d806a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.045+0000 7f35d77fe700 1 -- 172.21.15.111:0/1162277688 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35c80035b0 con 0x7f35d806a920 2023-03-29T10:59:12.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.046+0000 7f35d77fe700 1 -- 172.21.15.111:0/1162277688 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2245459806 0 0) 0x7f35c80035b0 con 0x7f35d806a920 2023-03-29T10:59:12.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.046+0000 7f35d77fe700 1 -- 172.21.15.111:0/1162277688 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35d815a5a0 msgr2=0x7f35d806a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:12.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.046+0000 7f35d77fe700 1 --2- 172.21.15.111:0/1162277688 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35d815a5a0 0x7f35d806a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.046+0000 7f35d77fe700 1 -- 172.21.15.111:0/1162277688 >> v2:172.21.15.119:3300/0 conn(0x7f35d8071200 msgr2=0x7f35d806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:12.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.046+0000 7f35d77fe700 1 --2- 172.21.15.111:0/1162277688 >> v2:172.21.15.119:3300/0 conn(0x7f35d8071200 0x7f35d806d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.046+0000 7f35d77fe700 1 -- 172.21.15.111:0/1162277688 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d8071700 con 0x7f35d806a920 2023-03-29T10:59:12.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.046+0000 7f35d77fe700 1 -- 172.21.15.111:0/1162277688 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f35c4001980 con 0x7f35d806a920 2023-03-29T10:59:12.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.046+0000 7f35d77fe700 1 -- 172.21.15.111:0/1162277688 >> v1:172.21.15.111:6789/0 conn(0x7f35d806a920 legacy=0x7f35d8070af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:12.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.046+0000 7f35d77fe700 1 --2- 172.21.15.111:0/1162277688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35c8003a00 0x7f35c8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.046+0000 7f35d77fe700 1 --2- 172.21.15.111:0/1162277688 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35d815a5a0 0x7f35c8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.046+0000 7f35d77fe700 1 --2- 172.21.15.111:0/1162277688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35c8004d90 0x7f35c8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.046+0000 7f35d77fe700 1 -- 172.21.15.111:0/1162277688 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d815a970 con 0x7f35c8004d90 2023-03-29T10:59:12.048 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.046+0000 7f35d77fe700 1 -- 172.21.15.111:0/1162277688 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d806ad90 con 0x7f35c8003a00 2023-03-29T10:59:12.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.046+0000 7f35d77fe700 1 -- 172.21.15.111:0/1162277688 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c800ba10 con 0x7f35d815a5a0 2023-03-29T10:59:12.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.046+0000 7f35dd19c700 1 --2- 172.21.15.111:0/1162277688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35c8004d90 0x7f35c8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.046+0000 7f35dc99b700 1 --2- 172.21.15.111:0/1162277688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35c8003a00 0x7f35c8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.046+0000 7f35d7fff700 1 --2- 172.21.15.111:0/1162277688 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35d815a5a0 0x7f35c8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.047+0000 7f35dd19c700 1 -- 172.21.15.111:0/1162277688 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35d815a5a0 msgr2=0x7f35c8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:12.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.047+0000 7f35dd19c700 1 --2- 172.21.15.111:0/1162277688 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35d815a5a0 0x7f35c8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.047+0000 7f35dd19c700 1 -- 172.21.15.111:0/1162277688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35c8003a00 msgr2=0x7f35c8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:12.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.047+0000 7f35dd19c700 1 --2- 172.21.15.111:0/1162277688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35c8003a00 0x7f35c8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.047+0000 7f35dd19c700 1 -- 172.21.15.111:0/1162277688 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c80035b0 con 0x7f35c8004d90 2023-03-29T10:59:12.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.047+0000 7f35dd19c700 1 --2- 172.21.15.111:0/1162277688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35c8004d90 0x7f35c8005200 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009b30 tx=0x7f35c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44001c62b68fe852 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:12.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.047+0000 7f35d77fe700 1 -- 172.21.15.111:0/1162277688 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d000e660 con 0x7f35c8004d90 2023-03-29T10:59:12.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.047+0000 7f35d77fe700 1 -- 172.21.15.111:0/1162277688 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d000e7c0 con 0x7f35c8004d90 2023-03-29T10:59:12.049 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.047+0000 7f35d77fe700 1 -- 172.21.15.111:0/1162277688 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d000eb30 con 0x7f35c8004d90 2023-03-29T10:59:12.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.048+0000 7f35debff700 1 -- 172.21.15.111:0/1162277688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35c8004d90 msgr2=0x7f35c8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:12.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.048+0000 7f35debff700 1 --2- 172.21.15.111:0/1162277688 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35c8004d90 0x7f35c8005200 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f35d0009b30 tx=0x7f35c8005880 comp rx=0 tx=0).stop 2023-03-29T10:59:12.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.048+0000 7f35dd19c700 1 -- 172.21.15.111:0/1162277688 reap_dead start 2023-03-29T10:59:12.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.048+0000 7f35debff700 1 -- 172.21.15.111:0/1162277688 shutdown_connections 2023-03-29T10:59:12.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.048+0000 7f35debff700 1 -- 172.21.15.111:0/1162277688 >> 172.21.15.111:0/1162277688 conn(0x7f35d80cf720 msgr2=0x7f35d80cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:12.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.048+0000 7f35debff700 1 -- 172.21.15.111:0/1162277688 shutdown_connections 2023-03-29T10:59:12.050 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.048+0000 7f35debff700 1 -- 172.21.15.111:0/1162277688 wait complete. 2023-03-29T10:59:12.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.050+0000 7f35debff700 1 Processor -- start 2023-03-29T10:59:12.051 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.050+0000 7f35debff700 1 -- start start 2023-03-29T10:59:12.052 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.052+0000 7f35debff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35c8004d90 0x7f35d815bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.052+0000 7f35debff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35d8155890 0x7f35d80c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.052+0000 7f35debff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35d8155dc0 0x7f35d8154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.052+0000 7f35debff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d80c3e50 con 0x7f35c8004d90 2023-03-29T10:59:12.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.052+0000 7f35debff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d815b500 con 0x7f35d8155dc0 2023-03-29T10:59:12.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.052+0000 7f35debff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d815c490 con 0x7f35d8155890 2023-03-29T10:59:12.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.052+0000 7f35dc99b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35c8004d90 0x7f35d815bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.052+0000 7f35dc99b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35c8004d90 0x7f35d815bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40276/0 (socket says 172.21.15.111:40276) 2023-03-29T10:59:12.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.052+0000 7f35dc99b700 1 -- 172.21.15.111:0/1405576018 learned_addr learned my addr 172.21.15.111:0/1405576018 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:12.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.052+0000 7f35d7fff700 1 --2- 172.21.15.111:0/1405576018 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35d8155890 0x7f35d80c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.052+0000 7f35dd19c700 1 --2- 172.21.15.111:0/1405576018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35d8155dc0 0x7f35d8154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.052+0000 7f35dc99b700 1 -- 172.21.15.111:0/1405576018 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35d8155890 msgr2=0x7f35d80c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:12.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.052+0000 7f35dc99b700 1 --2- 172.21.15.111:0/1405576018 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35d8155890 0x7f35d80c4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.052+0000 7f35dc99b700 1 -- 172.21.15.111:0/1405576018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35d8155dc0 msgr2=0x7f35d8154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:12.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.052+0000 7f35dc99b700 1 --2- 172.21.15.111:0/1405576018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35d8155dc0 0x7f35d8154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.052+0000 7f35dc99b700 1 -- 172.21.15.111:0/1405576018 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d806c6c0 con 0x7f35c8004d90 2023-03-29T10:59:12.054 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.053+0000 7f35dc99b700 1 --2- 172.21.15.111:0/1405576018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35c8004d90 0x7f35d815bf50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009fc0 tx=0x7f35cc00f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:12.054 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.053+0000 7f35d57fa700 1 -- 172.21.15.111:0/1405576018 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc00e040 con 0x7f35c8004d90 2023-03-29T10:59:12.054 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.053+0000 7f35d57fa700 1 -- 172.21.15.111:0/1405576018 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35cc00f040 con 0x7f35c8004d90 2023-03-29T10:59:12.054 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.053+0000 7f35debff700 1 -- 172.21.15.111:0/1405576018 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d80719e0 con 0x7f35c8004d90 2023-03-29T10:59:12.054 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.053+0000 7f35d57fa700 1 -- 172.21.15.111:0/1405576018 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc015430 con 0x7f35c8004d90 2023-03-29T10:59:12.054 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.053+0000 7f35debff700 1 -- 172.21.15.111:0/1405576018 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d806c9a0 con 0x7f35c8004d90 2023-03-29T10:59:12.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.054+0000 7f35debff700 1 -- 172.21.15.111:0/1405576018 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d80cbcf0 con 0x7f35c8004d90 2023-03-29T10:59:12.055 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.055+0000 7f35d57fa700 1 -- 172.21.15.111:0/1405576018 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f35cc015590 con 0x7f35c8004d90 2023-03-29T10:59:12.056 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.056+0000 7f35d57fa700 1 --2- 172.21.15.111:0/1405576018 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f35bc069de0 0x7f35bc06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.056+0000 7f35d57fa700 1 -- 172.21.15.111:0/1405576018 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f35cc088c50 con 0x7f35c8004d90 2023-03-29T10:59:12.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.056+0000 7f35d7fff700 1 --2- 172.21.15.111:0/1405576018 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f35bc069de0 0x7f35bc06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.057+0000 7f35d7fff700 1 --2- 172.21.15.111:0/1405576018 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f35bc069de0 0x7f35bc06c290 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f35d81579c0 tx=0x7f35c4007a60 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:12.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.061+0000 7f35d57fa700 1 -- 172.21.15.111:0/1405576018 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f35cc054380 con 0x7f35c8004d90 2023-03-29T10:59:12.296 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.296+0000 7f35debff700 1 -- 172.21.15.111:0/1405576018 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f35d80add10 con 0x7f35c8004d90 2023-03-29T10:59:12.305 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.305+0000 7f35d57fa700 1 -- 172.21.15.111:0/1405576018 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f35cc0579a0 con 0x7f35c8004d90 2023-03-29T10:59:12.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.310+0000 7f35debff700 1 -- 172.21.15.111:0/1405576018 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f35bc069de0 msgr2=0x7f35bc06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:12.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.310+0000 7f35debff700 1 --2- 172.21.15.111:0/1405576018 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f35bc069de0 0x7f35bc06c290 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f35d81579c0 tx=0x7f35c4007a60 comp rx=0 tx=0).stop 2023-03-29T10:59:12.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.310+0000 7f35debff700 1 -- 172.21.15.111:0/1405576018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35c8004d90 msgr2=0x7f35d815bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:12.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.310+0000 7f35debff700 1 --2- 172.21.15.111:0/1405576018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35c8004d90 0x7f35d815bf50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009fc0 tx=0x7f35cc00f490 comp rx=0 tx=0).stop 2023-03-29T10:59:12.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.311+0000 7f35debff700 1 -- 172.21.15.111:0/1405576018 shutdown_connections 2023-03-29T10:59:12.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.311+0000 7f35debff700 1 --2- 172.21.15.111:0/1405576018 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f35bc069de0 0x7f35bc06c290 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.311+0000 7f35debff700 1 --2- 172.21.15.111:0/1405576018 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f35c8004d90 0x7f35d815bf50 secure :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f35cc009fc0 tx=0x7f35cc00f490 comp rx=0 tx=0).stop 2023-03-29T10:59:12.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.311+0000 7f35debff700 1 --2- 172.21.15.111:0/1405576018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35d8155dc0 0x7f35d8154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.311+0000 7f35debff700 1 --2- 172.21.15.111:0/1405576018 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f35d8155890 0x7f35d80c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.311+0000 7f35debff700 1 -- 172.21.15.111:0/1405576018 >> 172.21.15.111:0/1405576018 conn(0x7f35d80cf720 msgr2=0x7f35d806da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:12.324 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.323+0000 7f35debff700 1 -- 172.21.15.111:0/1405576018 shutdown_connections 2023-03-29T10:59:12.327 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.324+0000 7f35debff700 1 -- 172.21.15.111:0/1405576018 wait complete. 2023-03-29T10:59:12.330 INFO:teuthology.orchestra.run.smithi111.stderr:new fs with metadata pool 6 and data pool 7 2023-03-29T10:59:12.351 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:59:12.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.533+0000 7f08dbdff700 1 Processor -- start 2023-03-29T10:59:12.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.533+0000 7f08dbdff700 1 -- start start 2023-03-29T10:59:12.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.535+0000 7f08dbdff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08d415db70 0x7f08d40cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.535+0000 7f08dbdff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f08d40d2ea0 0x7f08d40d3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.536 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.535+0000 7f08dbdff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d40d3860 con 0x7f08d40b2440 2023-03-29T10:59:12.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.535+0000 7f08dbdff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d40b2810 con 0x7f08d40d2ea0 2023-03-29T10:59:12.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.535+0000 7f08dbdff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d415ce30 con 0x7f08d415db70 2023-03-29T10:59:12.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.535+0000 7f08d9b9b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f08d40b2440 0x7f08d4160e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38194/0 (socket says 172.21.15.111:38194) 2023-03-29T10:59:12.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.535+0000 7f08d9b9b700 1 -- 172.21.15.111:0/986581023 learned_addr learned my addr 172.21.15.111:0/986581023 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:59:12.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.535+0000 7f08da39c700 1 --2- 172.21.15.111:0/986581023 >> v2:172.21.15.111:3300/0 conn(0x7f08d40d2ea0 0x7f08d40d3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.535+0000 7f08d939a700 1 --2- 172.21.15.111:0/986581023 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08d415db70 0x7f08d40cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.535+0000 7f08d8b99700 1 -- 172.21.15.111:0/986581023 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875850635 0 0) 0x7f08d40d3860 con 0x7f08d40b2440 2023-03-29T10:59:12.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.535+0000 7f08d8b99700 1 -- 172.21.15.111:0/986581023 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08c00035b0 con 0x7f08d40b2440 2023-03-29T10:59:12.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.535+0000 7f08da39c700 1 -- 172.21.15.111:0/986581023 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08d415db70 msgr2=0x7f08d40cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:12.537 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.535+0000 7f08da39c700 1 --2- 172.21.15.111:0/986581023 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08d415db70 0x7f08d40cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.535+0000 7f08da39c700 1 -- 172.21.15.111:0/986581023 >> v1:172.21.15.111:6789/0 conn(0x7f08d40b2440 legacy=0x7f08d4160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:12.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.535+0000 7f08da39c700 1 -- 172.21.15.111:0/986581023 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d40cdee0 con 0x7f08d40d2ea0 2023-03-29T10:59:12.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.536+0000 7f08da39c700 1 --2- 172.21.15.111:0/986581023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d40d2ea0 0x7f08d40d3320 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009d90 tx=0x7f08cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28210cf803315426 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:12.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.536+0000 7f08d8b99700 1 -- 172.21.15.111:0/986581023 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc00e070 con 0x7f08d40d2ea0 2023-03-29T10:59:12.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.536+0000 7f08d8b99700 1 -- 172.21.15.111:0/986581023 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08cc00bb20 con 0x7f08d40d2ea0 2023-03-29T10:59:12.538 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.536+0000 7f08d8b99700 1 -- 172.21.15.111:0/986581023 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc012430 con 0x7f08d40d2ea0 2023-03-29T10:59:12.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.536+0000 7f08dbdff700 1 -- 172.21.15.111:0/986581023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d40d2ea0 msgr2=0x7f08d40d3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:12.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.536+0000 7f08dbdff700 1 --2- 172.21.15.111:0/986581023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d40d2ea0 0x7f08d40d3320 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f08cc009d90 tx=0x7f08cc00b330 comp rx=0 tx=0).stop 2023-03-29T10:59:12.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.536+0000 7f08dbdff700 1 -- 172.21.15.111:0/986581023 shutdown_connections 2023-03-29T10:59:12.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.536+0000 7f08dbdff700 1 --2- 172.21.15.111:0/986581023 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d40d2ea0 0x7f08d40d3320 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.539 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.536+0000 7f08dbdff700 1 --2- 172.21.15.111:0/986581023 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08d415db70 0x7f08d40cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.536+0000 7f08dbdff700 1 -- 172.21.15.111:0/986581023 >> 172.21.15.111:0/986581023 conn(0x7f08d4071b10 msgr2=0x7f08d4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:12.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.537+0000 7f08dbdff700 1 -- 172.21.15.111:0/986581023 shutdown_connections 2023-03-29T10:59:12.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.537+0000 7f08dbdff700 1 -- 172.21.15.111:0/986581023 wait complete. 2023-03-29T10:59:12.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.538+0000 7f08dbdff700 1 Processor -- start 2023-03-29T10:59:12.540 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.538+0000 7f08dbdff700 1 -- start start 2023-03-29T10:59:12.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.539+0000 7f08dbdff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08d40b2440 0x7f08d414b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.539+0000 7f08dbdff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08d40d2ea0 0x7f08d40cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.539+0000 7f08dbdff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d415db70 0x7f08d40cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.539+0000 7f08dbdff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d40b7580 con 0x7f08d415db70 2023-03-29T10:59:12.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.539+0000 7f08dbdff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d4158940 con 0x7f08d40d2ea0 2023-03-29T10:59:12.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.539+0000 7f08dbdff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d40b7a10 con 0x7f08d40b2440 2023-03-29T10:59:12.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.539+0000 7f08da39c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d415db70 0x7f08d40cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.539+0000 7f08da39c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d415db70 0x7f08d40cbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40288/0 (socket says 172.21.15.111:40288) 2023-03-29T10:59:12.542 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.539+0000 7f08da39c700 1 -- 172.21.15.111:0/476287295 learned_addr learned my addr 172.21.15.111:0/476287295 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:12.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.540+0000 7f08d9b9b700 1 --2- 172.21.15.111:0/476287295 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08d40b2440 0x7f08d414b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.540+0000 7f08d939a700 1 --2- 172.21.15.111:0/476287295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08d40d2ea0 0x7f08d40cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.540+0000 7f08da39c700 1 -- 172.21.15.111:0/476287295 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08d40b2440 msgr2=0x7f08d414b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:12.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.540+0000 7f08da39c700 1 --2- 172.21.15.111:0/476287295 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08d40b2440 0x7f08d414b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.540+0000 7f08da39c700 1 -- 172.21.15.111:0/476287295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08d40d2ea0 msgr2=0x7f08d40cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:12.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.540+0000 7f08da39c700 1 --2- 172.21.15.111:0/476287295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08d40d2ea0 0x7f08d40cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.540+0000 7f08da39c700 1 -- 172.21.15.111:0/476287295 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08cc009a70 con 0x7f08d415db70 2023-03-29T10:59:12.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.540+0000 7f08d9b9b700 1 --2- 172.21.15.111:0/476287295 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08d40b2440 0x7f08d414b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:59:12.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.540+0000 7f08da39c700 1 --2- 172.21.15.111:0/476287295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d415db70 0x7f08d40cbe90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f08cc014040 tx=0x7f08cc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:12.543 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.540+0000 7f08ca7fc700 1 -- 172.21.15.111:0/476287295 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc0129f0 con 0x7f08d415db70 2023-03-29T10:59:12.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.540+0000 7f08ca7fc700 1 -- 172.21.15.111:0/476287295 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08cc020070 con 0x7f08d415db70 2023-03-29T10:59:12.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.540+0000 7f08ca7fc700 1 -- 172.21.15.111:0/476287295 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc012db0 con 0x7f08d415db70 2023-03-29T10:59:12.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.540+0000 7f08dbdff700 1 -- 172.21.15.111:0/476287295 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d415d090 con 0x7f08d415db70 2023-03-29T10:59:12.544 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.541+0000 7f08dbdff700 1 -- 172.21.15.111:0/476287295 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d40cdc00 con 0x7f08d415db70 2023-03-29T10:59:12.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.542+0000 7f08dbdff700 1 -- 172.21.15.111:0/476287295 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d415d370 con 0x7f08d415db70 2023-03-29T10:59:12.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.542+0000 7f08ca7fc700 1 -- 172.21.15.111:0/476287295 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f08cc004040 con 0x7f08d415db70 2023-03-29T10:59:12.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.543+0000 7f08ca7fc700 1 --2- 172.21.15.111:0/476287295 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f08b8069de0 0x7f08b806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.543+0000 7f08ca7fc700 1 -- 172.21.15.111:0/476287295 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f08cc022070 con 0x7f08d415db70 2023-03-29T10:59:12.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.544+0000 7f08d9b9b700 1 --2- 172.21.15.111:0/476287295 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f08b8069de0 0x7f08b806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.545 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.544+0000 7f08d9b9b700 1 --2- 172.21.15.111:0/476287295 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f08b8069de0 0x7f08b806c290 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f08d00011e0 tx=0x7f08d0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:12.548 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.548+0000 7f08ca7fc700 1 -- 172.21.15.111:0/476287295 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f08cc055690 con 0x7f08d415db70 2023-03-29T10:59:12.746 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.746+0000 7f08dbdff700 1 -- 172.21.15.111:0/476287295 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08d4074430 con 0x7f08d415db70 2023-03-29T10:59:12.748 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.747+0000 7f08ca7fc700 1 -- 172.21.15.111:0/476287295 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18189 (secure 0 0 0) 0x7f08cc02c020 con 0x7f08d415db70 2023-03-29T10:59:12.748 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:59:12.748 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":42,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:59:12.298098+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:59:10.633156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-03-29T10:59:11.274143+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:59:12.750 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.750+0000 7f08dbdff700 1 -- 172.21.15.111:0/476287295 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f08b8069de0 msgr2=0x7f08b806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:12.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.750+0000 7f08dbdff700 1 --2- 172.21.15.111:0/476287295 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f08b8069de0 0x7f08b806c290 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f08d00011e0 tx=0x7f08d0009040 comp rx=0 tx=0).stop 2023-03-29T10:59:12.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.750+0000 7f08dbdff700 1 -- 172.21.15.111:0/476287295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d415db70 msgr2=0x7f08d40cbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:12.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.750+0000 7f08dbdff700 1 --2- 172.21.15.111:0/476287295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d415db70 0x7f08d40cbe90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f08cc014040 tx=0x7f08cc004ad0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.750+0000 7f08dbdff700 1 -- 172.21.15.111:0/476287295 shutdown_connections 2023-03-29T10:59:12.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.750+0000 7f08dbdff700 1 --2- 172.21.15.111:0/476287295 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f08b8069de0 0x7f08b806c290 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.750+0000 7f08dbdff700 1 --2- 172.21.15.111:0/476287295 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f08d415db70 0x7f08d40cbe90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.750+0000 7f08dbdff700 1 --2- 172.21.15.111:0/476287295 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f08d40b2440 0x7f08d414b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.751 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.750+0000 7f08dbdff700 1 --2- 172.21.15.111:0/476287295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08d40d2ea0 0x7f08d40cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.750+0000 7f08dbdff700 1 -- 172.21.15.111:0/476287295 >> 172.21.15.111:0/476287295 conn(0x7f08d4071b10 msgr2=0x7f08d415b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:12.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.750+0000 7f08dbdff700 1 -- 172.21.15.111:0/476287295 shutdown_connections 2023-03-29T10:59:12.752 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.750+0000 7f08dbdff700 1 -- 172.21.15.111:0/476287295 wait complete. 2023-03-29T10:59:12.768 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:59:12.952 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.952+0000 7faa591ff700 1 Processor -- start 2023-03-29T10:59:12.953 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.952+0000 7faa591ff700 1 -- start start 2023-03-29T10:59:12.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.953+0000 7faa591ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa54158d30 0x7faa540ca330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.953+0000 7faa591ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa540cc9b0 con 0x7faa54071470 2023-03-29T10:59:12.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.953+0000 7faa591ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa540ccc90 con 0x7faa540cda50 2023-03-29T10:59:12.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.953+0000 7faa591ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa54071dd0 con 0x7faa54158d30 2023-03-29T10:59:12.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.953+0000 7faa5359e700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7faa54071470 0x7faa5406cdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38206/0 (socket says 172.21.15.111:38206) 2023-03-29T10:59:12.954 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.953+0000 7faa5359e700 1 -- 172.21.15.111:0/3861789109 learned_addr learned my addr 172.21.15.111:0/3861789109 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:59:12.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.954+0000 7faa51d9b700 1 -- 172.21.15.111:0/3861789109 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1290961045 0 0) 0x7faa540cc9b0 con 0x7faa54071470 2023-03-29T10:59:12.956 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.954+0000 7faa52d9d700 1 --2- 172.21.15.111:0/3861789109 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa54158d30 0x7faa540ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.954+0000 7faa51d9b700 1 -- 172.21.15.111:0/3861789109 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa3c0035b0 con 0x7faa54071470 2023-03-29T10:59:12.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.954+0000 7faa51d9b700 1 -- 172.21.15.111:0/3861789109 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2367243048 0 0) 0x7faa3c0035b0 con 0x7faa54071470 2023-03-29T10:59:12.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.954+0000 7faa51d9b700 1 -- 172.21.15.111:0/3861789109 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa54158d30 msgr2=0x7faa540ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:12.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.954+0000 7faa51d9b700 1 --2- 172.21.15.111:0/3861789109 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa54158d30 0x7faa540ca330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.954+0000 7faa51d9b700 1 -- 172.21.15.111:0/3861789109 >> v1:172.21.15.119:6789/0 conn(0x7faa540cda50 legacy=0x7faa54070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:12.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.954+0000 7faa51d9b700 1 -- 172.21.15.111:0/3861789109 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa540ccf70 con 0x7faa54071470 2023-03-29T10:59:12.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.954+0000 7faa51d9b700 1 -- 172.21.15.111:0/3861789109 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7faa4c0019f0 con 0x7faa54071470 2023-03-29T10:59:12.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.954+0000 7faa51d9b700 1 -- 172.21.15.111:0/3861789109 >> v1:172.21.15.111:6789/0 conn(0x7faa54071470 legacy=0x7faa5406cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:12.957 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.954+0000 7faa51d9b700 1 --2- 172.21.15.111:0/3861789109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c003aa0 0x7faa3c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.954+0000 7faa51d9b700 1 --2- 172.21.15.111:0/3861789109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa3c004850 0x7faa3c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.954+0000 7faa51d9b700 1 --2- 172.21.15.111:0/3861789109 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa54158d30 0x7faa3c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.954+0000 7faa51d9b700 1 -- 172.21.15.111:0/3861789109 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa54071dd0 con 0x7faa3c004850 2023-03-29T10:59:12.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.954+0000 7faa51d9b700 1 -- 172.21.15.111:0/3861789109 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c00baf0 con 0x7faa3c003aa0 2023-03-29T10:59:12.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.954+0000 7faa51d9b700 1 -- 172.21.15.111:0/3861789109 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c00dec0 con 0x7faa54158d30 2023-03-29T10:59:12.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.954+0000 7faa5259c700 1 --2- 172.21.15.111:0/3861789109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa3c004850 0x7faa3c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.955+0000 7faa52d9d700 1 --2- 172.21.15.111:0/3861789109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c003aa0 0x7faa3c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.955+0000 7faa5359e700 1 --2- 172.21.15.111:0/3861789109 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa54158d30 0x7faa3c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.958 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.955+0000 7faa5259c700 1 -- 172.21.15.111:0/3861789109 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa54158d30 msgr2=0x7faa3c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:12.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.955+0000 7faa5259c700 1 --2- 172.21.15.111:0/3861789109 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa54158d30 0x7faa3c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.955+0000 7faa5259c700 1 -- 172.21.15.111:0/3861789109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c003aa0 msgr2=0x7faa3c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:12.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.955+0000 7faa5259c700 1 --2- 172.21.15.111:0/3861789109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa3c003aa0 0x7faa3c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.955+0000 7faa5259c700 1 -- 172.21.15.111:0/3861789109 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c0035b0 con 0x7faa3c004850 2023-03-29T10:59:12.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.955+0000 7faa5259c700 1 --2- 172.21.15.111:0/3861789109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa3c004850 0x7faa3c004cc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7faa480018d0 tx=0x7faa3c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf7bdbbd1b6d951 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:12.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.955+0000 7faa51d9b700 1 -- 172.21.15.111:0/3861789109 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4800c690 con 0x7faa3c004850 2023-03-29T10:59:12.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.955+0000 7faa51d9b700 1 -- 172.21.15.111:0/3861789109 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4800c7f0 con 0x7faa3c004850 2023-03-29T10:59:12.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.955+0000 7faa51d9b700 1 -- 172.21.15.111:0/3861789109 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4800cba0 con 0x7faa3c004850 2023-03-29T10:59:12.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.956+0000 7faa591ff700 1 -- 172.21.15.111:0/3861789109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa3c004850 msgr2=0x7faa3c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:12.959 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.956+0000 7faa591ff700 1 --2- 172.21.15.111:0/3861789109 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa3c004850 0x7faa3c004cc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7faa480018d0 tx=0x7faa3c005880 comp rx=0 tx=0).stop 2023-03-29T10:59:12.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.956+0000 7faa5359e700 1 -- 172.21.15.111:0/3861789109 reap_dead start 2023-03-29T10:59:12.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.956+0000 7faa591ff700 1 -- 172.21.15.111:0/3861789109 shutdown_connections 2023-03-29T10:59:12.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.956+0000 7faa591ff700 1 -- 172.21.15.111:0/3861789109 >> 172.21.15.111:0/3861789109 conn(0x7faa540cb6c0 msgr2=0x7faa540cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:12.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.956+0000 7faa591ff700 1 -- 172.21.15.111:0/3861789109 shutdown_connections 2023-03-29T10:59:12.960 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.956+0000 7faa591ff700 1 -- 172.21.15.111:0/3861789109 wait complete. 2023-03-29T10:59:12.961 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.958+0000 7faa591ff700 1 Processor -- start 2023-03-29T10:59:12.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.958+0000 7faa591ff700 1 -- start start 2023-03-29T10:59:12.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.959+0000 7faa591ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa3c004850 0x7faa541e2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.959+0000 7faa591ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa540c6720 0x7faa540c6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.959+0000 7faa591ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa540c4dd0 0x7faa540c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.959+0000 7faa591ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa540c2be0 con 0x7faa3c004850 2023-03-29T10:59:12.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.959+0000 7faa591ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa541e2aa0 con 0x7faa540c4dd0 2023-03-29T10:59:12.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.959+0000 7faa591ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa54153670 con 0x7faa540c6720 2023-03-29T10:59:12.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.959+0000 7faa52d9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa3c004850 0x7faa541e2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.962 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.959+0000 7faa52d9d700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa3c004850 0x7faa541e2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40298/0 (socket says 172.21.15.111:40298) 2023-03-29T10:59:12.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.959+0000 7faa52d9d700 1 -- 172.21.15.111:0/3577660594 learned_addr learned my addr 172.21.15.111:0/3577660594 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:12.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.959+0000 7faa5259c700 1 --2- 172.21.15.111:0/3577660594 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa540c6720 0x7faa540c6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.959+0000 7faa5359e700 1 --2- 172.21.15.111:0/3577660594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa540c4dd0 0x7faa540c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.959+0000 7faa52d9d700 1 -- 172.21.15.111:0/3577660594 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa540c6720 msgr2=0x7faa540c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:12.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.959+0000 7faa52d9d700 1 --2- 172.21.15.111:0/3577660594 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa540c6720 0x7faa540c6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.959+0000 7faa52d9d700 1 -- 172.21.15.111:0/3577660594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa540c4dd0 msgr2=0x7faa540c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:12.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.959+0000 7faa52d9d700 1 --2- 172.21.15.111:0/3577660594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa540c4dd0 0x7faa540c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:12.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.959+0000 7faa52d9d700 1 -- 172.21.15.111:0/3577660594 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa540cd530 con 0x7faa3c004850 2023-03-29T10:59:12.963 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.959+0000 7faa5259c700 1 --2- 172.21.15.111:0/3577660594 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa540c6720 0x7faa540c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:59:12.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.960+0000 7faa52d9d700 1 --2- 172.21.15.111:0/3577660594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa3c004850 0x7faa541e2560 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7faa44011ad0 tx=0x7faa44011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:12.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.960+0000 7faa3b7fe700 1 -- 172.21.15.111:0/3577660594 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4400fa00 con 0x7faa3c004850 2023-03-29T10:59:12.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.960+0000 7faa3b7fe700 1 -- 172.21.15.111:0/3577660594 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4400fb60 con 0x7faa3c004850 2023-03-29T10:59:12.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.960+0000 7faa591ff700 1 -- 172.21.15.111:0/3577660594 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa540cd250 con 0x7faa3c004850 2023-03-29T10:59:12.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.960+0000 7faa3b7fe700 1 -- 172.21.15.111:0/3577660594 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa44014440 con 0x7faa3c004850 2023-03-29T10:59:12.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.960+0000 7faa591ff700 1 -- 172.21.15.111:0/3577660594 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa540b0730 con 0x7faa3c004850 2023-03-29T10:59:12.964 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.961+0000 7faa591ff700 1 -- 172.21.15.111:0/3577660594 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa54074430 con 0x7faa3c004850 2023-03-29T10:59:12.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.962+0000 7faa3b7fe700 1 -- 172.21.15.111:0/3577660594 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7faa440145a0 con 0x7faa3c004850 2023-03-29T10:59:12.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.963+0000 7faa3b7fe700 1 --2- 172.21.15.111:0/3577660594 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7faa40069de0 0x7faa4006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:12.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.963+0000 7faa3b7fe700 1 -- 172.21.15.111:0/3577660594 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6382+0+0 (secure 0 0 0) 0x7faa44088910 con 0x7faa3c004850 2023-03-29T10:59:12.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.963+0000 7faa5259c700 1 --2- 172.21.15.111:0/3577660594 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7faa40069de0 0x7faa4006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:12.965 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.964+0000 7faa5259c700 1 --2- 172.21.15.111:0/3577660594 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7faa40069de0 0x7faa4006c290 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faa54156a80 tx=0x7faa48005040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:12.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:12.967+0000 7faa3b7fe700 1 -- 172.21.15.111:0/3577660594 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7faa44054140 con 0x7faa3c004850 2023-03-29T10:59:13.162 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.162+0000 7faa591ff700 1 -- 172.21.15.111:0/3577660594 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa540b4bc0 con 0x7faa3c004850 2023-03-29T10:59:13.164 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.163+0000 7faa3b7fe700 1 -- 172.21.15.111:0/3577660594 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18189 (secure 0 0 0) 0x7faa44057760 con 0x7faa3c004850 2023-03-29T10:59:13.164 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:59:13.164 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":42,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:59:12.298098+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:59:10.633156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-03-29T10:59:11.274143+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:59:13.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.165+0000 7faa591ff700 1 -- 172.21.15.111:0/3577660594 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7faa40069de0 msgr2=0x7faa4006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:13.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.165+0000 7faa591ff700 1 --2- 172.21.15.111:0/3577660594 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7faa40069de0 0x7faa4006c290 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faa54156a80 tx=0x7faa48005040 comp rx=0 tx=0).stop 2023-03-29T10:59:13.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.165+0000 7faa591ff700 1 -- 172.21.15.111:0/3577660594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa3c004850 msgr2=0x7faa541e2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:13.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.165+0000 7faa591ff700 1 --2- 172.21.15.111:0/3577660594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa3c004850 0x7faa541e2560 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7faa44011ad0 tx=0x7faa44011e90 comp rx=0 tx=0).stop 2023-03-29T10:59:13.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.166+0000 7faa591ff700 1 -- 172.21.15.111:0/3577660594 shutdown_connections 2023-03-29T10:59:13.166 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.166+0000 7faa591ff700 1 --2- 172.21.15.111:0/3577660594 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7faa40069de0 0x7faa4006c290 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:13.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.166+0000 7faa591ff700 1 --2- 172.21.15.111:0/3577660594 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7faa3c004850 0x7faa541e2560 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:13.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.166+0000 7faa591ff700 1 --2- 172.21.15.111:0/3577660594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa540c4dd0 0x7faa540c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:13.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.166+0000 7faa591ff700 1 --2- 172.21.15.111:0/3577660594 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7faa540c6720 0x7faa540c6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:13.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.166+0000 7faa591ff700 1 -- 172.21.15.111:0/3577660594 >> 172.21.15.111:0/3577660594 conn(0x7faa540cb6c0 msgr2=0x7faa54157280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:13.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.166+0000 7faa591ff700 1 -- 172.21.15.111:0/3577660594 shutdown_connections 2023-03-29T10:59:13.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.166+0000 7faa591ff700 1 -- 172.21.15.111:0/3577660594 wait complete. 2023-03-29T10:59:13.184 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-03-29T10:59:13.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.373+0000 7fad4abff700 1 Processor -- start 2023-03-29T10:59:13.374 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.373+0000 7fad4abff700 1 -- start start 2023-03-29T10:59:13.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.374+0000 7fad4abff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fad440b87a0 0x7fad440cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:13.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.374+0000 7fad4abff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fad441601e0 0x7fad44160630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:13.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.374+0000 7fad4abff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad440cdc60 con 0x7fad44160b70 2023-03-29T10:59:13.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.374+0000 7fad4abff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad440d02d0 con 0x7fad441601e0 2023-03-29T10:59:13.375 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.374+0000 7fad4abff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad440d0cb0 con 0x7fad440b87a0 2023-03-29T10:59:13.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.375+0000 7fad43fff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fad441601e0 0x7fad44160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:13.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.375+0000 7fad4919c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fad44160b70 0x7fad440d2de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38210/0 (socket says 172.21.15.111:38210) 2023-03-29T10:59:13.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.375+0000 7fad4919c700 1 -- 172.21.15.111:0/1657429952 learned_addr learned my addr 172.21.15.111:0/1657429952 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:59:13.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.375+0000 7fad4899b700 1 --2- 172.21.15.111:0/1657429952 >> v2:172.21.15.119:3300/0 conn(0x7fad440b87a0 0x7fad440cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:13.376 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.375+0000 7fad43fff700 1 -- 172.21.15.111:0/1657429952 >> v2:172.21.15.119:3300/0 conn(0x7fad440b87a0 msgr2=0x7fad440cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:13.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.375+0000 7fad43fff700 1 --2- 172.21.15.111:0/1657429952 >> v2:172.21.15.119:3300/0 conn(0x7fad440b87a0 0x7fad440cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:13.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.375+0000 7fad43fff700 1 -- 172.21.15.111:0/1657429952 >> v1:172.21.15.111:6789/0 conn(0x7fad44160b70 legacy=0x7fad440d2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:13.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.375+0000 7fad43fff700 1 -- 172.21.15.111:0/1657429952 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad440cd6a0 con 0x7fad441601e0 2023-03-29T10:59:13.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.375+0000 7fad437fe700 1 -- 172.21.15.111:0/1657429952 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2776511676 0 0) 0x7fad440cdc60 con 0x7fad44160b70 2023-03-29T10:59:13.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.375+0000 7fad43fff700 1 --2- 172.21.15.111:0/1657429952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad441601e0 0x7fad44160630 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fad440b4ac0 tx=0x7fad30009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=915799afd6f9dd81 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:13.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.376+0000 7fad437fe700 1 -- 172.21.15.111:0/1657429952 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad30011830 con 0x7fad441601e0 2023-03-29T10:59:13.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.376+0000 7fad437fe700 1 -- 172.21.15.111:0/1657429952 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad30011990 con 0x7fad441601e0 2023-03-29T10:59:13.377 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.376+0000 7fad437fe700 1 -- 172.21.15.111:0/1657429952 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad30011d20 con 0x7fad441601e0 2023-03-29T10:59:13.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.376+0000 7fad4abff700 1 -- 172.21.15.111:0/1657429952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad441601e0 msgr2=0x7fad44160630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:13.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.376+0000 7fad4abff700 1 --2- 172.21.15.111:0/1657429952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad441601e0 0x7fad44160630 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fad440b4ac0 tx=0x7fad30009b50 comp rx=0 tx=0).stop 2023-03-29T10:59:13.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.376+0000 7fad4abff700 1 -- 172.21.15.111:0/1657429952 shutdown_connections 2023-03-29T10:59:13.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.376+0000 7fad4abff700 1 --2- 172.21.15.111:0/1657429952 >> v2:172.21.15.119:3300/0 conn(0x7fad440b87a0 0x7fad440cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:13.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.376+0000 7fad4abff700 1 --2- 172.21.15.111:0/1657429952 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad441601e0 0x7fad44160630 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:13.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.376+0000 7fad4abff700 1 -- 172.21.15.111:0/1657429952 >> 172.21.15.111:0/1657429952 conn(0x7fad44071b10 msgr2=0x7fad44071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:13.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.376+0000 7fad4abff700 1 -- 172.21.15.111:0/1657429952 shutdown_connections 2023-03-29T10:59:13.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.376+0000 7fad4abff700 1 -- 172.21.15.111:0/1657429952 wait complete. 2023-03-29T10:59:13.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.378+0000 7fad4abff700 1 Processor -- start 2023-03-29T10:59:13.378 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.378+0000 7fad4abff700 1 -- start start 2023-03-29T10:59:13.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.379+0000 7fad4abff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fad440b87a0 0x7fad4415d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:13.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.379+0000 7fad4abff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad441601e0 0x7fad441f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:13.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.379+0000 7fad4abff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad44160b70 0x7fad440b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:13.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.379+0000 7fad4abff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4415e280 con 0x7fad44160b70 2023-03-29T10:59:13.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.379+0000 7fad4abff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4415e560 con 0x7fad441601e0 2023-03-29T10:59:13.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.379+0000 7fad4abff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad4415e840 con 0x7fad440b87a0 2023-03-29T10:59:13.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.379+0000 7fad4919c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad44160b70 0x7fad440b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:13.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.379+0000 7fad4919c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad44160b70 0x7fad440b3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40312/0 (socket says 172.21.15.111:40312) 2023-03-29T10:59:13.380 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.379+0000 7fad4919c700 1 -- 172.21.15.111:0/2927077176 learned_addr learned my addr 172.21.15.111:0/2927077176 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:13.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.380+0000 7fad43fff700 1 --2- 172.21.15.111:0/2927077176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad441601e0 0x7fad441f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:13.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.380+0000 7fad4919c700 1 -- 172.21.15.111:0/2927077176 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fad440b87a0 msgr2=0x7fad4415d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:13.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.380+0000 7fad4919c700 1 --2- 172.21.15.111:0/2927077176 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fad440b87a0 0x7fad4415d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:13.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.380+0000 7fad4919c700 1 -- 172.21.15.111:0/2927077176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad441601e0 msgr2=0x7fad441f1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:13.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.380+0000 7fad4919c700 1 --2- 172.21.15.111:0/2927077176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad441601e0 0x7fad441f1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:13.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.380+0000 7fad4919c700 1 -- 172.21.15.111:0/2927077176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad300094a0 con 0x7fad44160b70 2023-03-29T10:59:13.381 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.380+0000 7fad4919c700 1 --2- 172.21.15.111:0/2927077176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad44160b70 0x7fad440b3960 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00cba0 tx=0x7fad3c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:13.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.380+0000 7fad417fa700 1 -- 172.21.15.111:0/2927077176 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c00ab80 con 0x7fad44160b70 2023-03-29T10:59:13.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.380+0000 7fad417fa700 1 -- 172.21.15.111:0/2927077176 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3c00ad00 con 0x7fad44160b70 2023-03-29T10:59:13.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.380+0000 7fad417fa700 1 -- 172.21.15.111:0/2927077176 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c007310 con 0x7fad44160b70 2023-03-29T10:59:13.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.380+0000 7fad4abff700 1 -- 172.21.15.111:0/2927077176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad44160f90 con 0x7fad44160b70 2023-03-29T10:59:13.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.380+0000 7fad4abff700 1 -- 172.21.15.111:0/2927077176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad440cd980 con 0x7fad44160b70 2023-03-29T10:59:13.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.381+0000 7fad4abff700 1 -- 172.21.15.111:0/2927077176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad440d2110 con 0x7fad44160b70 2023-03-29T10:59:13.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.382+0000 7fad417fa700 1 -- 172.21.15.111:0/2927077176 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fad3c007470 con 0x7fad44160b70 2023-03-29T10:59:13.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.383+0000 7fad417fa700 1 --2- 172.21.15.111:0/2927077176 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fad28069de0 0x7fad2806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:13.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.383+0000 7fad417fa700 1 -- 172.21.15.111:0/2927077176 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fad3c087bc0 con 0x7fad44160b70 2023-03-29T10:59:13.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.383+0000 7fad4899b700 1 --2- 172.21.15.111:0/2927077176 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fad28069de0 0x7fad2806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:13.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.384+0000 7fad4899b700 1 --2- 172.21.15.111:0/2927077176 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fad28069de0 0x7fad2806c290 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fad380097f0 tx=0x7fad38008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:13.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.387+0000 7fad417fa700 1 -- 172.21.15.111:0/2927077176 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fad3c053320 con 0x7fad44160b70 2023-03-29T10:59:13.626 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:13.626+0000 7fad4abff700 1 -- 172.21.15.111:0/2927077176 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fad44074430 con 0x7fad44160b70 2023-03-29T10:59:14.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.308+0000 7fad417fa700 1 -- 172.21.15.111:0/2927077176 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fad3c0180b0 con 0x7fad44160b70 2023-03-29T10:59:14.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.311+0000 7fad4abff700 1 -- 172.21.15.111:0/2927077176 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fad28069de0 msgr2=0x7fad2806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:14.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.311+0000 7fad4abff700 1 --2- 172.21.15.111:0/2927077176 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fad28069de0 0x7fad2806c290 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fad380097f0 tx=0x7fad38008040 comp rx=0 tx=0).stop 2023-03-29T10:59:14.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.311+0000 7fad4abff700 1 -- 172.21.15.111:0/2927077176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad44160b70 msgr2=0x7fad440b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:14.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.311+0000 7fad4abff700 1 --2- 172.21.15.111:0/2927077176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad44160b70 0x7fad440b3960 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00cba0 tx=0x7fad3c00cf60 comp rx=0 tx=0).stop 2023-03-29T10:59:14.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.311+0000 7fad4abff700 1 -- 172.21.15.111:0/2927077176 shutdown_connections 2023-03-29T10:59:14.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.311+0000 7fad4abff700 1 --2- 172.21.15.111:0/2927077176 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fad28069de0 0x7fad2806c290 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:14.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.311+0000 7fad4abff700 1 --2- 172.21.15.111:0/2927077176 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fad44160b70 0x7fad440b3960 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:14.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.311+0000 7fad4abff700 1 --2- 172.21.15.111:0/2927077176 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fad440b87a0 0x7fad4415d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:14.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.311+0000 7fad4abff700 1 --2- 172.21.15.111:0/2927077176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad441601e0 0x7fad441f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:14.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.311+0000 7fad4abff700 1 -- 172.21.15.111:0/2927077176 >> 172.21.15.111:0/2927077176 conn(0x7fad44071b10 msgr2=0x7fad4415f300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:14.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.311+0000 7fad4abff700 1 -- 172.21.15.111:0/2927077176 shutdown_connections 2023-03-29T10:59:14.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.311+0000 7fad4abff700 1 -- 172.21.15.111:0/2927077176 wait complete. 2023-03-29T10:59:14.330 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-03-29T10:59:14.331 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-03-29T10:59:14.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.519+0000 7f40167ff700 1 Processor -- start 2023-03-29T10:59:14.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.520+0000 7f40167ff700 1 -- start start 2023-03-29T10:59:14.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.521+0000 7f40167ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f40100cfbf0 0x7f40101608f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:14.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.521+0000 7f40167ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f40100d1100 0x7f40100d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:14.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.521+0000 7f40167ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40100d0020 con 0x7f40100b87a0 2023-03-29T10:59:14.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.521+0000 7f40167ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40100d02d0 con 0x7f40100d1100 2023-03-29T10:59:14.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.521+0000 7f40167ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40100d0cb0 con 0x7f40100cfbf0 2023-03-29T10:59:14.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.521+0000 7f400ffff700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f40100b87a0 0x7f40101601e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38218/0 (socket says 172.21.15.111:38218) 2023-03-29T10:59:14.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.521+0000 7f400ffff700 1 -- 172.21.15.111:0/4188086502 learned_addr learned my addr 172.21.15.111:0/4188086502 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:59:14.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.521+0000 7f4014d9c700 1 --2- 172.21.15.111:0/4188086502 >> v2:172.21.15.111:3300/0 conn(0x7f40100d1100 0x7f40100d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:14.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.521+0000 7f400f7fe700 1 --2- 172.21.15.111:0/4188086502 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f40100cfbf0 0x7f40101608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:14.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.521+0000 7f400effd700 1 -- 172.21.15.111:0/4188086502 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201807235 0 0) 0x7f40100d0020 con 0x7f40100b87a0 2023-03-29T10:59:14.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.521+0000 7f400effd700 1 -- 172.21.15.111:0/4188086502 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ff80035b0 con 0x7f40100b87a0 2023-03-29T10:59:14.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.522+0000 7f4014d9c700 1 -- 172.21.15.111:0/4188086502 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f40100cfbf0 msgr2=0x7f40101608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:14.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.522+0000 7f4014d9c700 1 --2- 172.21.15.111:0/4188086502 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f40100cfbf0 0x7f40101608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:14.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.522+0000 7f4014d9c700 1 -- 172.21.15.111:0/4188086502 >> v1:172.21.15.111:6789/0 conn(0x7f40100b87a0 legacy=0x7f40101601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.522+0000 7f4014d9c700 1 -- 172.21.15.111:0/4188086502 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40100cdc60 con 0x7f40100d1100 2023-03-29T10:59:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.522+0000 7f4014d9c700 1 --2- 172.21.15.111:0/4188086502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40100d1100 0x7f40100d1580 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4008009d90 tx=0x7f400800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd8e1e1504bfb3 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.522+0000 7f400effd700 1 -- 172.21.15.111:0/4188086502 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400800e070 con 0x7f40100d1100 2023-03-29T10:59:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.522+0000 7f400effd700 1 -- 172.21.15.111:0/4188086502 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400800bb20 con 0x7f40100d1100 2023-03-29T10:59:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.522+0000 7f400effd700 1 -- 172.21.15.111:0/4188086502 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4008012430 con 0x7f40100d1100 2023-03-29T10:59:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.523+0000 7f40167ff700 1 -- 172.21.15.111:0/4188086502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40100d1100 msgr2=0x7f40100d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.523+0000 7f40167ff700 1 --2- 172.21.15.111:0/4188086502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40100d1100 0x7f40100d1580 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4008009d90 tx=0x7f400800b330 comp rx=0 tx=0).stop 2023-03-29T10:59:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.523+0000 7f40167ff700 1 -- 172.21.15.111:0/4188086502 shutdown_connections 2023-03-29T10:59:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.523+0000 7f40167ff700 1 --2- 172.21.15.111:0/4188086502 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40100d1100 0x7f40100d1580 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.523+0000 7f40167ff700 1 --2- 172.21.15.111:0/4188086502 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f40100cfbf0 0x7f40101608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.523+0000 7f40167ff700 1 -- 172.21.15.111:0/4188086502 >> 172.21.15.111:0/4188086502 conn(0x7f4010071b10 msgr2=0x7f4010071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.523+0000 7f40167ff700 1 -- 172.21.15.111:0/4188086502 shutdown_connections 2023-03-29T10:59:14.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.523+0000 7f40167ff700 1 -- 172.21.15.111:0/4188086502 wait complete. 2023-03-29T10:59:14.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.524+0000 7f40167ff700 1 Processor -- start 2023-03-29T10:59:14.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.525+0000 7f40167ff700 1 -- start start 2023-03-29T10:59:14.527 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.525+0000 7f40167ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40100b87a0 0x7f40100caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:14.527 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.525+0000 7f40167ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f40100cfbf0 0x7f401015f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:14.527 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.526+0000 7f40167ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40100d1100 0x7f401014a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:14.527 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.526+0000 7f40167ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40101585e0 con 0x7f40100b87a0 2023-03-29T10:59:14.527 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.526+0000 7f40167ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401015fa30 con 0x7f40100d1100 2023-03-29T10:59:14.527 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.526+0000 7f40167ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40101587f0 con 0x7f40100cfbf0 2023-03-29T10:59:14.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.526+0000 7f400ffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40100b87a0 0x7f40100caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:14.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.526+0000 7f400ffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40100b87a0 0x7f40100caf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40332/0 (socket says 172.21.15.111:40332) 2023-03-29T10:59:14.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.526+0000 7f400ffff700 1 -- 172.21.15.111:0/4191852405 learned_addr learned my addr 172.21.15.111:0/4191852405 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:14.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.526+0000 7f400f7fe700 1 --2- 172.21.15.111:0/4191852405 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f40100cfbf0 0x7f401015f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:14.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.526+0000 7f4014d9c700 1 --2- 172.21.15.111:0/4191852405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40100d1100 0x7f401014a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:14.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.526+0000 7f400ffff700 1 -- 172.21.15.111:0/4191852405 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f40100cfbf0 msgr2=0x7f401015f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:14.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.526+0000 7f400ffff700 1 --2- 172.21.15.111:0/4191852405 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f40100cfbf0 0x7f401015f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:14.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.526+0000 7f400ffff700 1 -- 172.21.15.111:0/4191852405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40100d1100 msgr2=0x7f401014a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:14.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.526+0000 7f400ffff700 1 --2- 172.21.15.111:0/4191852405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40100d1100 0x7f401014a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:14.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.526+0000 7f400ffff700 1 -- 172.21.15.111:0/4191852405 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4008009a70 con 0x7f40100b87a0 2023-03-29T10:59:14.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.526+0000 7f400ffff700 1 --2- 172.21.15.111:0/4191852405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40100b87a0 0x7f40100caf00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4000001eb0 tx=0x7f400000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:14.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.527+0000 7f400cff9700 1 -- 172.21.15.111:0/4191852405 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4000005d70 con 0x7f40100b87a0 2023-03-29T10:59:14.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.527+0000 7f400cff9700 1 -- 172.21.15.111:0/4191852405 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40000070d0 con 0x7f40100b87a0 2023-03-29T10:59:14.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.527+0000 7f400cff9700 1 -- 172.21.15.111:0/4191852405 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4000007460 con 0x7f40100b87a0 2023-03-29T10:59:14.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.527+0000 7f40167ff700 1 -- 172.21.15.111:0/4191852405 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40100cd980 con 0x7f40100b87a0 2023-03-29T10:59:14.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.527+0000 7f40167ff700 1 -- 172.21.15.111:0/4191852405 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40100cd6a0 con 0x7f40100b87a0 2023-03-29T10:59:14.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.528+0000 7f40167ff700 1 -- 172.21.15.111:0/4191852405 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4010160e30 con 0x7f40100b87a0 2023-03-29T10:59:14.529 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.529+0000 7f400cff9700 1 -- 172.21.15.111:0/4191852405 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f40000075c0 con 0x7f40100b87a0 2023-03-29T10:59:14.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.529+0000 7f400cff9700 1 --2- 172.21.15.111:0/4191852405 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3ffc069de0 0x7f3ffc06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:14.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.530+0000 7f400cff9700 1 -- 172.21.15.111:0/4191852405 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4000087730 con 0x7f40100b87a0 2023-03-29T10:59:14.530 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.530+0000 7f400f7fe700 1 --2- 172.21.15.111:0/4191852405 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3ffc069de0 0x7f3ffc06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:14.531 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.530+0000 7f400f7fe700 1 --2- 172.21.15.111:0/4191852405 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3ffc069de0 0x7f3ffc06c290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f40100ca820 tx=0x7f4004008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:14.534 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.533+0000 7f400cff9700 1 -- 172.21.15.111:0/4191852405 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f4000052e00 con 0x7f40100b87a0 2023-03-29T10:59:14.769 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:14.769+0000 7f40167ff700 1 -- 172.21.15.111:0/4191852405 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4010074430 con 0x7f40100b87a0 2023-03-29T10:59:14.796 INFO:tasks.ceph.mds.c.smithi111.stderr:2023-03-29T10:59:14.795+0000 7f1559df1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-03-29T10:59:15.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.309+0000 7f400cff9700 1 -- 172.21.15.111:0/4191852405 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4000056420 con 0x7f40100b87a0 2023-03-29T10:59:15.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.312+0000 7f40167ff700 1 -- 172.21.15.111:0/4191852405 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3ffc069de0 msgr2=0x7f3ffc06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:15.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.312+0000 7f40167ff700 1 --2- 172.21.15.111:0/4191852405 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3ffc069de0 0x7f3ffc06c290 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f40100ca820 tx=0x7f4004008040 comp rx=0 tx=0).stop 2023-03-29T10:59:15.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.312+0000 7f40167ff700 1 -- 172.21.15.111:0/4191852405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40100b87a0 msgr2=0x7f40100caf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:15.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.312+0000 7f40167ff700 1 --2- 172.21.15.111:0/4191852405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40100b87a0 0x7f40100caf00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4000001eb0 tx=0x7f400000c860 comp rx=0 tx=0).stop 2023-03-29T10:59:15.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.312+0000 7f40167ff700 1 -- 172.21.15.111:0/4191852405 shutdown_connections 2023-03-29T10:59:15.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.312+0000 7f40167ff700 1 --2- 172.21.15.111:0/4191852405 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f3ffc069de0 0x7f3ffc06c290 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:15.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.312+0000 7f40167ff700 1 --2- 172.21.15.111:0/4191852405 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f40100b87a0 0x7f40100caf00 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:15.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.312+0000 7f40167ff700 1 --2- 172.21.15.111:0/4191852405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40100d1100 0x7f401014a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:15.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.312+0000 7f40167ff700 1 --2- 172.21.15.111:0/4191852405 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f40100cfbf0 0x7f401015f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:15.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.312+0000 7f40167ff700 1 -- 172.21.15.111:0/4191852405 >> 172.21.15.111:0/4191852405 conn(0x7f4010071b10 msgr2=0x7f4010071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:15.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.312+0000 7f40167ff700 1 -- 172.21.15.111:0/4191852405 shutdown_connections 2023-03-29T10:59:15.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.312+0000 7f40167ff700 1 -- 172.21.15.111:0/4191852405 wait complete. 2023-03-29T10:59:15.331 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-03-29T10:59:15.331 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-29T10:59:15.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.514+0000 7f93463ff700 1 Processor -- start 2023-03-29T10:59:15.515 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.514+0000 7f93463ff700 1 -- start start 2023-03-29T10:59:15.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.515+0000 7f93463ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f93400cfd40 0x7f93400d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:15.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.515+0000 7f93463ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f93400b70f0 0x7f93400b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:15.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.515+0000 7f93463ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93400cdc10 con 0x7f93400cf970 2023-03-29T10:59:15.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.515+0000 7f93463ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93400b6c60 con 0x7f93400cfd40 2023-03-29T10:59:15.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.515+0000 7f93463ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93400cd650 con 0x7f93400b70f0 2023-03-29T10:59:15.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.515+0000 7f933f7fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f93400cfd40 0x7f93400d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:15.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.516+0000 7f933f7fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f93400cfd40 0x7f93400d01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40348/0 (socket says 172.21.15.111:40348) 2023-03-29T10:59:15.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.516+0000 7f933f7fe700 1 -- 172.21.15.111:0/150628869 learned_addr learned my addr 172.21.15.111:0/150628869 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:15.516 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.516+0000 7f934499c700 1 --2- 172.21.15.111:0/150628869 >> v2:172.21.15.119:3300/0 conn(0x7f93400b70f0 0x7f93400b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:15.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.516+0000 7f933f7fe700 1 -- 172.21.15.111:0/150628869 >> v2:172.21.15.119:3300/0 conn(0x7f93400b70f0 msgr2=0x7f93400b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:15.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.516+0000 7f933f7fe700 1 --2- 172.21.15.111:0/150628869 >> v2:172.21.15.119:3300/0 conn(0x7f93400b70f0 0x7f93400b74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:15.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.516+0000 7f933f7fe700 1 -- 172.21.15.111:0/150628869 >> v1:172.21.15.119:6789/0 conn(0x7f93400cf970 legacy=0x7f93400cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:15.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.516+0000 7f933f7fe700 1 -- 172.21.15.111:0/150628869 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93400cd930 con 0x7f93400cfd40 2023-03-29T10:59:15.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.516+0000 7f933f7fe700 1 --2- 172.21.15.111:0/150628869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93400cfd40 0x7f93400d01c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9340161330 tx=0x7f9334009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5683ea8a83c3b4 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:15.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.516+0000 7f933effd700 1 -- 172.21.15.111:0/150628869 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334011830 con 0x7f93400cfd40 2023-03-29T10:59:15.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.516+0000 7f933effd700 1 -- 172.21.15.111:0/150628869 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9334011990 con 0x7f93400cfd40 2023-03-29T10:59:15.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.516+0000 7f933effd700 1 -- 172.21.15.111:0/150628869 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334011d20 con 0x7f93400cfd40 2023-03-29T10:59:15.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.517+0000 7f93463ff700 1 -- 172.21.15.111:0/150628869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93400cfd40 msgr2=0x7f93400d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:15.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.517+0000 7f93463ff700 1 --2- 172.21.15.111:0/150628869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93400cfd40 0x7f93400d01c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9340161330 tx=0x7f9334009bd0 comp rx=0 tx=0).stop 2023-03-29T10:59:15.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.517+0000 7f93463ff700 1 -- 172.21.15.111:0/150628869 shutdown_connections 2023-03-29T10:59:15.518 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.517+0000 7f93463ff700 1 --2- 172.21.15.111:0/150628869 >> v2:172.21.15.119:3300/0 conn(0x7f93400b70f0 0x7f93400b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:15.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.517+0000 7f93463ff700 1 --2- 172.21.15.111:0/150628869 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93400cfd40 0x7f93400d01c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:15.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.517+0000 7f93463ff700 1 -- 172.21.15.111:0/150628869 >> 172.21.15.111:0/150628869 conn(0x7f9340071b10 msgr2=0x7f9340071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:15.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.517+0000 7f93463ff700 1 -- 172.21.15.111:0/150628869 shutdown_connections 2023-03-29T10:59:15.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.517+0000 7f93463ff700 1 -- 172.21.15.111:0/150628869 wait complete. 2023-03-29T10:59:15.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.519+0000 7f93463ff700 1 Processor -- start 2023-03-29T10:59:15.519 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.519+0000 7f93463ff700 1 -- start start 2023-03-29T10:59:15.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.520+0000 7f93463ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93400b70f0 0x7f934014b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:15.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.520+0000 7f93463ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93400cf970 0x7f934015ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:15.520 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.520+0000 7f93463ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93400cfd40 0x7f9340160510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:15.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.520+0000 7f93463ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93401f21c0 con 0x7f93400cf970 2023-03-29T10:59:15.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.520+0000 7f93463ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93401f2d00 con 0x7f93400b70f0 2023-03-29T10:59:15.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.520+0000 7f93463ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93401f3190 con 0x7f93400cfd40 2023-03-29T10:59:15.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.520+0000 7f933f7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93400cf970 0x7f934015ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:15.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.520+0000 7f933f7fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93400cf970 0x7f934015ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40364/0 (socket says 172.21.15.111:40364) 2023-03-29T10:59:15.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.520+0000 7f933f7fe700 1 -- 172.21.15.111:0/2726765795 learned_addr learned my addr 172.21.15.111:0/2726765795 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:15.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.520+0000 7f934499c700 1 --2- 172.21.15.111:0/2726765795 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93400cfd40 0x7f9340160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:15.521 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.520+0000 7f933ffff700 1 --2- 172.21.15.111:0/2726765795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93400b70f0 0x7f934014b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:15.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.520+0000 7f933f7fe700 1 -- 172.21.15.111:0/2726765795 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93400cfd40 msgr2=0x7f9340160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:15.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.520+0000 7f933f7fe700 1 --2- 172.21.15.111:0/2726765795 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93400cfd40 0x7f9340160510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:15.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.520+0000 7f933f7fe700 1 -- 172.21.15.111:0/2726765795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93400b70f0 msgr2=0x7f934014b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:15.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.520+0000 7f933f7fe700 1 --2- 172.21.15.111:0/2726765795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93400b70f0 0x7f934014b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:15.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.520+0000 7f933f7fe700 1 -- 172.21.15.111:0/2726765795 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9334009550 con 0x7f93400cf970 2023-03-29T10:59:15.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.521+0000 7f933f7fe700 1 --2- 172.21.15.111:0/2726765795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93400cf970 0x7f934015ffd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9334009c10 tx=0x7f9334004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:15.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.521+0000 7f933cff9700 1 -- 172.21.15.111:0/2726765795 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334004380 con 0x7f93400cf970 2023-03-29T10:59:15.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.521+0000 7f933cff9700 1 -- 172.21.15.111:0/2726765795 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933401f070 con 0x7f93400cf970 2023-03-29T10:59:15.522 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.521+0000 7f933cff9700 1 -- 172.21.15.111:0/2726765795 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93340036a0 con 0x7f93400cf970 2023-03-29T10:59:15.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.521+0000 7f93463ff700 1 -- 172.21.15.111:0/2726765795 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93400d0700 con 0x7f93400cf970 2023-03-29T10:59:15.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.521+0000 7f93463ff700 1 -- 172.21.15.111:0/2726765795 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93400b68f0 con 0x7f93400cf970 2023-03-29T10:59:15.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.522+0000 7f93463ff700 1 -- 172.21.15.111:0/2726765795 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93400d36d0 con 0x7f93400cf970 2023-03-29T10:59:15.523 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.523+0000 7f933cff9700 1 -- 172.21.15.111:0/2726765795 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f9334021030 con 0x7f93400cf970 2023-03-29T10:59:15.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.524+0000 7f933cff9700 1 --2- 172.21.15.111:0/2726765795 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f932c069de0 0x7f932c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:15.524 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.524+0000 7f933cff9700 1 -- 172.21.15.111:0/2726765795 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9334089420 con 0x7f93400cf970 2023-03-29T10:59:15.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.524+0000 7f933ffff700 1 --2- 172.21.15.111:0/2726765795 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f932c069de0 0x7f932c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:15.525 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.525+0000 7f933ffff700 1 --2- 172.21.15.111:0/2726765795 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f932c069de0 0x7f932c06c290 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f93400ce940 tx=0x7f9330009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:15.528 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.528+0000 7f933cff9700 1 -- 172.21.15.111:0/2726765795 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f9334054b80 con 0x7f93400cf970 2023-03-29T10:59:15.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.763+0000 7f93463ff700 1 -- 172.21.15.111:0/2726765795 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93400cc4a0 con 0x7f93400cf970 2023-03-29T10:59:15.764 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.764+0000 7f933cff9700 1 -- 172.21.15.111:0/2726765795 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4683 (secure 0 0 0) 0x7f93340581a0 con 0x7f93400cf970 2023-03-29T10:59:15.764 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:59:15.765 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5021,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/1004725536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1004725536},{"type":"v1","addr":"172.21.15.111:6835","nonce":1004725536}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5341,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2981418361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":2981418361},{"type":"v1","addr":"172.21.15.119:6835","nonce":2981418361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/4270377263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":4270377263},{"type":"v1","addr":"172.21.15.119:6834","nonce":4270377263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-29T10:59:12.298455+0000","modified":"2023-03-29T10:59:15.307952+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6837/3450732660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3450732660},{"type":"v1","addr":"172.21.15.111:6837","nonce":3450732660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-03-29T10:59:15.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.766+0000 7f93463ff700 1 -- 172.21.15.111:0/2726765795 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f932c069de0 msgr2=0x7f932c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:15.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.766+0000 7f93463ff700 1 --2- 172.21.15.111:0/2726765795 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f932c069de0 0x7f932c06c290 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f93400ce940 tx=0x7f9330009040 comp rx=0 tx=0).stop 2023-03-29T10:59:15.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.766+0000 7f93463ff700 1 -- 172.21.15.111:0/2726765795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93400cf970 msgr2=0x7f934015ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:15.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.766+0000 7f93463ff700 1 --2- 172.21.15.111:0/2726765795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93400cf970 0x7f934015ffd0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9334009c10 tx=0x7f9334004750 comp rx=0 tx=0).stop 2023-03-29T10:59:15.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.766+0000 7f93463ff700 1 -- 172.21.15.111:0/2726765795 shutdown_connections 2023-03-29T10:59:15.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.766+0000 7f93463ff700 1 --2- 172.21.15.111:0/2726765795 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f932c069de0 0x7f932c06c290 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:15.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.766+0000 7f93463ff700 1 --2- 172.21.15.111:0/2726765795 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f93400cfd40 0x7f9340160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:15.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.766+0000 7f93463ff700 1 --2- 172.21.15.111:0/2726765795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93400b70f0 0x7f934014b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:15.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.766+0000 7f93463ff700 1 --2- 172.21.15.111:0/2726765795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f93400cf970 0x7f934015ffd0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:15.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.766+0000 7f93463ff700 1 -- 172.21.15.111:0/2726765795 >> 172.21.15.111:0/2726765795 conn(0x7f9340071b10 msgr2=0x7f93400d2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:15.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.766+0000 7f93463ff700 1 -- 172.21.15.111:0/2726765795 shutdown_connections 2023-03-29T10:59:15.767 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.766+0000 7f93463ff700 1 -- 172.21.15.111:0/2726765795 wait complete. 2023-03-29T10:59:15.768 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 17 2023-03-29T10:59:15.784 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-03-29T10:59:15.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.967+0000 7f11eb3ff700 1 Processor -- start 2023-03-29T10:59:15.968 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.967+0000 7f11eb3ff700 1 -- start start 2023-03-29T10:59:15.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.968+0000 7f11eb3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f11e40b0dd0 0x7f11e40c9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:15.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.968+0000 7f11eb3ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f11e40c97d0 0x7f11e40c2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:15.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.969+0000 7f11eb3ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e40b16b0 con 0x7f11e4143f10 2023-03-29T10:59:15.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.969+0000 7f11eb3ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e415ac50 con 0x7f11e40c97d0 2023-03-29T10:59:15.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.969+0000 7f11eb3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e40c9c50 con 0x7f11e40b0dd0 2023-03-29T10:59:15.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.969+0000 7f11e919b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f11e4143f10 0x7f11e40b06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38230/0 (socket says 172.21.15.111:38230) 2023-03-29T10:59:15.969 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.969+0000 7f11e999c700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f11e40c97d0 0x7f11e40c2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:15.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.969+0000 7f11e919b700 1 -- 172.21.15.111:0/2389958661 learned_addr learned my addr 172.21.15.111:0/2389958661 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:59:15.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.969+0000 7f11e899a700 1 --2- 172.21.15.111:0/2389958661 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f11e40b0dd0 0x7f11e40c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:15.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.969+0000 7f11e999c700 1 -- 172.21.15.111:0/2389958661 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f11e40b0dd0 msgr2=0x7f11e40c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:15.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.969+0000 7f11e999c700 1 --2- 172.21.15.111:0/2389958661 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f11e40b0dd0 0x7f11e40c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:15.970 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.969+0000 7f11e999c700 1 -- 172.21.15.111:0/2389958661 >> v1:172.21.15.111:6789/0 conn(0x7f11e4143f10 legacy=0x7f11e40b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:15.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.969+0000 7f11e999c700 1 -- 172.21.15.111:0/2389958661 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e406d970 con 0x7f11e40c97d0 2023-03-29T10:59:15.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.970+0000 7f11e999c700 1 --2- 172.21.15.111:0/2389958661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11e40c97d0 0x7f11e40c2a70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009d90 tx=0x7f11dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fadeed8e997ae server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:15.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.970+0000 7f11dbfff700 1 -- 172.21.15.111:0/2389958661 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc00e070 con 0x7f11e40c97d0 2023-03-29T10:59:15.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.970+0000 7f11dbfff700 1 -- 172.21.15.111:0/2389958661 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11dc00bb20 con 0x7f11e40c97d0 2023-03-29T10:59:15.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.970+0000 7f11dbfff700 1 -- 172.21.15.111:0/2389958661 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc012430 con 0x7f11e40c97d0 2023-03-29T10:59:15.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.970+0000 7f11eb3ff700 1 -- 172.21.15.111:0/2389958661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11e40c97d0 msgr2=0x7f11e40c2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:15.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.970+0000 7f11eb3ff700 1 --2- 172.21.15.111:0/2389958661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11e40c97d0 0x7f11e40c2a70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009d90 tx=0x7f11dc00b330 comp rx=0 tx=0).stop 2023-03-29T10:59:15.971 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.970+0000 7f11eb3ff700 1 -- 172.21.15.111:0/2389958661 shutdown_connections 2023-03-29T10:59:15.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.970+0000 7f11eb3ff700 1 --2- 172.21.15.111:0/2389958661 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11e40c97d0 0x7f11e40c2a70 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:15.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.970+0000 7f11eb3ff700 1 --2- 172.21.15.111:0/2389958661 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f11e40b0dd0 0x7f11e40c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:15.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.970+0000 7f11eb3ff700 1 -- 172.21.15.111:0/2389958661 >> 172.21.15.111:0/2389958661 conn(0x7f11e40cb6c0 msgr2=0x7f11e40cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:15.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.970+0000 7f11eb3ff700 1 -- 172.21.15.111:0/2389958661 shutdown_connections 2023-03-29T10:59:15.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.971+0000 7f11eb3ff700 1 -- 172.21.15.111:0/2389958661 wait complete. 2023-03-29T10:59:15.972 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.972+0000 7f11eb3ff700 1 Processor -- start 2023-03-29T10:59:15.973 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.972+0000 7f11eb3ff700 1 -- start start 2023-03-29T10:59:15.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.973+0000 7f11eb3ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11e40b0dd0 0x7f11e40b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:15.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.973+0000 7f11eb3ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f11e40c97d0 0x7f11e41f0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:15.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.973+0000 7f11eb3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e4143f10 0x7f11e415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:15.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.973+0000 7f11eb3ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e40c66a0 con 0x7f11e40b0dd0 2023-03-29T10:59:15.974 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.973+0000 7f11eb3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e40c5db0 con 0x7f11e4143f10 2023-03-29T10:59:15.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.973+0000 7f11eb3ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e415b2d0 con 0x7f11e40c97d0 2023-03-29T10:59:15.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.973+0000 7f11e919b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11e40b0dd0 0x7f11e40b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:15.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.973+0000 7f11e919b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11e40b0dd0 0x7f11e40b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40376/0 (socket says 172.21.15.111:40376) 2023-03-29T10:59:15.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.973+0000 7f11e919b700 1 -- 172.21.15.111:0/2336718100 learned_addr learned my addr 172.21.15.111:0/2336718100 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:15.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.974+0000 7f11e899a700 1 --2- 172.21.15.111:0/2336718100 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f11e40c97d0 0x7f11e41f0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:15.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.974+0000 7f11e999c700 1 --2- 172.21.15.111:0/2336718100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e4143f10 0x7f11e415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:15.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.974+0000 7f11e919b700 1 -- 172.21.15.111:0/2336718100 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f11e40c97d0 msgr2=0x7f11e41f0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:15.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.974+0000 7f11e919b700 1 --2- 172.21.15.111:0/2336718100 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f11e40c97d0 0x7f11e41f0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:15.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.974+0000 7f11e919b700 1 -- 172.21.15.111:0/2336718100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e4143f10 msgr2=0x7f11e415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:15.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.974+0000 7f11e919b700 1 --2- 172.21.15.111:0/2336718100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e4143f10 0x7f11e415ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:15.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.974+0000 7f11e919b700 1 -- 172.21.15.111:0/2336718100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11dc009a70 con 0x7f11e40b0dd0 2023-03-29T10:59:15.975 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.974+0000 7f11e919b700 1 --2- 172.21.15.111:0/2336718100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11e40b0dd0 0x7f11e40b04e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f11e000c520 tx=0x7f11e000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:15.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.974+0000 7f11d9ffb700 1 -- 172.21.15.111:0/2336718100 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e0005d70 con 0x7f11e40b0dd0 2023-03-29T10:59:15.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.974+0000 7f11d9ffb700 1 -- 172.21.15.111:0/2336718100 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e00070d0 con 0x7f11e40b0dd0 2023-03-29T10:59:15.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.974+0000 7f11eb3ff700 1 -- 172.21.15.111:0/2336718100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e406df30 con 0x7f11e40b0dd0 2023-03-29T10:59:15.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.974+0000 7f11d9ffb700 1 -- 172.21.15.111:0/2336718100 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e0007440 con 0x7f11e40b0dd0 2023-03-29T10:59:15.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.974+0000 7f11eb3ff700 1 -- 172.21.15.111:0/2336718100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e406dc50 con 0x7f11e40b0dd0 2023-03-29T10:59:15.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.975+0000 7f11eb3ff700 1 -- 172.21.15.111:0/2336718100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e406e210 con 0x7f11e40b0dd0 2023-03-29T10:59:15.976 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.976+0000 7f11d9ffb700 1 -- 172.21.15.111:0/2336718100 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f11e00075a0 con 0x7f11e40b0dd0 2023-03-29T10:59:15.977 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.977+0000 7f11d9ffb700 1 --2- 172.21.15.111:0/2336718100 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f11c8069de0 0x7f11c806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:15.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.977+0000 7f11d9ffb700 1 -- 172.21.15.111:0/2336718100 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f11e0087550 con 0x7f11e40b0dd0 2023-03-29T10:59:15.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.977+0000 7f11e899a700 1 --2- 172.21.15.111:0/2336718100 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f11c8069de0 0x7f11c806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:15.978 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.978+0000 7f11e899a700 1 --2- 172.21.15.111:0/2336718100 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f11c8069de0 0x7f11c806c290 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f11e41574e0 tx=0x7f11d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:15.981 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:15.981+0000 7f11d9ffb700 1 -- 172.21.15.111:0/2336718100 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f11e004c160 con 0x7f11e40b0dd0 2023-03-29T10:59:16.177 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.176+0000 7f11eb3ff700 1 -- 172.21.15.111:0/2336718100 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11e4074430 con 0x7f11e40b0dd0 2023-03-29T10:59:16.178 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.178+0000 7f11d9ffb700 1 -- 172.21.15.111:0/2336718100 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+18176 (secure 0 0 0) 0x7f11e0052cd0 con 0x7f11e40b0dd0 2023-03-29T10:59:16.179 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:59:16.179 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":43,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","created":"2023-03-29T10:57:01.498338+0000","modified":"2023-03-29T10:59:13.301418+0000","last_up_change":"2023-03-29T10:57:14.257868+0000","last_in_change":"2023-03-29T10:57:04.868340+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-03-29T10:57:16.183979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-03-29T10:57:21.239809+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-03-29T10:59:10.633156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-03-29T10:59:11.274143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"07696850-0517-429c-be4a-d6f8c2cf3d01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6816","nonce":110484},{"type":"v1","addr":"172.21.15.111:6817","nonce":110484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6818","nonce":110484},{"type":"v1","addr":"172.21.15.111:6819","nonce":110484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6822","nonce":110484},{"type":"v1","addr":"172.21.15.111:6823","nonce":110484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6820","nonce":110484},{"type":"v1","addr":"172.21.15.111:6821","nonce":110484}]},"public_addr":"172.21.15.111:6817/110484","cluster_addr":"172.21.15.111:6819/110484","heartbeat_back_addr":"172.21.15.111:6823/110484","heartbeat_front_addr":"172.21.15.111:6821/110484","state":["exists","up"]},{"osd":1,"uuid":"818f1f56-1dcf-4d4c-870f-5ca3c9830c45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6808","nonce":110483},{"type":"v1","addr":"172.21.15.111:6809","nonce":110483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6810","nonce":110483},{"type":"v1","addr":"172.21.15.111:6811","nonce":110483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6814","nonce":110483},{"type":"v1","addr":"172.21.15.111:6815","nonce":110483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6812","nonce":110483},{"type":"v1","addr":"172.21.15.111:6813","nonce":110483}]},"public_addr":"172.21.15.111:6809/110483","cluster_addr":"172.21.15.111:6811/110483","heartbeat_back_addr":"172.21.15.111:6815/110483","heartbeat_front_addr":"172.21.15.111:6813/110483","state":["exists","up"]},{"osd":2,"uuid":"7943699f-e24a-4905-a599-f7139d3811ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6824","nonce":110486},{"type":"v1","addr":"172.21.15.111:6825","nonce":110486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6826","nonce":110486},{"type":"v1","addr":"172.21.15.111:6827","nonce":110486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6830","nonce":110486},{"type":"v1","addr":"172.21.15.111:6831","nonce":110486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6828","nonce":110486},{"type":"v1","addr":"172.21.15.111:6829","nonce":110486}]},"public_addr":"172.21.15.111:6825/110486","cluster_addr":"172.21.15.111:6827/110486","heartbeat_back_addr":"172.21.15.111:6831/110486","heartbeat_front_addr":"172.21.15.111:6829/110486","state":["exists","up"]},{"osd":3,"uuid":"73e253f3-86e1-4da0-b800-ce2b864835df","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6800","nonce":110485},{"type":"v1","addr":"172.21.15.111:6801","nonce":110485}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6802","nonce":110485},{"type":"v1","addr":"172.21.15.111:6803","nonce":110485}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6806","nonce":110485},{"type":"v1","addr":"172.21.15.111:6807","nonce":110485}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6804","nonce":110485},{"type":"v1","addr":"172.21.15.111:6805","nonce":110485}]},"public_addr":"172.21.15.111:6801/110485","cluster_addr":"172.21.15.111:6803/110485","heartbeat_back_addr":"172.21.15.111:6807/110485","heartbeat_front_addr":"172.21.15.111:6805/110485","state":["exists","up"]},{"osd":4,"uuid":"095d437b-3816-4afb-86a9-6d78e19880c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":110644},{"type":"v1","addr":"172.21.15.119:6801","nonce":110644}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":110644},{"type":"v1","addr":"172.21.15.119:6803","nonce":110644}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":110644},{"type":"v1","addr":"172.21.15.119:6807","nonce":110644}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":110644},{"type":"v1","addr":"172.21.15.119:6805","nonce":110644}]},"public_addr":"172.21.15.119:6801/110644","cluster_addr":"172.21.15.119:6803/110644","heartbeat_back_addr":"172.21.15.119:6807/110644","heartbeat_front_addr":"172.21.15.119:6805/110644","state":["exists","up"]},{"osd":5,"uuid":"3e7fe27b-3125-4d00-be7b-c7a93679bfaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":110646},{"type":"v1","addr":"172.21.15.119:6817","nonce":110646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6818","nonce":110646},{"type":"v1","addr":"172.21.15.119:6819","nonce":110646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":110646},{"type":"v1","addr":"172.21.15.119:6823","nonce":110646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":110646},{"type":"v1","addr":"172.21.15.119:6821","nonce":110646}]},"public_addr":"172.21.15.119:6817/110646","cluster_addr":"172.21.15.119:6819/110646","heartbeat_back_addr":"172.21.15.119:6823/110646","heartbeat_front_addr":"172.21.15.119:6821/110646","state":["exists","up"]},{"osd":6,"uuid":"fb37ec18-5612-46aa-ae6b-682bc8fc784f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":110645},{"type":"v1","addr":"172.21.15.119:6809","nonce":110645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":110645},{"type":"v1","addr":"172.21.15.119:6811","nonce":110645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":110645},{"type":"v1","addr":"172.21.15.119:6815","nonce":110645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":110645},{"type":"v1","addr":"172.21.15.119:6813","nonce":110645}]},"public_addr":"172.21.15.119:6809/110645","cluster_addr":"172.21.15.119:6811/110645","heartbeat_back_addr":"172.21.15.119:6815/110645","heartbeat_front_addr":"172.21.15.119:6813/110645","state":["exists","up"]},{"osd":7,"uuid":"859085a7-fa35-4b0b-8583-f6e43ebfa7ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":110655},{"type":"v1","addr":"172.21.15.119:6825","nonce":110655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":110655},{"type":"v1","addr":"172.21.15.119:6827","nonce":110655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":110655},{"type":"v1","addr":"172.21.15.119:6831","nonce":110655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":110655},{"type":"v1","addr":"172.21.15.119:6829","nonce":110655}]},"public_addr":"172.21.15.119:6825/110655","cluster_addr":"172.21.15.119:6827/110655","heartbeat_back_addr":"172.21.15.119:6831/110655","heartbeat_front_addr":"172.21.15.119:6829/110655","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.249682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.205603+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.202338+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.244910+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.166651+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.201508+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.115669+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-03-29T10:57:10.169185+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-03-29T10:59:16.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.180+0000 7f11eb3ff700 1 -- 172.21.15.111:0/2336718100 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f11c8069de0 msgr2=0x7f11c806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:16.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.180+0000 7f11eb3ff700 1 --2- 172.21.15.111:0/2336718100 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f11c8069de0 0x7f11c806c290 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f11e41574e0 tx=0x7f11d4008040 comp rx=0 tx=0).stop 2023-03-29T10:59:16.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.180+0000 7f11eb3ff700 1 -- 172.21.15.111:0/2336718100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11e40b0dd0 msgr2=0x7f11e40b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:16.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.180+0000 7f11eb3ff700 1 --2- 172.21.15.111:0/2336718100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11e40b0dd0 0x7f11e40b04e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f11e000c520 tx=0x7f11e000c8e0 comp rx=0 tx=0).stop 2023-03-29T10:59:16.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.180+0000 7f11eb3ff700 1 -- 172.21.15.111:0/2336718100 shutdown_connections 2023-03-29T10:59:16.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.180+0000 7f11eb3ff700 1 --2- 172.21.15.111:0/2336718100 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f11c8069de0 0x7f11c806c290 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:16.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.180+0000 7f11eb3ff700 1 --2- 172.21.15.111:0/2336718100 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f11e40b0dd0 0x7f11e40b04e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:16.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.180+0000 7f11eb3ff700 1 --2- 172.21.15.111:0/2336718100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e4143f10 0x7f11e415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:16.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.180+0000 7f11eb3ff700 1 --2- 172.21.15.111:0/2336718100 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f11e40c97d0 0x7f11e41f0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:16.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.180+0000 7f11eb3ff700 1 -- 172.21.15.111:0/2336718100 >> 172.21.15.111:0/2336718100 conn(0x7f11e40cb6c0 msgr2=0x7f11e40b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:16.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.180+0000 7f11eb3ff700 1 -- 172.21.15.111:0/2336718100 shutdown_connections 2023-03-29T10:59:16.181 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.180+0000 7f11eb3ff700 1 -- 172.21.15.111:0/2336718100 wait complete. 2023-03-29T10:59:16.198 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-03-29T10:59:16.198 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T10:59:16.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.381+0000 7f45e43ff700 1 Processor -- start 2023-03-29T10:59:16.382 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.382+0000 7f45e43ff700 1 -- start start 2023-03-29T10:59:16.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.383+0000 7f45e43ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45dc0d3a30 0x7f45dc0b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:16.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.383+0000 7f45e43ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45dc0cdc80 con 0x7f45dc0cf950 2023-03-29T10:59:16.383 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.383+0000 7f45e43ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45dc0cd6c0 con 0x7f45dc15c950 2023-03-29T10:59:16.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.383+0000 7f45e43ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc0cd9a0 con 0x7f45dc0d3a30 2023-03-29T10:59:16.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.383+0000 7f45e219b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f45dc0cf950 0x7f45dc0cfd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38242/0 (socket says 172.21.15.111:38242) 2023-03-29T10:59:16.384 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.383+0000 7f45e219b700 1 -- 172.21.15.111:0/3759630333 learned_addr learned my addr 172.21.15.111:0/3759630333 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:59:16.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.383+0000 7f45e299c700 1 --2- 172.21.15.111:0/3759630333 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45dc0d3a30 0x7f45dc0b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:16.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.383+0000 7f45e1199700 1 -- 172.21.15.111:0/3759630333 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1952070987 0 0) 0x7f45dc0cdc80 con 0x7f45dc0cf950 2023-03-29T10:59:16.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.383+0000 7f45e1199700 1 -- 172.21.15.111:0/3759630333 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45c80035b0 con 0x7f45dc0cf950 2023-03-29T10:59:16.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.384+0000 7f45e1199700 1 -- 172.21.15.111:0/3759630333 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456756707 0 0) 0x7f45dc0cd6c0 con 0x7f45dc15c950 2023-03-29T10:59:16.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.384+0000 7f45e1199700 1 -- 172.21.15.111:0/3759630333 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45dc0cdc80 con 0x7f45dc15c950 2023-03-29T10:59:16.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.384+0000 7f45e1199700 1 -- 172.21.15.111:0/3759630333 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2040768265 0 0) 0x7f45c80035b0 con 0x7f45dc0cf950 2023-03-29T10:59:16.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.384+0000 7f45e299c700 1 -- 172.21.15.111:0/3759630333 >> v1:172.21.15.119:6789/0 conn(0x7f45dc15c950 legacy=0x7f45dc0cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:16.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.384+0000 7f45e299c700 1 -- 172.21.15.111:0/3759630333 >> v1:172.21.15.111:6789/0 conn(0x7f45dc0cf950 legacy=0x7f45dc0cfd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:16.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.384+0000 7f45e299c700 1 -- 172.21.15.111:0/3759630333 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45dc0d16c0 con 0x7f45dc0d3a30 2023-03-29T10:59:16.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.384+0000 7f45e299c700 1 --2- 172.21.15.111:0/3759630333 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45dc0d3a30 0x7f45dc0b5d90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009dc0 tx=0x7f45d400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac886da72f8ee570 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:16.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.384+0000 7f45e1199700 1 -- 172.21.15.111:0/3759630333 <== mon.2 v2:172.21.15.119:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d400e070 con 0x7f45dc0d3a30 2023-03-29T10:59:16.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.384+0000 7f45e1199700 1 -- 172.21.15.111:0/3759630333 <== mon.2 v2:172.21.15.119:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d4003a20 con 0x7f45dc0d3a30 2023-03-29T10:59:16.385 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.384+0000 7f45e1199700 1 -- 172.21.15.111:0/3759630333 <== mon.2 v2:172.21.15.119:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d8001670 con 0x7f45dc0d3a30 2023-03-29T10:59:16.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.385+0000 7f45e43ff700 1 -- 172.21.15.111:0/3759630333 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45dc0d3a30 msgr2=0x7f45dc0b5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:16.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.385+0000 7f45e43ff700 1 --2- 172.21.15.111:0/3759630333 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45dc0d3a30 0x7f45dc0b5d90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009dc0 tx=0x7f45d400b330 comp rx=0 tx=0).stop 2023-03-29T10:59:16.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.385+0000 7f45e43ff700 1 -- 172.21.15.111:0/3759630333 shutdown_connections 2023-03-29T10:59:16.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.385+0000 7f45e43ff700 1 --2- 172.21.15.111:0/3759630333 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45dc0d3a30 0x7f45dc0b5d90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:16.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.385+0000 7f45e43ff700 1 -- 172.21.15.111:0/3759630333 >> 172.21.15.111:0/3759630333 conn(0x7f45dc071b10 msgr2=0x7f45dc071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:16.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.385+0000 7f45e43ff700 1 -- 172.21.15.111:0/3759630333 shutdown_connections 2023-03-29T10:59:16.386 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.385+0000 7f45e43ff700 1 -- 172.21.15.111:0/3759630333 wait complete. 2023-03-29T10:59:16.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.387+0000 7f45e43ff700 1 Processor -- start 2023-03-29T10:59:16.387 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.387+0000 7f45e43ff700 1 -- start start 2023-03-29T10:59:16.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.388+0000 7f45e43ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45dc0cf950 0x7f45dc0c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:16.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.388+0000 7f45e43ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45dc0d3a30 0x7f45dc1480d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:16.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.388+0000 7f45e43ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45dc15c950 0x7f45dc0cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:16.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.388+0000 7f45e43ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc148610 con 0x7f45dc0cf950 2023-03-29T10:59:16.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.388+0000 7f45e43ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc15dd90 con 0x7f45dc15c950 2023-03-29T10:59:16.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.388+0000 7f45e43ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45dc0c6550 con 0x7f45dc0d3a30 2023-03-29T10:59:16.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.388+0000 7f45e219b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45dc0cf950 0x7f45dc0c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:16.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.388+0000 7f45e219b700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45dc0cf950 0x7f45dc0c93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40384/0 (socket says 172.21.15.111:40384) 2023-03-29T10:59:16.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.388+0000 7f45e219b700 1 -- 172.21.15.111:0/71771483 learned_addr learned my addr 172.21.15.111:0/71771483 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:16.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.388+0000 7f45e299c700 1 --2- 172.21.15.111:0/71771483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45dc15c950 0x7f45dc0cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:16.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.388+0000 7f45e199a700 1 --2- 172.21.15.111:0/71771483 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45dc0d3a30 0x7f45dc1480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:16.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.388+0000 7f45e219b700 1 -- 172.21.15.111:0/71771483 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45dc0d3a30 msgr2=0x7f45dc1480d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:16.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.388+0000 7f45e219b700 1 --2- 172.21.15.111:0/71771483 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45dc0d3a30 0x7f45dc1480d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:16.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.388+0000 7f45e219b700 1 -- 172.21.15.111:0/71771483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45dc15c950 msgr2=0x7f45dc0cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:16.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.388+0000 7f45e219b700 1 --2- 172.21.15.111:0/71771483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45dc15c950 0x7f45dc0cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:16.389 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.388+0000 7f45e219b700 1 -- 172.21.15.111:0/71771483 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d4009a70 con 0x7f45dc0cf950 2023-03-29T10:59:16.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.388+0000 7f45e219b700 1 --2- 172.21.15.111:0/71771483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45dc0cf950 0x7f45dc0c93e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f45d800c730 tx=0x7f45d800caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:16.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.389+0000 7f45d2ffd700 1 -- 172.21.15.111:0/71771483 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d8004c80 con 0x7f45dc0cf950 2023-03-29T10:59:16.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.389+0000 7f45d2ffd700 1 -- 172.21.15.111:0/71771483 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d8004e00 con 0x7f45dc0cf950 2023-03-29T10:59:16.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.389+0000 7f45d2ffd700 1 -- 172.21.15.111:0/71771483 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d8007310 con 0x7f45dc0cf950 2023-03-29T10:59:16.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.389+0000 7f45e43ff700 1 -- 172.21.15.111:0/71771483 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45dc0b62d0 con 0x7f45dc0cf950 2023-03-29T10:59:16.390 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.389+0000 7f45e43ff700 1 -- 172.21.15.111:0/71771483 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45dc0d19a0 con 0x7f45dc0cf950 2023-03-29T10:59:16.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.390+0000 7f45e43ff700 1 -- 172.21.15.111:0/71771483 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45dc0b6460 con 0x7f45dc0cf950 2023-03-29T10:59:16.391 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.391+0000 7f45d2ffd700 1 -- 172.21.15.111:0/71771483 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f45d8007470 con 0x7f45dc0cf950 2023-03-29T10:59:16.392 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.392+0000 7f45d2ffd700 1 --2- 172.21.15.111:0/71771483 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f45c0069de0 0x7f45c006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:16.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.392+0000 7f45d2ffd700 1 -- 172.21.15.111:0/71771483 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f45d8087f80 con 0x7f45dc0cf950 2023-03-29T10:59:16.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.392+0000 7f45e199a700 1 --2- 172.21.15.111:0/71771483 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f45c0069de0 0x7f45c006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:16.393 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.393+0000 7f45e199a700 1 --2- 172.21.15.111:0/71771483 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f45c0069de0 0x7f45c006c290 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f45cc007fd0 tx=0x7f45cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:16.397 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.397+0000 7f45d2ffd700 1 -- 172.21.15.111:0/71771483 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f45d80536e0 con 0x7f45dc0cf950 2023-03-29T10:59:16.587 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.587+0000 7f45e43ff700 1 -- 172.21.15.111:0/71771483 --> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f45dc074430 con 0x7f45c0069de0 2023-03-29T10:59:16.597 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.597+0000 7f45d2ffd700 1 -- 172.21.15.111:0/71771483 <== mgr.4103 v2:172.21.15.111:6832/110141 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311160 (secure 0 0 0) 0x7f45dc074430 con 0x7f45c0069de0 2023-03-29T10:59:16.601 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:59:16.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.601+0000 7f45e43ff700 1 -- 172.21.15.111:0/71771483 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f45c0069de0 msgr2=0x7f45c006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:16.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.601+0000 7f45e43ff700 1 --2- 172.21.15.111:0/71771483 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f45c0069de0 0x7f45c006c290 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f45cc007fd0 tx=0x7f45cc009040 comp rx=0 tx=0).stop 2023-03-29T10:59:16.602 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.601+0000 7f45e43ff700 1 -- 172.21.15.111:0/71771483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45dc0cf950 msgr2=0x7f45dc0c93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:16.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.601+0000 7f45e43ff700 1 --2- 172.21.15.111:0/71771483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45dc0cf950 0x7f45dc0c93e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f45d800c730 tx=0x7f45d800caf0 comp rx=0 tx=0).stop 2023-03-29T10:59:16.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.601+0000 7f45e43ff700 1 -- 172.21.15.111:0/71771483 shutdown_connections 2023-03-29T10:59:16.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.601+0000 7f45e43ff700 1 --2- 172.21.15.111:0/71771483 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f45c0069de0 0x7f45c006c290 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:16.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.601+0000 7f45e43ff700 1 --2- 172.21.15.111:0/71771483 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f45dc0cf950 0x7f45dc0c93e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:16.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.601+0000 7f45e43ff700 1 --2- 172.21.15.111:0/71771483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45dc15c950 0x7f45dc0cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:16.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.601+0000 7f45e43ff700 1 --2- 172.21.15.111:0/71771483 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f45dc0d3a30 0x7f45dc1480d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:16.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.601+0000 7f45e43ff700 1 -- 172.21.15.111:0/71771483 >> 172.21.15.111:0/71771483 conn(0x7f45dc071b10 msgr2=0x7f45dc0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:16.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.601+0000 7f45e43ff700 1 -- 172.21.15.111:0/71771483 shutdown_connections 2023-03-29T10:59:16.603 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.601+0000 7f45e43ff700 1 -- 172.21.15.111:0/71771483 wait complete. 2023-03-29T10:59:16.603 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2023-03-29T10:59:16.619 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2023-03-29T10:59:16.148971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592734,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":71936,"kb_used_data":5120,"kb_used_omap":0,"kb_used_meta":66752,"kb_avail":754902784,"statfs":{"total":773094113280,"available":773020450816,"internally_reserved":0,"allocated":5242880,"data_stored":2971806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68354048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2366,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.336731"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313958+0000","last_change":"2023-03-29T10:59:12.324127+0000","last_active":"2023-03-29T10:59:13.313958+0000","last_peered":"2023-03-29T10:59:13.313958+0000","last_clean":"2023-03-29T10:59:13.313958+0000","last_became_active":"2023-03-29T10:59:12.323800+0000","last_became_peered":"2023-03-29T10:59:12.323800+0000","last_unstale":"2023-03-29T10:59:13.313958+0000","last_undegraded":"2023-03-29T10:59:13.313958+0000","last_fullsized":"2023-03-29T10:59:13.313958+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:42:57.025474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718453+0000","last_change":"2023-03-29T10:59:12.329201+0000","last_active":"2023-03-29T10:59:13.718453+0000","last_peered":"2023-03-29T10:59:13.718453+0000","last_clean":"2023-03-29T10:59:13.718453+0000","last_became_active":"2023-03-29T10:59:12.328922+0000","last_became_peered":"2023-03-29T10:59:12.328922+0000","last_unstale":"2023-03-29T10:59:13.718453+0000","last_undegraded":"2023-03-29T10:59:13.718453+0000","last_fullsized":"2023-03-29T10:59:13.718453+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:34:44.160250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718830+0000","last_change":"2023-03-29T10:59:12.325166+0000","last_active":"2023-03-29T10:59:13.718830+0000","last_peered":"2023-03-29T10:59:13.718830+0000","last_clean":"2023-03-29T10:59:13.718830+0000","last_became_active":"2023-03-29T10:59:12.324879+0000","last_became_peered":"2023-03-29T10:59:12.324879+0000","last_unstale":"2023-03-29T10:59:13.718830+0000","last_undegraded":"2023-03-29T10:59:13.718830+0000","last_fullsized":"2023-03-29T10:59:13.718830+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:31:56.427766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637378+0000","last_change":"2023-03-29T10:59:12.337300+0000","last_active":"2023-03-29T10:59:13.637378+0000","last_peered":"2023-03-29T10:59:13.637378+0000","last_clean":"2023-03-29T10:59:13.637378+0000","last_became_active":"2023-03-29T10:59:12.337030+0000","last_became_peered":"2023-03-29T10:59:12.337030+0000","last_unstale":"2023-03-29T10:59:13.637378+0000","last_undegraded":"2023-03-29T10:59:13.637378+0000","last_fullsized":"2023-03-29T10:59:13.637378+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:31:56.427766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718167+0000","last_change":"2023-03-29T10:59:12.329253+0000","last_active":"2023-03-29T10:59:13.718167+0000","last_peered":"2023-03-29T10:59:13.718167+0000","last_clean":"2023-03-29T10:59:13.718167+0000","last_became_active":"2023-03-29T10:59:12.328998+0000","last_became_peered":"2023-03-29T10:59:12.328998+0000","last_unstale":"2023-03-29T10:59:13.718167+0000","last_undegraded":"2023-03-29T10:59:13.718167+0000","last_fullsized":"2023-03-29T10:59:13.718167+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:05:53.919758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.522861+0000","last_change":"2023-03-29T10:59:12.332574+0000","last_active":"2023-03-29T10:59:13.522861+0000","last_peered":"2023-03-29T10:59:13.522861+0000","last_clean":"2023-03-29T10:59:13.522861+0000","last_became_active":"2023-03-29T10:59:12.332261+0000","last_became_peered":"2023-03-29T10:59:12.332261+0000","last_unstale":"2023-03-29T10:59:13.522861+0000","last_undegraded":"2023-03-29T10:59:13.522861+0000","last_fullsized":"2023-03-29T10:59:13.522861+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:05:53.919758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313825+0000","last_change":"2023-03-29T10:59:12.323148+0000","last_active":"2023-03-29T10:59:13.313825+0000","last_peered":"2023-03-29T10:59:13.313825+0000","last_clean":"2023-03-29T10:59:13.313825+0000","last_became_active":"2023-03-29T10:59:12.322854+0000","last_became_peered":"2023-03-29T10:59:12.322854+0000","last_unstale":"2023-03-29T10:59:13.313825+0000","last_undegraded":"2023-03-29T10:59:13.313825+0000","last_fullsized":"2023-03-29T10:59:13.313825+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:25:53.380937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637031+0000","last_change":"2023-03-29T10:59:12.336206+0000","last_active":"2023-03-29T10:59:13.637031+0000","last_peered":"2023-03-29T10:59:13.637031+0000","last_clean":"2023-03-29T10:59:13.637031+0000","last_became_active":"2023-03-29T10:59:12.334168+0000","last_became_peered":"2023-03-29T10:59:12.334168+0000","last_unstale":"2023-03-29T10:59:13.637031+0000","last_undegraded":"2023-03-29T10:59:13.637031+0000","last_fullsized":"2023-03-29T10:59:13.637031+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:32:53.291657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637458+0000","last_change":"2023-03-29T10:59:12.325862+0000","last_active":"2023-03-29T10:59:13.637458+0000","last_peered":"2023-03-29T10:59:13.637458+0000","last_clean":"2023-03-29T10:59:13.637458+0000","last_became_active":"2023-03-29T10:59:12.325492+0000","last_became_peered":"2023-03-29T10:59:12.325492+0000","last_unstale":"2023-03-29T10:59:13.637458+0000","last_undegraded":"2023-03-29T10:59:13.637458+0000","last_fullsized":"2023-03-29T10:59:13.637458+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:18:03.154980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637519+0000","last_change":"2023-03-29T10:59:12.334651+0000","last_active":"2023-03-29T10:59:13.637519+0000","last_peered":"2023-03-29T10:59:13.637519+0000","last_clean":"2023-03-29T10:59:13.637519+0000","last_became_active":"2023-03-29T10:59:12.334332+0000","last_became_peered":"2023-03-29T10:59:12.334332+0000","last_unstale":"2023-03-29T10:59:13.637519+0000","last_undegraded":"2023-03-29T10:59:13.637519+0000","last_fullsized":"2023-03-29T10:59:13.637519+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:21:44.320888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637210+0000","last_change":"2023-03-29T10:59:12.335223+0000","last_active":"2023-03-29T10:59:13.637210+0000","last_peered":"2023-03-29T10:59:13.637210+0000","last_clean":"2023-03-29T10:59:13.637210+0000","last_became_active":"2023-03-29T10:59:12.335037+0000","last_became_peered":"2023-03-29T10:59:12.335037+0000","last_unstale":"2023-03-29T10:59:13.637210+0000","last_undegraded":"2023-03-29T10:59:13.637210+0000","last_fullsized":"2023-03-29T10:59:13.637210+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:34:44.160250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718623+0000","last_change":"2023-03-29T10:59:12.311070+0000","last_active":"2023-03-29T10:59:13.718623+0000","last_peered":"2023-03-29T10:59:13.718623+0000","last_clean":"2023-03-29T10:59:13.718623+0000","last_became_active":"2023-03-29T10:59:12.310696+0000","last_became_peered":"2023-03-29T10:59:12.310696+0000","last_unstale":"2023-03-29T10:59:13.718623+0000","last_undegraded":"2023-03-29T10:59:13.718623+0000","last_fullsized":"2023-03-29T10:59:13.718623+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:29:07.728609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.308691+0000","last_change":"2023-03-29T10:59:12.336284+0000","last_active":"2023-03-29T10:59:13.308691+0000","last_peered":"2023-03-29T10:59:13.308691+0000","last_clean":"2023-03-29T10:59:13.308691+0000","last_became_active":"2023-03-29T10:59:12.335973+0000","last_became_peered":"2023-03-29T10:59:12.335973+0000","last_unstale":"2023-03-29T10:59:13.308691+0000","last_undegraded":"2023-03-29T10:59:13.308691+0000","last_fullsized":"2023-03-29T10:59:13.308691+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:13:18.091340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637124+0000","last_change":"2023-03-29T10:59:12.323063+0000","last_active":"2023-03-29T10:59:13.637124+0000","last_peered":"2023-03-29T10:59:13.637124+0000","last_clean":"2023-03-29T10:59:13.637124+0000","last_became_active":"2023-03-29T10:59:12.322888+0000","last_became_peered":"2023-03-29T10:59:12.322888+0000","last_unstale":"2023-03-29T10:59:13.637124+0000","last_undegraded":"2023-03-29T10:59:13.637124+0000","last_fullsized":"2023-03-29T10:59:13.637124+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:04:15.453097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.522854+0000","last_change":"2023-03-29T10:59:12.333923+0000","last_active":"2023-03-29T10:59:13.522854+0000","last_peered":"2023-03-29T10:59:13.522854+0000","last_clean":"2023-03-29T10:59:13.522854+0000","last_became_active":"2023-03-29T10:59:12.333348+0000","last_became_peered":"2023-03-29T10:59:12.333348+0000","last_unstale":"2023-03-29T10:59:13.522854+0000","last_undegraded":"2023-03-29T10:59:13.522854+0000","last_fullsized":"2023-03-29T10:59:13.522854+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:34:44.160250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718559+0000","last_change":"2023-03-29T10:59:12.329173+0000","last_active":"2023-03-29T10:59:13.718559+0000","last_peered":"2023-03-29T10:59:13.718559+0000","last_clean":"2023-03-29T10:59:13.718559+0000","last_became_active":"2023-03-29T10:59:12.328874+0000","last_became_peered":"2023-03-29T10:59:12.328874+0000","last_unstale":"2023-03-29T10:59:13.718559+0000","last_undegraded":"2023-03-29T10:59:13.718559+0000","last_fullsized":"2023-03-29T10:59:13.718559+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:44.599265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718877+0000","last_change":"2023-03-29T10:59:12.321393+0000","last_active":"2023-03-29T10:59:13.718877+0000","last_peered":"2023-03-29T10:59:13.718877+0000","last_clean":"2023-03-29T10:59:13.718877+0000","last_became_active":"2023-03-29T10:59:12.321116+0000","last_became_peered":"2023-03-29T10:59:12.321116+0000","last_unstale":"2023-03-29T10:59:13.718877+0000","last_undegraded":"2023-03-29T10:59:13.718877+0000","last_fullsized":"2023-03-29T10:59:13.718877+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:04:11.191595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719440+0000","last_change":"2023-03-29T10:59:12.327840+0000","last_active":"2023-03-29T10:59:13.719440+0000","last_peered":"2023-03-29T10:59:13.719440+0000","last_clean":"2023-03-29T10:59:13.719440+0000","last_became_active":"2023-03-29T10:59:12.327137+0000","last_became_peered":"2023-03-29T10:59:12.327137+0000","last_unstale":"2023-03-29T10:59:13.719440+0000","last_undegraded":"2023-03-29T10:59:13.719440+0000","last_fullsized":"2023-03-29T10:59:13.719440+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:21:44.320888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719024+0000","last_change":"2023-03-29T10:59:12.332453+0000","last_active":"2023-03-29T10:59:13.719024+0000","last_peered":"2023-03-29T10:59:13.719024+0000","last_clean":"2023-03-29T10:59:13.719024+0000","last_became_active":"2023-03-29T10:59:12.331831+0000","last_became_peered":"2023-03-29T10:59:12.331831+0000","last_unstale":"2023-03-29T10:59:13.719024+0000","last_undegraded":"2023-03-29T10:59:13.719024+0000","last_fullsized":"2023-03-29T10:59:13.719024+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:16:57.395398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.308707+0000","last_change":"2023-03-29T10:59:12.326835+0000","last_active":"2023-03-29T10:59:13.308707+0000","last_peered":"2023-03-29T10:59:13.308707+0000","last_clean":"2023-03-29T10:59:13.308707+0000","last_became_active":"2023-03-29T10:59:12.326682+0000","last_became_peered":"2023-03-29T10:59:12.326682+0000","last_unstale":"2023-03-29T10:59:13.308707+0000","last_undegraded":"2023-03-29T10:59:13.308707+0000","last_fullsized":"2023-03-29T10:59:13.308707+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:44.599265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.522902+0000","last_change":"2023-03-29T10:59:12.332900+0000","last_active":"2023-03-29T10:59:13.522902+0000","last_peered":"2023-03-29T10:59:13.522902+0000","last_clean":"2023-03-29T10:59:13.522902+0000","last_became_active":"2023-03-29T10:59:12.332634+0000","last_became_peered":"2023-03-29T10:59:12.332634+0000","last_unstale":"2023-03-29T10:59:13.522902+0000","last_undegraded":"2023-03-29T10:59:13.522902+0000","last_fullsized":"2023-03-29T10:59:13.522902+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:04:15.453097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637326+0000","last_change":"2023-03-29T10:59:12.333400+0000","last_active":"2023-03-29T10:59:13.637326+0000","last_peered":"2023-03-29T10:59:13.637326+0000","last_clean":"2023-03-29T10:59:13.637326+0000","last_became_active":"2023-03-29T10:59:12.333155+0000","last_became_peered":"2023-03-29T10:59:12.333155+0000","last_unstale":"2023-03-29T10:59:13.637326+0000","last_undegraded":"2023-03-29T10:59:13.637326+0000","last_fullsized":"2023-03-29T10:59:13.637326+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:14:43.981498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718771+0000","last_change":"2023-03-29T10:59:12.323414+0000","last_active":"2023-03-29T10:59:13.718771+0000","last_peered":"2023-03-29T10:59:13.718771+0000","last_clean":"2023-03-29T10:59:13.718771+0000","last_became_active":"2023-03-29T10:59:12.322807+0000","last_became_peered":"2023-03-29T10:59:12.322807+0000","last_unstale":"2023-03-29T10:59:13.718771+0000","last_undegraded":"2023-03-29T10:59:13.718771+0000","last_fullsized":"2023-03-29T10:59:13.718771+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:13:05.082148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718663+0000","last_change":"2023-03-29T10:59:12.322363+0000","last_active":"2023-03-29T10:59:13.718663+0000","last_peered":"2023-03-29T10:59:13.718663+0000","last_clean":"2023-03-29T10:59:13.718663+0000","last_became_active":"2023-03-29T10:59:12.322076+0000","last_became_peered":"2023-03-29T10:59:12.322076+0000","last_unstale":"2023-03-29T10:59:13.718663+0000","last_undegraded":"2023-03-29T10:59:13.718663+0000","last_fullsized":"2023-03-29T10:59:13.718663+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:35:58.000512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719420+0000","last_change":"2023-03-29T10:59:12.324550+0000","last_active":"2023-03-29T10:59:13.719420+0000","last_peered":"2023-03-29T10:59:13.719420+0000","last_clean":"2023-03-29T10:59:13.719420+0000","last_became_active":"2023-03-29T10:59:12.323215+0000","last_became_peered":"2023-03-29T10:59:12.323215+0000","last_unstale":"2023-03-29T10:59:13.719420+0000","last_undegraded":"2023-03-29T10:59:13.719420+0000","last_fullsized":"2023-03-29T10:59:13.719420+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:34:24.360304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.308817+0000","last_change":"2023-03-29T10:59:12.337050+0000","last_active":"2023-03-29T10:59:13.308817+0000","last_peered":"2023-03-29T10:59:13.308817+0000","last_clean":"2023-03-29T10:59:13.308817+0000","last_became_active":"2023-03-29T10:59:12.336800+0000","last_became_peered":"2023-03-29T10:59:12.336800+0000","last_unstale":"2023-03-29T10:59:13.308817+0000","last_undegraded":"2023-03-29T10:59:13.308817+0000","last_fullsized":"2023-03-29T10:59:13.308817+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:13:05.082148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718961+0000","last_change":"2023-03-29T10:59:12.320868+0000","last_active":"2023-03-29T10:59:13.718961+0000","last_peered":"2023-03-29T10:59:13.718961+0000","last_clean":"2023-03-29T10:59:13.718961+0000","last_became_active":"2023-03-29T10:59:12.320555+0000","last_became_peered":"2023-03-29T10:59:12.320555+0000","last_unstale":"2023-03-29T10:59:13.718961+0000","last_undegraded":"2023-03-29T10:59:13.718961+0000","last_fullsized":"2023-03-29T10:59:13.718961+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:04:15.453097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718346+0000","last_change":"2023-03-29T10:59:12.311066+0000","last_active":"2023-03-29T10:59:13.718346+0000","last_peered":"2023-03-29T10:59:13.718346+0000","last_clean":"2023-03-29T10:59:13.718346+0000","last_became_active":"2023-03-29T10:59:12.310698+0000","last_became_peered":"2023-03-29T10:59:12.310698+0000","last_unstale":"2023-03-29T10:59:13.718346+0000","last_undegraded":"2023-03-29T10:59:13.718346+0000","last_fullsized":"2023-03-29T10:59:13.718346+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:06:46.183216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718798+0000","last_change":"2023-03-29T10:59:12.324113+0000","last_active":"2023-03-29T10:59:13.718798+0000","last_peered":"2023-03-29T10:59:13.718798+0000","last_clean":"2023-03-29T10:59:13.718798+0000","last_became_active":"2023-03-29T10:59:12.323855+0000","last_became_peered":"2023-03-29T10:59:12.323855+0000","last_unstale":"2023-03-29T10:59:13.718798+0000","last_undegraded":"2023-03-29T10:59:13.718798+0000","last_fullsized":"2023-03-29T10:59:13.718798+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:07:41.583160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718727+0000","last_change":"2023-03-29T10:59:12.323918+0000","last_active":"2023-03-29T10:59:13.718727+0000","last_peered":"2023-03-29T10:59:13.718727+0000","last_clean":"2023-03-29T10:59:13.718727+0000","last_became_active":"2023-03-29T10:59:12.323637+0000","last_became_peered":"2023-03-29T10:59:12.323637+0000","last_unstale":"2023-03-29T10:59:13.718727+0000","last_undegraded":"2023-03-29T10:59:13.718727+0000","last_fullsized":"2023-03-29T10:59:13.718727+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:18:03.154980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.522752+0000","last_change":"2023-03-29T10:59:12.334069+0000","last_active":"2023-03-29T10:59:13.522752+0000","last_peered":"2023-03-29T10:59:13.522752+0000","last_clean":"2023-03-29T10:59:13.522752+0000","last_became_active":"2023-03-29T10:59:12.333665+0000","last_became_peered":"2023-03-29T10:59:12.333665+0000","last_unstale":"2023-03-29T10:59:13.522752+0000","last_undegraded":"2023-03-29T10:59:13.522752+0000","last_fullsized":"2023-03-29T10:59:13.522752+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:29:07.728609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.308505+0000","last_change":"2023-03-29T10:59:12.320236+0000","last_active":"2023-03-29T10:59:13.308505+0000","last_peered":"2023-03-29T10:59:13.308505+0000","last_clean":"2023-03-29T10:59:13.308505+0000","last_became_active":"2023-03-29T10:59:12.319962+0000","last_became_peered":"2023-03-29T10:59:12.319962+0000","last_unstale":"2023-03-29T10:59:13.308505+0000","last_undegraded":"2023-03-29T10:59:13.308505+0000","last_fullsized":"2023-03-29T10:59:13.308505+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:16:57.395398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313465+0000","last_change":"2023-03-29T10:59:12.333949+0000","last_active":"2023-03-29T10:59:13.313465+0000","last_peered":"2023-03-29T10:59:13.313465+0000","last_clean":"2023-03-29T10:59:13.313465+0000","last_became_active":"2023-03-29T10:59:12.333698+0000","last_became_peered":"2023-03-29T10:59:12.333698+0000","last_unstale":"2023-03-29T10:59:13.313465+0000","last_undegraded":"2023-03-29T10:59:13.313465+0000","last_fullsized":"2023-03-29T10:59:13.313465+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:48:31.147706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.523225+0000","last_change":"2023-03-29T10:59:11.838304+0000","last_active":"2023-03-29T10:59:13.523225+0000","last_peered":"2023-03-29T10:59:13.523225+0000","last_clean":"2023-03-29T10:59:13.523225+0000","last_became_active":"2023-03-29T10:59:11.838018+0000","last_became_peered":"2023-03-29T10:59:11.838018+0000","last_unstale":"2023-03-29T10:59:13.523225+0000","last_undegraded":"2023-03-29T10:59:13.523225+0000","last_fullsized":"2023-03-29T10:59:13.523225+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:00:07.668045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719316+0000","last_change":"2023-03-29T10:59:12.330616+0000","last_active":"2023-03-29T10:59:13.719316+0000","last_peered":"2023-03-29T10:59:13.719316+0000","last_clean":"2023-03-29T10:59:13.719316+0000","last_became_active":"2023-03-29T10:59:12.330329+0000","last_became_peered":"2023-03-29T10:59:12.330329+0000","last_unstale":"2023-03-29T10:59:13.719316+0000","last_undegraded":"2023-03-29T10:59:13.719316+0000","last_fullsized":"2023-03-29T10:59:13.719316+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:21:09.442824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719395+0000","last_change":"2023-03-29T10:59:11.840914+0000","last_active":"2023-03-29T10:59:13.719395+0000","last_peered":"2023-03-29T10:59:13.719395+0000","last_clean":"2023-03-29T10:59:13.719395+0000","last_became_active":"2023-03-29T10:59:11.840630+0000","last_became_peered":"2023-03-29T10:59:11.840630+0000","last_unstale":"2023-03-29T10:59:13.719395+0000","last_undegraded":"2023-03-29T10:59:13.719395+0000","last_fullsized":"2023-03-29T10:59:13.719395+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:40:10.721415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718452+0000","last_change":"2023-03-29T10:59:12.340614+0000","last_active":"2023-03-29T10:59:13.718452+0000","last_peered":"2023-03-29T10:59:13.718452+0000","last_clean":"2023-03-29T10:59:13.718452+0000","last_became_active":"2023-03-29T10:59:12.340354+0000","last_became_peered":"2023-03-29T10:59:12.340354+0000","last_unstale":"2023-03-29T10:59:13.718452+0000","last_undegraded":"2023-03-29T10:59:13.718452+0000","last_fullsized":"2023-03-29T10:59:13.718452+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:58:16.409418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313459+0000","last_change":"2023-03-29T10:59:11.838077+0000","last_active":"2023-03-29T10:59:13.313459+0000","last_peered":"2023-03-29T10:59:13.313459+0000","last_clean":"2023-03-29T10:59:13.313459+0000","last_became_active":"2023-03-29T10:59:11.837825+0000","last_became_peered":"2023-03-29T10:59:11.837825+0000","last_unstale":"2023-03-29T10:59:13.313459+0000","last_undegraded":"2023-03-29T10:59:13.313459+0000","last_fullsized":"2023-03-29T10:59:13.313459+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:22:23.341915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313295+0000","last_change":"2023-03-29T10:59:12.320564+0000","last_active":"2023-03-29T10:59:13.313295+0000","last_peered":"2023-03-29T10:59:13.313295+0000","last_clean":"2023-03-29T10:59:13.313295+0000","last_became_active":"2023-03-29T10:59:12.320300+0000","last_became_peered":"2023-03-29T10:59:12.320300+0000","last_unstale":"2023-03-29T10:59:13.313295+0000","last_undegraded":"2023-03-29T10:59:13.313295+0000","last_fullsized":"2023-03-29T10:59:13.313295+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:57:29.081200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313780+0000","last_change":"2023-03-29T10:59:11.826501+0000","last_active":"2023-03-29T10:59:13.313780+0000","last_peered":"2023-03-29T10:59:13.313780+0000","last_clean":"2023-03-29T10:59:13.313780+0000","last_became_active":"2023-03-29T10:59:11.826236+0000","last_became_peered":"2023-03-29T10:59:11.826236+0000","last_unstale":"2023-03-29T10:59:13.313780+0000","last_undegraded":"2023-03-29T10:59:13.313780+0000","last_fullsized":"2023-03-29T10:59:13.313780+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:13:17.086112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718851+0000","last_change":"2023-03-29T10:59:12.321197+0000","last_active":"2023-03-29T10:59:13.718851+0000","last_peered":"2023-03-29T10:59:13.718851+0000","last_clean":"2023-03-29T10:59:13.718851+0000","last_became_active":"2023-03-29T10:59:12.320920+0000","last_became_peered":"2023-03-29T10:59:12.320920+0000","last_unstale":"2023-03-29T10:59:13.718851+0000","last_undegraded":"2023-03-29T10:59:13.718851+0000","last_fullsized":"2023-03-29T10:59:13.718851+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:29:48.533623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.523004+0000","last_change":"2023-03-29T10:59:11.827680+0000","last_active":"2023-03-29T10:59:13.523004+0000","last_peered":"2023-03-29T10:59:13.523004+0000","last_clean":"2023-03-29T10:59:13.523004+0000","last_became_active":"2023-03-29T10:59:11.827408+0000","last_became_peered":"2023-03-29T10:59:11.827408+0000","last_unstale":"2023-03-29T10:59:13.523004+0000","last_undegraded":"2023-03-29T10:59:13.523004+0000","last_fullsized":"2023-03-29T10:59:13.523004+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:25:52.375709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637210+0000","last_change":"2023-03-29T10:59:12.330741+0000","last_active":"2023-03-29T10:59:13.637210+0000","last_peered":"2023-03-29T10:59:13.637210+0000","last_clean":"2023-03-29T10:59:13.637210+0000","last_became_active":"2023-03-29T10:59:12.330283+0000","last_became_peered":"2023-03-29T10:59:12.330283+0000","last_unstale":"2023-03-29T10:59:13.637210+0000","last_undegraded":"2023-03-29T10:59:13.637210+0000","last_fullsized":"2023-03-29T10:59:13.637210+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:56:05.521539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719190+0000","last_change":"2023-03-29T10:59:11.837742+0000","last_active":"2023-03-29T10:59:13.719190+0000","last_peered":"2023-03-29T10:59:13.719190+0000","last_clean":"2023-03-29T10:59:13.719190+0000","last_became_active":"2023-03-29T10:59:11.837390+0000","last_became_peered":"2023-03-29T10:59:11.837390+0000","last_unstale":"2023-03-29T10:59:13.719190+0000","last_undegraded":"2023-03-29T10:59:13.719190+0000","last_fullsized":"2023-03-29T10:59:13.719190+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:19:12.384738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637266+0000","last_change":"2023-03-29T10:59:12.335828+0000","last_active":"2023-03-29T10:59:13.637266+0000","last_peered":"2023-03-29T10:59:13.637266+0000","last_clean":"2023-03-29T10:59:13.637266+0000","last_became_active":"2023-03-29T10:59:12.331831+0000","last_became_peered":"2023-03-29T10:59:12.331831+0000","last_unstale":"2023-03-29T10:59:13.637266+0000","last_undegraded":"2023-03-29T10:59:13.637266+0000","last_fullsized":"2023-03-29T10:59:13.637266+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:29:48.533623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.717964+0000","last_change":"2023-03-29T10:59:11.830858+0000","last_active":"2023-03-29T10:59:13.717964+0000","last_peered":"2023-03-29T10:59:13.717964+0000","last_clean":"2023-03-29T10:59:13.717964+0000","last_became_active":"2023-03-29T10:59:11.830523+0000","last_became_peered":"2023-03-29T10:59:11.830523+0000","last_unstale":"2023-03-29T10:59:13.717964+0000","last_undegraded":"2023-03-29T10:59:13.717964+0000","last_fullsized":"2023-03-29T10:59:13.717964+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:50:48.449260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718878+0000","last_change":"2023-03-29T10:59:12.322950+0000","last_active":"2023-03-29T10:59:13.718878+0000","last_peered":"2023-03-29T10:59:13.718878+0000","last_clean":"2023-03-29T10:59:13.718878+0000","last_became_active":"2023-03-29T10:59:12.322642+0000","last_became_peered":"2023-03-29T10:59:12.322642+0000","last_unstale":"2023-03-29T10:59:13.718878+0000","last_undegraded":"2023-03-29T10:59:13.718878+0000","last_fullsized":"2023-03-29T10:59:13.718878+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:22:24.347143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637124+0000","last_change":"2023-03-29T10:59:11.835227+0000","last_active":"2023-03-29T10:59:13.637124+0000","last_peered":"2023-03-29T10:59:13.637124+0000","last_clean":"2023-03-29T10:59:13.637124+0000","last_became_active":"2023-03-29T10:59:11.834779+0000","last_became_peered":"2023-03-29T10:59:11.834779+0000","last_unstale":"2023-03-29T10:59:13.637124+0000","last_undegraded":"2023-03-29T10:59:13.637124+0000","last_fullsized":"2023-03-29T10:59:13.637124+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:07:59.525950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719216+0000","last_change":"2023-03-29T10:59:12.328010+0000","last_active":"2023-03-29T10:59:13.719216+0000","last_peered":"2023-03-29T10:59:13.719216+0000","last_clean":"2023-03-29T10:59:13.719216+0000","last_became_active":"2023-03-29T10:59:12.327520+0000","last_became_peered":"2023-03-29T10:59:12.327520+0000","last_unstale":"2023-03-29T10:59:13.719216+0000","last_undegraded":"2023-03-29T10:59:13.719216+0000","last_fullsized":"2023-03-29T10:59:13.719216+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:14:43.981498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719073+0000","last_change":"2023-03-29T10:59:11.838354+0000","last_active":"2023-03-29T10:59:13.719073+0000","last_peered":"2023-03-29T10:59:13.719073+0000","last_clean":"2023-03-29T10:59:13.719073+0000","last_became_active":"2023-03-29T10:59:11.838055+0000","last_became_peered":"2023-03-29T10:59:11.838055+0000","last_unstale":"2023-03-29T10:59:13.719073+0000","last_undegraded":"2023-03-29T10:59:13.719073+0000","last_fullsized":"2023-03-29T10:59:13.719073+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:52:54.550636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.314077+0000","last_change":"2023-03-29T10:59:12.317257+0000","last_active":"2023-03-29T10:59:13.314077+0000","last_peered":"2023-03-29T10:59:13.314077+0000","last_clean":"2023-03-29T10:59:13.314077+0000","last_became_active":"2023-03-29T10:59:12.316967+0000","last_became_peered":"2023-03-29T10:59:12.316967+0000","last_unstale":"2023-03-29T10:59:13.314077+0000","last_undegraded":"2023-03-29T10:59:13.314077+0000","last_fullsized":"2023-03-29T10:59:13.314077+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:00:08.673273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637638+0000","last_change":"2023-03-29T10:59:11.836706+0000","last_active":"2023-03-29T10:59:13.637638+0000","last_peered":"2023-03-29T10:59:13.637638+0000","last_clean":"2023-03-29T10:59:13.637638+0000","last_became_active":"2023-03-29T10:59:11.836419+0000","last_became_peered":"2023-03-29T10:59:11.836419+0000","last_unstale":"2023-03-29T10:59:13.637638+0000","last_undegraded":"2023-03-29T10:59:13.637638+0000","last_fullsized":"2023-03-29T10:59:13.637638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:14:15.356259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.522995+0000","last_change":"2023-03-29T10:59:12.339181+0000","last_active":"2023-03-29T10:59:13.522995+0000","last_peered":"2023-03-29T10:59:13.522995+0000","last_clean":"2023-03-29T10:59:13.522995+0000","last_became_active":"2023-03-29T10:59:12.338919+0000","last_became_peered":"2023-03-29T10:59:12.338919+0000","last_unstale":"2023-03-29T10:59:13.522995+0000","last_undegraded":"2023-03-29T10:59:13.522995+0000","last_fullsized":"2023-03-29T10:59:13.522995+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:56:05.521539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.314087+0000","last_change":"2023-03-29T10:59:11.840807+0000","last_active":"2023-03-29T10:59:13.314087+0000","last_peered":"2023-03-29T10:59:13.314087+0000","last_clean":"2023-03-29T10:59:13.314087+0000","last_became_active":"2023-03-29T10:59:11.840428+0000","last_became_peered":"2023-03-29T10:59:11.840428+0000","last_unstale":"2023-03-29T10:59:13.314087+0000","last_undegraded":"2023-03-29T10:59:13.314087+0000","last_fullsized":"2023-03-29T10:59:13.314087+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:28:42.096962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313937+0000","last_change":"2023-03-29T10:59:12.324099+0000","last_active":"2023-03-29T10:59:13.313937+0000","last_peered":"2023-03-29T10:59:13.313937+0000","last_clean":"2023-03-29T10:59:13.313937+0000","last_became_active":"2023-03-29T10:59:12.323837+0000","last_became_peered":"2023-03-29T10:59:12.323837+0000","last_unstale":"2023-03-29T10:59:13.313937+0000","last_undegraded":"2023-03-29T10:59:13.313937+0000","last_fullsized":"2023-03-29T10:59:13.313937+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:18:30.372255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.308795+0000","last_change":"2023-03-29T10:59:11.838153+0000","last_active":"2023-03-29T10:59:13.308795+0000","last_peered":"2023-03-29T10:59:13.308795+0000","last_clean":"2023-03-29T10:59:13.308795+0000","last_became_active":"2023-03-29T10:59:11.837886+0000","last_became_peered":"2023-03-29T10:59:11.837886+0000","last_unstale":"2023-03-29T10:59:13.308795+0000","last_undegraded":"2023-03-29T10:59:13.308795+0000","last_fullsized":"2023-03-29T10:59:13.308795+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:52:55.411525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718359+0000","last_change":"2023-03-29T10:59:12.328294+0000","last_active":"2023-03-29T10:59:13.718359+0000","last_peered":"2023-03-29T10:59:13.718359+0000","last_clean":"2023-03-29T10:59:13.718359+0000","last_became_active":"2023-03-29T10:59:12.327976+0000","last_became_peered":"2023-03-29T10:59:12.327976+0000","last_unstale":"2023-03-29T10:59:13.718359+0000","last_undegraded":"2023-03-29T10:59:13.718359+0000","last_fullsized":"2023-03-29T10:59:13.718359+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:28:43.102190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.523197+0000","last_change":"2023-03-29T10:59:11.834754+0000","last_active":"2023-03-29T10:59:13.523197+0000","last_peered":"2023-03-29T10:59:13.523197+0000","last_clean":"2023-03-29T10:59:13.523197+0000","last_became_active":"2023-03-29T10:59:11.834488+0000","last_became_peered":"2023-03-29T10:59:11.834488+0000","last_unstale":"2023-03-29T10:59:13.523197+0000","last_undegraded":"2023-03-29T10:59:13.523197+0000","last_fullsized":"2023-03-29T10:59:13.523197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:07:59.525950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637230+0000","last_change":"2023-03-29T10:59:12.334159+0000","last_active":"2023-03-29T10:59:13.637230+0000","last_peered":"2023-03-29T10:59:13.637230+0000","last_clean":"2023-03-29T10:59:13.637230+0000","last_became_active":"2023-03-29T10:59:12.333911+0000","last_became_peered":"2023-03-29T10:59:12.333911+0000","last_unstale":"2023-03-29T10:59:13.637230+0000","last_undegraded":"2023-03-29T10:59:13.637230+0000","last_fullsized":"2023-03-29T10:59:13.637230+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:05:53.919758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313894+0000","last_change":"2023-03-29T10:59:11.835498+0000","last_active":"2023-03-29T10:59:13.313894+0000","last_peered":"2023-03-29T10:59:13.313894+0000","last_clean":"2023-03-29T10:59:13.313894+0000","last_became_active":"2023-03-29T10:59:11.835175+0000","last_became_peered":"2023-03-29T10:59:11.835175+0000","last_unstale":"2023-03-29T10:59:13.313894+0000","last_undegraded":"2023-03-29T10:59:13.313894+0000","last_fullsized":"2023-03-29T10:59:13.313894+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:07:40.577932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.522747+0000","last_change":"2023-03-29T10:59:12.328454+0000","last_active":"2023-03-29T10:59:13.522747+0000","last_peered":"2023-03-29T10:59:13.522747+0000","last_clean":"2023-03-29T10:59:13.522747+0000","last_became_active":"2023-03-29T10:59:12.328200+0000","last_became_peered":"2023-03-29T10:59:12.328200+0000","last_unstale":"2023-03-29T10:59:13.522747+0000","last_undegraded":"2023-03-29T10:59:13.522747+0000","last_fullsized":"2023-03-29T10:59:13.522747+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:06:46.183216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.522816+0000","last_change":"2023-03-29T10:59:11.835242+0000","last_active":"2023-03-29T10:59:13.522816+0000","last_peered":"2023-03-29T10:59:13.522816+0000","last_clean":"2023-03-29T10:59:13.522816+0000","last_became_active":"2023-03-29T10:59:11.834983+0000","last_became_peered":"2023-03-29T10:59:11.834983+0000","last_unstale":"2023-03-29T10:59:13.522816+0000","last_undegraded":"2023-03-29T10:59:13.522816+0000","last_fullsized":"2023-03-29T10:59:13.522816+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:18:29.367027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313957+0000","last_change":"2023-03-29T10:59:12.321082+0000","last_active":"2023-03-29T10:59:13.313957+0000","last_peered":"2023-03-29T10:59:13.313957+0000","last_clean":"2023-03-29T10:59:13.313957+0000","last_became_active":"2023-03-29T10:59:12.320808+0000","last_became_peered":"2023-03-29T10:59:12.320808+0000","last_unstale":"2023-03-29T10:59:13.313957+0000","last_undegraded":"2023-03-29T10:59:13.313957+0000","last_fullsized":"2023-03-29T10:59:13.313957+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:50:49.454488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637228+0000","last_change":"2023-03-29T10:59:11.835056+0000","last_active":"2023-03-29T10:59:13.637228+0000","last_peered":"2023-03-29T10:59:13.637228+0000","last_clean":"2023-03-29T10:59:13.637228+0000","last_became_active":"2023-03-29T10:59:11.834456+0000","last_became_peered":"2023-03-29T10:59:11.834456+0000","last_unstale":"2023-03-29T10:59:13.637228+0000","last_undegraded":"2023-03-29T10:59:13.637228+0000","last_fullsized":"2023-03-29T10:59:13.637228+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:00:07.668045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.308811+0000","last_change":"2023-03-29T10:59:12.316834+0000","last_active":"2023-03-29T10:59:13.308811+0000","last_peered":"2023-03-29T10:59:13.308811+0000","last_clean":"2023-03-29T10:59:13.308811+0000","last_became_active":"2023-03-29T10:59:12.316587+0000","last_became_peered":"2023-03-29T10:59:12.316587+0000","last_unstale":"2023-03-29T10:59:13.308811+0000","last_undegraded":"2023-03-29T10:59:13.308811+0000","last_fullsized":"2023-03-29T10:59:13.308811+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:57:29.081200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.308825+0000","last_change":"2023-03-29T10:59:11.839924+0000","last_active":"2023-03-29T10:59:13.308825+0000","last_peered":"2023-03-29T10:59:13.308825+0000","last_clean":"2023-03-29T10:59:13.308825+0000","last_became_active":"2023-03-29T10:59:11.839675+0000","last_became_peered":"2023-03-29T10:59:11.839675+0000","last_unstale":"2023-03-29T10:59:13.308825+0000","last_undegraded":"2023-03-29T10:59:13.308825+0000","last_fullsized":"2023-03-29T10:59:13.308825+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:45:50.136907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637503+0000","last_change":"2023-03-29T10:59:12.341367+0000","last_active":"2023-03-29T10:59:13.637503+0000","last_peered":"2023-03-29T10:59:13.637503+0000","last_clean":"2023-03-29T10:59:13.637503+0000","last_became_active":"2023-03-29T10:59:12.341098+0000","last_became_peered":"2023-03-29T10:59:12.341098+0000","last_unstale":"2023-03-29T10:59:13.637503+0000","last_undegraded":"2023-03-29T10:59:13.637503+0000","last_fullsized":"2023-03-29T10:59:13.637503+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:04:11.191595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719096+0000","last_change":"2023-03-29T10:59:11.840768+0000","last_active":"2023-03-29T10:59:13.719096+0000","last_peered":"2023-03-29T10:59:13.719096+0000","last_clean":"2023-03-29T10:59:13.719096+0000","last_became_active":"2023-03-29T10:59:11.840493+0000","last_became_peered":"2023-03-29T10:59:11.840493+0000","last_unstale":"2023-03-29T10:59:13.719096+0000","last_undegraded":"2023-03-29T10:59:13.719096+0000","last_fullsized":"2023-03-29T10:59:13.719096+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:40:07.850917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719184+0000","last_change":"2023-03-29T10:59:12.334169+0000","last_active":"2023-03-29T10:59:13.719184+0000","last_peered":"2023-03-29T10:59:13.719184+0000","last_clean":"2023-03-29T10:59:13.719184+0000","last_became_active":"2023-03-29T10:59:12.333918+0000","last_became_peered":"2023-03-29T10:59:12.333918+0000","last_unstale":"2023-03-29T10:59:13.719184+0000","last_undegraded":"2023-03-29T10:59:13.719184+0000","last_fullsized":"2023-03-29T10:59:13.719184+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:48:31.147706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718188+0000","last_change":"2023-03-29T10:59:11.833433+0000","last_active":"2023-03-29T10:59:13.718188+0000","last_peered":"2023-03-29T10:59:13.718188+0000","last_clean":"2023-03-29T10:59:13.718188+0000","last_became_active":"2023-03-29T10:59:11.833119+0000","last_became_peered":"2023-03-29T10:59:11.833119+0000","last_unstale":"2023-03-29T10:59:13.718188+0000","last_undegraded":"2023-03-29T10:59:13.718188+0000","last_fullsized":"2023-03-29T10:59:13.718188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:40:07.850917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719435+0000","last_change":"2023-03-29T10:59:12.324243+0000","last_active":"2023-03-29T10:59:13.719435+0000","last_peered":"2023-03-29T10:59:13.719435+0000","last_clean":"2023-03-29T10:59:13.719435+0000","last_became_active":"2023-03-29T10:59:12.323921+0000","last_became_peered":"2023-03-29T10:59:12.323921+0000","last_unstale":"2023-03-29T10:59:13.719435+0000","last_undegraded":"2023-03-29T10:59:13.719435+0000","last_fullsized":"2023-03-29T10:59:13.719435+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:13:18.091340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.523104+0000","last_change":"2023-03-29T10:59:11.840764+0000","last_active":"2023-03-29T10:59:13.523104+0000","last_peered":"2023-03-29T10:59:13.523104+0000","last_clean":"2023-03-29T10:59:13.523104+0000","last_became_active":"2023-03-29T10:59:11.840459+0000","last_became_peered":"2023-03-29T10:59:11.840459+0000","last_unstale":"2023-03-29T10:59:13.523104+0000","last_undegraded":"2023-03-29T10:59:13.523104+0000","last_fullsized":"2023-03-29T10:59:13.523104+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:58:54.085911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.522872+0000","last_change":"2023-03-29T10:59:12.339015+0000","last_active":"2023-03-29T10:59:13.522872+0000","last_peered":"2023-03-29T10:59:13.522872+0000","last_clean":"2023-03-29T10:59:13.522872+0000","last_became_active":"2023-03-29T10:59:12.338775+0000","last_became_peered":"2023-03-29T10:59:12.338775+0000","last_unstale":"2023-03-29T10:59:13.522872+0000","last_undegraded":"2023-03-29T10:59:13.522872+0000","last_fullsized":"2023-03-29T10:59:13.522872+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:58:16.409418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.523284+0000","last_change":"2023-03-29T10:59:11.826976+0000","last_active":"2023-03-29T10:59:13.523284+0000","last_peered":"2023-03-29T10:59:13.523284+0000","last_clean":"2023-03-29T10:59:13.523284+0000","last_became_active":"2023-03-29T10:59:11.826702+0000","last_became_peered":"2023-03-29T10:59:11.826702+0000","last_unstale":"2023-03-29T10:59:13.523284+0000","last_undegraded":"2023-03-29T10:59:13.523284+0000","last_fullsized":"2023-03-29T10:59:13.523284+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:14:15.356259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313293+0000","last_change":"2023-03-29T10:59:12.322297+0000","last_active":"2023-03-29T10:59:13.313293+0000","last_peered":"2023-03-29T10:59:13.313293+0000","last_clean":"2023-03-29T10:59:13.313293+0000","last_became_active":"2023-03-29T10:59:12.321962+0000","last_became_peered":"2023-03-29T10:59:12.321962+0000","last_unstale":"2023-03-29T10:59:13.313293+0000","last_undegraded":"2023-03-29T10:59:13.313293+0000","last_fullsized":"2023-03-29T10:59:13.313293+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:28:43.102190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718016+0000","last_change":"2023-03-29T10:59:11.840676+0000","last_active":"2023-03-29T10:59:13.718016+0000","last_peered":"2023-03-29T10:59:13.718016+0000","last_clean":"2023-03-29T10:59:13.718016+0000","last_became_active":"2023-03-29T10:59:11.840214+0000","last_became_peered":"2023-03-29T10:59:11.840214+0000","last_unstale":"2023-03-29T10:59:13.718016+0000","last_undegraded":"2023-03-29T10:59:13.718016+0000","last_fullsized":"2023-03-29T10:59:13.718016+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:46:14.252695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.308496+0000","last_change":"2023-03-29T10:59:12.317597+0000","last_active":"2023-03-29T10:59:13.308496+0000","last_peered":"2023-03-29T10:59:13.308496+0000","last_clean":"2023-03-29T10:59:13.308496+0000","last_became_active":"2023-03-29T10:59:12.317340+0000","last_became_peered":"2023-03-29T10:59:12.317340+0000","last_unstale":"2023-03-29T10:59:13.308496+0000","last_undegraded":"2023-03-29T10:59:13.308496+0000","last_fullsized":"2023-03-29T10:59:13.308496+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:21:09.442824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637194+0000","last_change":"2023-03-29T10:59:11.837124+0000","last_active":"2023-03-29T10:59:13.637194+0000","last_peered":"2023-03-29T10:59:13.637194+0000","last_clean":"2023-03-29T10:59:13.637194+0000","last_became_active":"2023-03-29T10:59:11.836884+0000","last_became_peered":"2023-03-29T10:59:11.836884+0000","last_unstale":"2023-03-29T10:59:13.637194+0000","last_undegraded":"2023-03-29T10:59:13.637194+0000","last_fullsized":"2023-03-29T10:59:13.637194+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:58:54.085911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637079+0000","last_change":"2023-03-29T10:59:12.326052+0000","last_active":"2023-03-29T10:59:13.637079+0000","last_peered":"2023-03-29T10:59:13.637079+0000","last_clean":"2023-03-29T10:59:13.637079+0000","last_became_active":"2023-03-29T10:59:12.325717+0000","last_became_peered":"2023-03-29T10:59:12.325717+0000","last_unstale":"2023-03-29T10:59:13.637079+0000","last_undegraded":"2023-03-29T10:59:13.637079+0000","last_fullsized":"2023-03-29T10:59:13.637079+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:20:37.878273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313485+0000","last_change":"2023-03-29T10:59:11.835546+0000","last_active":"2023-03-29T10:59:13.313485+0000","last_peered":"2023-03-29T10:59:13.313485+0000","last_clean":"2023-03-29T10:59:13.313485+0000","last_became_active":"2023-03-29T10:59:11.835013+0000","last_became_peered":"2023-03-29T10:59:11.835013+0000","last_unstale":"2023-03-29T10:59:13.313485+0000","last_undegraded":"2023-03-29T10:59:13.313485+0000","last_fullsized":"2023-03-29T10:59:13.313485+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:34:23.355075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719137+0000","last_change":"2023-03-29T10:59:12.339213+0000","last_active":"2023-03-29T10:59:13.719137+0000","last_peered":"2023-03-29T10:59:13.719137+0000","last_clean":"2023-03-29T10:59:13.719137+0000","last_became_active":"2023-03-29T10:59:12.339024+0000","last_became_peered":"2023-03-29T10:59:12.339024+0000","last_unstale":"2023-03-29T10:59:13.719137+0000","last_undegraded":"2023-03-29T10:59:13.719137+0000","last_fullsized":"2023-03-29T10:59:13.719137+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:56:05.521539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637356+0000","last_change":"2023-03-29T10:59:11.840280+0000","last_active":"2023-03-29T10:59:13.637356+0000","last_peered":"2023-03-29T10:59:13.637356+0000","last_clean":"2023-03-29T10:59:13.637356+0000","last_became_active":"2023-03-29T10:59:11.839982+0000","last_became_peered":"2023-03-29T10:59:11.839982+0000","last_unstale":"2023-03-29T10:59:13.637356+0000","last_undegraded":"2023-03-29T10:59:13.637356+0000","last_fullsized":"2023-03-29T10:59:13.637356+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:25:52.375709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-29T10:59:12.310697+0000","last_change":"2023-03-29T10:57:22.318583+0000","last_active":"2023-03-29T10:59:12.310697+0000","last_peered":"2023-03-29T10:59:12.310697+0000","last_clean":"2023-03-29T10:59:12.310697+0000","last_became_active":"2023-03-29T10:57:22.318223+0000","last_became_peered":"2023-03-29T10:57:22.318223+0000","last_unstale":"2023-03-29T10:59:12.310697+0000","last_undegraded":"2023-03-29T10:59:12.310697+0000","last_fullsized":"2023-03-29T10:59:12.310697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:31:03.349241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719354+0000","last_change":"2023-03-29T10:59:12.330220+0000","last_active":"2023-03-29T10:59:13.719354+0000","last_peered":"2023-03-29T10:59:13.719354+0000","last_clean":"2023-03-29T10:59:13.719354+0000","last_became_active":"2023-03-29T10:59:12.329936+0000","last_became_peered":"2023-03-29T10:59:12.329936+0000","last_unstale":"2023-03-29T10:59:13.719354+0000","last_undegraded":"2023-03-29T10:59:13.719354+0000","last_fullsized":"2023-03-29T10:59:13.719354+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:57:29.081200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"43'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:14.027623+0000","last_change":"2023-03-29T10:59:11.841525+0000","last_active":"2023-03-29T10:59:14.027623+0000","last_peered":"2023-03-29T10:59:14.027623+0000","last_clean":"2023-03-29T10:59:14.027623+0000","last_became_active":"2023-03-29T10:59:11.841261+0000","last_became_peered":"2023-03-29T10:59:11.841261+0000","last_unstale":"2023-03-29T10:59:14.027623+0000","last_undegraded":"2023-03-29T10:59:14.027623+0000","last_fullsized":"2023-03-29T10:59:14.027623+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:42:56.020246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":30,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-29T10:59:12.305983+0000","last_change":"2023-03-29T10:57:22.320330+0000","last_active":"2023-03-29T10:59:12.305983+0000","last_peered":"2023-03-29T10:59:12.305983+0000","last_clean":"2023-03-29T10:59:12.305983+0000","last_became_active":"2023-03-29T10:57:22.320011+0000","last_became_peered":"2023-03-29T10:57:22.320011+0000","last_unstale":"2023-03-29T10:59:12.305983+0000","last_undegraded":"2023-03-29T10:59:12.305983+0000","last_fullsized":"2023-03-29T10:59:12.305983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:47:01.099069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637384+0000","last_change":"2023-03-29T10:59:12.325157+0000","last_active":"2023-03-29T10:59:13.637384+0000","last_peered":"2023-03-29T10:59:13.637384+0000","last_clean":"2023-03-29T10:59:13.637384+0000","last_became_active":"2023-03-29T10:59:12.324927+0000","last_became_peered":"2023-03-29T10:59:12.324927+0000","last_unstale":"2023-03-29T10:59:13.637384+0000","last_undegraded":"2023-03-29T10:59:13.637384+0000","last_fullsized":"2023-03-29T10:59:13.637384+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:22:01.039112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.314064+0000","last_change":"2023-03-29T10:59:11.837671+0000","last_active":"2023-03-29T10:59:13.314064+0000","last_peered":"2023-03-29T10:59:13.314064+0000","last_clean":"2023-03-29T10:59:13.314064+0000","last_became_active":"2023-03-29T10:59:11.837354+0000","last_became_peered":"2023-03-29T10:59:11.837354+0000","last_unstale":"2023-03-29T10:59:13.314064+0000","last_undegraded":"2023-03-29T10:59:13.314064+0000","last_fullsized":"2023-03-29T10:59:13.314064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:48:30.142478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-29T10:59:12.305312+0000","last_change":"2023-03-29T10:57:22.467148+0000","last_active":"2023-03-29T10:59:12.305312+0000","last_peered":"2023-03-29T10:59:12.305312+0000","last_clean":"2023-03-29T10:59:12.305312+0000","last_became_active":"2023-03-29T10:57:22.466822+0000","last_became_peered":"2023-03-29T10:57:22.466822+0000","last_unstale":"2023-03-29T10:59:12.305312+0000","last_undegraded":"2023-03-29T10:59:12.305312+0000","last_fullsized":"2023-03-29T10:59:12.305312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:53:23.007205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637591+0000","last_change":"2023-03-29T10:59:12.334749+0000","last_active":"2023-03-29T10:59:13.637591+0000","last_peered":"2023-03-29T10:59:13.637591+0000","last_clean":"2023-03-29T10:59:13.637591+0000","last_became_active":"2023-03-29T10:59:12.334494+0000","last_became_peered":"2023-03-29T10:59:12.334494+0000","last_unstale":"2023-03-29T10:59:13.637591+0000","last_undegraded":"2023-03-29T10:59:13.637591+0000","last_fullsized":"2023-03-29T10:59:13.637591+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:06:46.183216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718494+0000","last_change":"2023-03-29T10:59:11.825421+0000","last_active":"2023-03-29T10:59:13.718494+0000","last_peered":"2023-03-29T10:59:13.718494+0000","last_clean":"2023-03-29T10:59:13.718494+0000","last_became_active":"2023-03-29T10:59:11.825124+0000","last_became_peered":"2023-03-29T10:59:11.825124+0000","last_unstale":"2023-03-29T10:59:13.718494+0000","last_undegraded":"2023-03-29T10:59:13.718494+0000","last_fullsized":"2023-03-29T10:59:13.718494+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:39:48.277218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":30,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-29T10:59:12.305875+0000","last_change":"2023-03-29T10:57:22.321523+0000","last_active":"2023-03-29T10:59:12.305875+0000","last_peered":"2023-03-29T10:59:12.305875+0000","last_clean":"2023-03-29T10:59:12.305875+0000","last_became_active":"2023-03-29T10:57:22.321183+0000","last_became_peered":"2023-03-29T10:57:22.321183+0000","last_unstale":"2023-03-29T10:59:12.305875+0000","last_undegraded":"2023-03-29T10:59:12.305875+0000","last_fullsized":"2023-03-29T10:59:12.305875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:35:11.664657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":186,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-29T10:59:12.305133+0000","last_change":"2023-03-29T10:57:18.176665+0000","last_active":"2023-03-29T10:59:12.305133+0000","last_peered":"2023-03-29T10:59:12.305133+0000","last_clean":"2023-03-29T10:59:12.305133+0000","last_became_active":"2023-03-29T10:57:18.175739+0000","last_became_peered":"2023-03-29T10:57:18.175739+0000","last_unstale":"2023-03-29T10:59:12.305133+0000","last_undegraded":"2023-03-29T10:59:12.305133+0000","last_fullsized":"2023-03-29T10:59:12.305133+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:16.275989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:16.275989+0000","last_clean_scrub_stamp":"2023-03-29T10:57:16.275989+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:06:32.872344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313983+0000","last_change":"2023-03-29T10:59:12.314644+0000","last_active":"2023-03-29T10:59:13.313983+0000","last_peered":"2023-03-29T10:59:13.313983+0000","last_clean":"2023-03-29T10:59:13.313983+0000","last_became_active":"2023-03-29T10:59:12.314361+0000","last_became_peered":"2023-03-29T10:59:12.314361+0000","last_unstale":"2023-03-29T10:59:13.313983+0000","last_undegraded":"2023-03-29T10:59:13.313983+0000","last_fullsized":"2023-03-29T10:59:13.313983+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:08:00.531178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313157+0000","last_change":"2023-03-29T10:59:11.833778+0000","last_active":"2023-03-29T10:59:13.313157+0000","last_peered":"2023-03-29T10:59:13.313157+0000","last_clean":"2023-03-29T10:59:13.313157+0000","last_became_active":"2023-03-29T10:59:11.833478+0000","last_became_peered":"2023-03-29T10:59:11.833478+0000","last_unstale":"2023-03-29T10:59:13.313157+0000","last_undegraded":"2023-03-29T10:59:13.313157+0000","last_fullsized":"2023-03-29T10:59:13.313157+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:52:55.411525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":33,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-29T10:59:12.305230+0000","last_change":"2023-03-29T10:57:22.322607+0000","last_active":"2023-03-29T10:59:12.305230+0000","last_peered":"2023-03-29T10:59:12.305230+0000","last_clean":"2023-03-29T10:59:12.305230+0000","last_became_active":"2023-03-29T10:57:22.321613+0000","last_became_peered":"2023-03-29T10:57:22.321613+0000","last_unstale":"2023-03-29T10:59:12.305230+0000","last_undegraded":"2023-03-29T10:59:12.305230+0000","last_fullsized":"2023-03-29T10:59:12.305230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:35:11.664657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.717916+0000","last_change":"2023-03-29T10:59:12.318928+0000","last_active":"2023-03-29T10:59:13.717916+0000","last_peered":"2023-03-29T10:59:13.717916+0000","last_clean":"2023-03-29T10:59:13.717916+0000","last_became_active":"2023-03-29T10:59:12.318485+0000","last_became_peered":"2023-03-29T10:59:12.318485+0000","last_unstale":"2023-03-29T10:59:13.717916+0000","last_undegraded":"2023-03-29T10:59:13.717916+0000","last_fullsized":"2023-03-29T10:59:13.717916+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:20:37.878273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313161+0000","last_change":"2023-03-29T10:59:11.840495+0000","last_active":"2023-03-29T10:59:13.313161+0000","last_peered":"2023-03-29T10:59:13.313161+0000","last_clean":"2023-03-29T10:59:13.313161+0000","last_became_active":"2023-03-29T10:59:11.840245+0000","last_became_peered":"2023-03-29T10:59:11.840245+0000","last_unstale":"2023-03-29T10:59:13.313161+0000","last_undegraded":"2023-03-29T10:59:13.313161+0000","last_fullsized":"2023-03-29T10:59:13.313161+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:52:57.622991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-29T10:59:12.306604+0000","last_change":"2023-03-29T10:57:22.315905+0000","last_active":"2023-03-29T10:59:12.306604+0000","last_peered":"2023-03-29T10:59:12.306604+0000","last_clean":"2023-03-29T10:59:12.306604+0000","last_became_active":"2023-03-29T10:57:22.315603+0000","last_became_peered":"2023-03-29T10:57:22.315603+0000","last_unstale":"2023-03-29T10:59:12.306604+0000","last_undegraded":"2023-03-29T10:59:12.306604+0000","last_fullsized":"2023-03-29T10:59:12.306604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T10:59:44.030013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.523131+0000","last_change":"2023-03-29T10:59:12.332368+0000","last_active":"2023-03-29T10:59:13.523131+0000","last_peered":"2023-03-29T10:59:13.523131+0000","last_clean":"2023-03-29T10:59:13.523131+0000","last_became_active":"2023-03-29T10:59:12.332085+0000","last_became_peered":"2023-03-29T10:59:12.332085+0000","last_unstale":"2023-03-29T10:59:13.523131+0000","last_undegraded":"2023-03-29T10:59:13.523131+0000","last_fullsized":"2023-03-29T10:59:13.523131+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:26:51.059416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719288+0000","last_change":"2023-03-29T10:59:11.840268+0000","last_active":"2023-03-29T10:59:13.719288+0000","last_peered":"2023-03-29T10:59:13.719288+0000","last_clean":"2023-03-29T10:59:13.719288+0000","last_became_active":"2023-03-29T10:59:11.839947+0000","last_became_peered":"2023-03-29T10:59:11.839947+0000","last_unstale":"2023-03-29T10:59:13.719288+0000","last_undegraded":"2023-03-29T10:59:13.719288+0000","last_fullsized":"2023-03-29T10:59:13.719288+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:46:14.252695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-29T10:59:12.309709+0000","last_change":"2023-03-29T10:57:22.319554+0000","last_active":"2023-03-29T10:59:12.309709+0000","last_peered":"2023-03-29T10:59:12.309709+0000","last_clean":"2023-03-29T10:59:12.309709+0000","last_became_active":"2023-03-29T10:57:22.319198+0000","last_became_peered":"2023-03-29T10:57:22.319198+0000","last_unstale":"2023-03-29T10:59:12.309709+0000","last_undegraded":"2023-03-29T10:59:12.309709+0000","last_fullsized":"2023-03-29T10:59:12.309709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:47:01.099069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.523002+0000","last_change":"2023-03-29T10:59:12.332711+0000","last_active":"2023-03-29T10:59:13.523002+0000","last_peered":"2023-03-29T10:59:13.523002+0000","last_clean":"2023-03-29T10:59:13.523002+0000","last_became_active":"2023-03-29T10:59:12.332410+0000","last_became_peered":"2023-03-29T10:59:12.332410+0000","last_unstale":"2023-03-29T10:59:13.523002+0000","last_undegraded":"2023-03-29T10:59:13.523002+0000","last_fullsized":"2023-03-29T10:59:13.523002+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:20:37.878273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313910+0000","last_change":"2023-03-29T10:59:11.840781+0000","last_active":"2023-03-29T10:59:13.313910+0000","last_peered":"2023-03-29T10:59:13.313910+0000","last_clean":"2023-03-29T10:59:13.313910+0000","last_became_active":"2023-03-29T10:59:11.840480+0000","last_became_peered":"2023-03-29T10:59:11.840480+0000","last_unstale":"2023-03-29T10:59:13.313910+0000","last_undegraded":"2023-03-29T10:59:13.313910+0000","last_fullsized":"2023-03-29T10:59:13.313910+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:35:56.995284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-29T10:59:12.314367+0000","last_change":"2023-03-29T10:57:22.322319+0000","last_active":"2023-03-29T10:59:12.314367+0000","last_peered":"2023-03-29T10:59:12.314367+0000","last_clean":"2023-03-29T10:59:12.314367+0000","last_became_active":"2023-03-29T10:57:22.321943+0000","last_became_peered":"2023-03-29T10:57:22.321943+0000","last_unstale":"2023-03-29T10:59:12.314367+0000","last_undegraded":"2023-03-29T10:59:12.314367+0000","last_fullsized":"2023-03-29T10:59:12.314367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:58:04.969453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2366,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9848,"kb_used_data":1072,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94361992,"statfs":{"total":96636764160,"available":96626679808,"internally_reserved":0,"allocated":1097728,"data_stored":813905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 29 10:58:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed Mar 29 10:58:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.496},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.69099999999999995}]},{"osd":2,"last update":"Wed Mar 29 10:58:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.53600000000000003}]},{"osd":3,"last update":"Wed Mar 29 10:58:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.60199999999999998}]},{"osd":4,"last update":"Wed Mar 29 10:58:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.52000000000000002}]},{"osd":5,"last update":"Wed Mar 29 10:58:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.626},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.73199999999999998}]},{"osd":6,"last update":"Wed Mar 29 10:58:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8632,"kb_used_data":496,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363208,"statfs":{"total":96636764160,"available":96627924992,"internally_reserved":0,"allocated":507904,"data_stored":224119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.79900000000000004}]},{"osd":1,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.70899999999999996}]},{"osd":2,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.755}]},{"osd":3,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.73399999999999999}]},{"osd":4,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.82799999999999996}]},{"osd":5,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.77900000000000003}]},{"osd":7,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.81299999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8756,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363084,"statfs":{"total":96636764160,"available":96627798016,"internally_reserved":0,"allocated":503808,"data_stored":223517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 29 10:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.38},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.35699999999999998}]},{"osd":2,"last update":"Wed Mar 29 10:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.42499999999999999}]},{"osd":3,"last update":"Wed Mar 29 10:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.41599999999999998}]},{"osd":4,"last update":"Wed Mar 29 10:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.623}]},{"osd":5,"last update":"Wed Mar 29 10:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.48899999999999999}]},{"osd":6,"last update":"Wed Mar 29 10:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.57599999999999996}]},{"osd":7,"last update":"Wed Mar 29 10:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.59099999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9844,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94361996,"statfs":{"total":96636764160,"available":96626683904,"internally_reserved":0,"allocated":1093632,"data_stored":813234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.40100000000000002}]},{"osd":2,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.55400000000000005}]},{"osd":3,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.42299999999999999}]},{"osd":4,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.57699999999999996}]},{"osd":5,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":4.4610000000000003,"5min":4.4610000000000003,"15min":4.4610000000000003},"last":0.748},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.60699999999999998}]},{"osd":6,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":4.4820000000000002,"5min":4.4820000000000002,"15min":4.4820000000000002},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":4.4989999999999997,"5min":4.4989999999999997,"15min":4.4989999999999997},"last":0.71899999999999997}]},{"osd":7,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.63600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8636,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363204,"statfs":{"total":96636764160,"available":96627920896,"internally_reserved":0,"allocated":512000,"data_stored":224414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.5609999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":0.47899999999999998}]},{"osd":1,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":2.3570000000000002,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.40799999999999997}]},{"osd":3,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":0.439},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.54900000000000004}]},{"osd":4,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.46999999999999997}]},{"osd":5,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.4940000000000002,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":0.65300000000000002}]},{"osd":6,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.5899999999999999,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.63500000000000001}]},{"osd":7,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.528,"5min":2.528,"15min":2.528},"last":0.61899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8764,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363076,"statfs":{"total":96636764160,"available":96627789824,"internally_reserved":0,"allocated":512000,"data_stored":224089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.29099999999999998}]},{"osd":1,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.312}]},{"osd":2,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.42699999999999999}]},{"osd":4,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.46999999999999997}]},{"osd":5,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.63200000000000001}]},{"osd":6,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.56899999999999995}]},{"osd":7,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.58899999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8764,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363076,"statfs":{"total":96636764160,"available":96627789824,"internally_reserved":0,"allocated":512000,"data_stored":225011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.68700000000000006}]},{"osd":1,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.73799999999999999}]},{"osd":2,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.72299999999999998}]},{"osd":3,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":2.528,"5min":2.528,"15min":2.528},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.76200000000000001}]},{"osd":5,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.47599999999999998}]},{"osd":6,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.53700000000000003}]},{"osd":7,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.48799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8692,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363148,"statfs":{"total":96636764160,"available":96627863552,"internally_reserved":0,"allocated":503808,"data_stored":223517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 29 10:58:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.753}]},{"osd":1,"last update":"Wed Mar 29 10:58:16 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.72099999999999997}]},{"osd":2,"last update":"Wed Mar 29 10:58:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.628},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.66700000000000004}]},{"osd":3,"last update":"Wed Mar 29 10:58:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.3719999999999999,"5min":2.3719999999999999,"15min":2.3719999999999999},"last":0.69899999999999995}]},{"osd":4,"last update":"Wed Mar 29 10:58:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.82999999999999996}]},{"osd":6,"last update":"Wed Mar 29 10:58:16 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.85699999999999998}]},{"osd":7,"last update":"Wed Mar 29 10:58:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.80400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-03-29T10:59:16.639 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-03-29T10:59:16.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.842+0000 7f55bfdff700 1 Processor -- start 2023-03-29T10:59:16.843 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.842+0000 7f55bfdff700 1 -- start start 2023-03-29T10:59:16.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.843+0000 7f55bfdff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f55b815dce0 0x7f55b80cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:16.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.843+0000 7f55bfdff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f55b80cd9f0 0x7f55b80cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:16.844 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.843+0000 7f55bfdff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b80ce300 con 0x7f55b81594c0 2023-03-29T10:59:16.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.843+0000 7f55bfdff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b80b0dd0 con 0x7f55b815dce0 2023-03-29T10:59:16.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.843+0000 7f55bfdff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b80b0fc0 con 0x7f55b80cd9f0 2023-03-29T10:59:16.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.844+0000 7f55bd39a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f55b815dce0 0x7f55b80cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:16.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.844+0000 7f55bd39a700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f55b815dce0 0x7f55b80cd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40388/0 (socket says 172.21.15.111:40388) 2023-03-29T10:59:16.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.844+0000 7f55bd39a700 1 -- 172.21.15.111:0/4136157689 learned_addr learned my addr 172.21.15.111:0/4136157689 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:16.845 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.844+0000 7f55be39c700 1 --2- 172.21.15.111:0/4136157689 >> v2:172.21.15.119:3300/0 conn(0x7f55b80cd9f0 0x7f55b80cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:16.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.844+0000 7f55bd39a700 1 -- 172.21.15.111:0/4136157689 >> v2:172.21.15.119:3300/0 conn(0x7f55b80cd9f0 msgr2=0x7f55b80cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:16.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.844+0000 7f55bd39a700 1 --2- 172.21.15.111:0/4136157689 >> v2:172.21.15.119:3300/0 conn(0x7f55b80cd9f0 0x7f55b80cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:16.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.844+0000 7f55bd39a700 1 -- 172.21.15.111:0/4136157689 >> v1:172.21.15.119:6789/0 conn(0x7f55b81594c0 legacy=0x7f55b80b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:16.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.844+0000 7f55bd39a700 1 -- 172.21.15.111:0/4136157689 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b80cc9b0 con 0x7f55b815dce0 2023-03-29T10:59:16.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.845+0000 7f55bd39a700 1 --2- 172.21.15.111:0/4136157689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b815dce0 0x7f55b80cd470 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f55b80b4d80 tx=0x7f55a8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2bd37b63e37e6e9 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:16.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.845+0000 7f55bcb99700 1 -- 172.21.15.111:0/4136157689 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a800b870 con 0x7f55b815dce0 2023-03-29T10:59:16.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.845+0000 7f55bcb99700 1 -- 172.21.15.111:0/4136157689 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a800b9d0 con 0x7f55b815dce0 2023-03-29T10:59:16.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.845+0000 7f55bcb99700 1 -- 172.21.15.111:0/4136157689 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a800bd60 con 0x7f55b815dce0 2023-03-29T10:59:16.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.845+0000 7f55bfdff700 1 -- 172.21.15.111:0/4136157689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b815dce0 msgr2=0x7f55b80cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:16.846 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.845+0000 7f55bfdff700 1 --2- 172.21.15.111:0/4136157689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b815dce0 0x7f55b80cd470 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f55b80b4d80 tx=0x7f55a8009bd0 comp rx=0 tx=0).stop 2023-03-29T10:59:16.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.845+0000 7f55bfdff700 1 -- 172.21.15.111:0/4136157689 shutdown_connections 2023-03-29T10:59:16.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.845+0000 7f55bfdff700 1 --2- 172.21.15.111:0/4136157689 >> v2:172.21.15.119:3300/0 conn(0x7f55b80cd9f0 0x7f55b80cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:16.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.845+0000 7f55bfdff700 1 --2- 172.21.15.111:0/4136157689 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b815dce0 0x7f55b80cd470 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:16.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.845+0000 7f55bfdff700 1 -- 172.21.15.111:0/4136157689 >> 172.21.15.111:0/4136157689 conn(0x7f55b80cb6c0 msgr2=0x7f55b80cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:16.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.845+0000 7f55bfdff700 1 -- 172.21.15.111:0/4136157689 shutdown_connections 2023-03-29T10:59:16.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.846+0000 7f55bfdff700 1 -- 172.21.15.111:0/4136157689 wait complete. 2023-03-29T10:59:16.847 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.847+0000 7f55bfdff700 1 Processor -- start 2023-03-29T10:59:16.848 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.847+0000 7f55bfdff700 1 -- start start 2023-03-29T10:59:16.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.848+0000 7f55bfdff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b80cd9f0 0x7f55b81489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:16.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.848+0000 7f55bfdff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f55b81594c0 0x7f55b81f1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:16.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.848+0000 7f55bfdff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b815dce0 0x7f55b8147010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:16.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.848+0000 7f55bfdff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b8147940 con 0x7f55b815dce0 2023-03-29T10:59:16.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.848+0000 7f55bfdff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b8147c20 con 0x7f55b80cd9f0 2023-03-29T10:59:16.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.848+0000 7f55bfdff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b8147f00 con 0x7f55b81594c0 2023-03-29T10:59:16.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.848+0000 7f55be39c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b815dce0 0x7f55b8147010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:16.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.849+0000 7f55be39c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b815dce0 0x7f55b8147010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40394/0 (socket says 172.21.15.111:40394) 2023-03-29T10:59:16.849 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.849+0000 7f55be39c700 1 -- 172.21.15.111:0/2628278947 learned_addr learned my addr 172.21.15.111:0/2628278947 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:16.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.849+0000 7f55bdb9b700 1 --2- 172.21.15.111:0/2628278947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b80cd9f0 0x7f55b81489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:16.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.849+0000 7f55bd39a700 1 --2- 172.21.15.111:0/2628278947 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f55b81594c0 0x7f55b81f1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:16.850 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.849+0000 7f55be39c700 1 -- 172.21.15.111:0/2628278947 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f55b81594c0 msgr2=0x7f55b81f1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:16.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.849+0000 7f55be39c700 1 --2- 172.21.15.111:0/2628278947 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f55b81594c0 0x7f55b81f1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:16.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.849+0000 7f55be39c700 1 -- 172.21.15.111:0/2628278947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b80cd9f0 msgr2=0x7f55b81489e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:16.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.849+0000 7f55be39c700 1 --2- 172.21.15.111:0/2628278947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b80cd9f0 0x7f55b81489e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:16.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.849+0000 7f55be39c700 1 -- 172.21.15.111:0/2628278947 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a8009550 con 0x7f55b815dce0 2023-03-29T10:59:16.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.849+0000 7f55be39c700 1 --2- 172.21.15.111:0/2628278947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b815dce0 0x7f55b8147010 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f55b0007e40 tx=0x7f55b000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:16.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.849+0000 7f55ae7fc700 1 -- 172.21.15.111:0/2628278947 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b000eea0 con 0x7f55b815dce0 2023-03-29T10:59:16.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.849+0000 7f55ae7fc700 1 -- 172.21.15.111:0/2628278947 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b0014420 con 0x7f55b815dce0 2023-03-29T10:59:16.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.849+0000 7f55bfdff700 1 -- 172.21.15.111:0/2628278947 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55b80ccf70 con 0x7f55b815dce0 2023-03-29T10:59:16.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.849+0000 7f55ae7fc700 1 -- 172.21.15.111:0/2628278947 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b00147b0 con 0x7f55b815dce0 2023-03-29T10:59:16.851 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.850+0000 7f55bfdff700 1 -- 172.21.15.111:0/2628278947 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b80ccc90 con 0x7f55b815dce0 2023-03-29T10:59:16.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.851+0000 7f55bfdff700 1 -- 172.21.15.111:0/2628278947 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b8159890 con 0x7f55b815dce0 2023-03-29T10:59:16.852 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.851+0000 7f55ae7fc700 1 -- 172.21.15.111:0/2628278947 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f55b0014910 con 0x7f55b815dce0 2023-03-29T10:59:16.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.853+0000 7f55ae7fc700 1 --2- 172.21.15.111:0/2628278947 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f559c069de0 0x7f559c06c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:16.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.853+0000 7f55ae7fc700 1 -- 172.21.15.111:0/2628278947 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f55b0088930 con 0x7f55b815dce0 2023-03-29T10:59:16.853 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.853+0000 7f55bdb9b700 1 --2- 172.21.15.111:0/2628278947 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f559c069de0 0x7f559c06c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:16.854 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.853+0000 7f55bdb9b700 1 --2- 172.21.15.111:0/2628278947 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f559c069de0 0x7f559c06c290 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f55b4000f60 tx=0x7f55b4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:16.857 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:16.857+0000 7f55ae7fc700 1 -- 172.21.15.111:0/2628278947 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f55b00540a0 con 0x7f55b815dce0 2023-03-29T10:59:17.047 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.046+0000 7f55bfdff700 1 -- 172.21.15.111:0/2628278947 --> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f55b815d060 con 0x7f559c069de0 2023-03-29T10:59:17.056 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.055+0000 7f55ae7fc700 1 -- 172.21.15.111:0/2628278947 <== mgr.4103 v2:172.21.15.111:6832/110141 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311160 (secure 0 0 0) 0x7f55b815d060 con 0x7f559c069de0 2023-03-29T10:59:17.059 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:59:17.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.060+0000 7f55bfdff700 1 -- 172.21.15.111:0/2628278947 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f559c069de0 msgr2=0x7f559c06c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:17.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.060+0000 7f55bfdff700 1 --2- 172.21.15.111:0/2628278947 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f559c069de0 0x7f559c06c290 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f55b4000f60 tx=0x7f55b4009040 comp rx=0 tx=0).stop 2023-03-29T10:59:17.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.060+0000 7f55bfdff700 1 -- 172.21.15.111:0/2628278947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b815dce0 msgr2=0x7f55b8147010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:17.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.060+0000 7f55bfdff700 1 --2- 172.21.15.111:0/2628278947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b815dce0 0x7f55b8147010 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f55b0007e40 tx=0x7f55b000e3c0 comp rx=0 tx=0).stop 2023-03-29T10:59:17.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.060+0000 7f55bfdff700 1 -- 172.21.15.111:0/2628278947 shutdown_connections 2023-03-29T10:59:17.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.060+0000 7f55bfdff700 1 --2- 172.21.15.111:0/2628278947 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f559c069de0 0x7f559c06c290 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:17.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.060+0000 7f55bfdff700 1 --2- 172.21.15.111:0/2628278947 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f55b815dce0 0x7f55b8147010 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:17.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.060+0000 7f55bfdff700 1 --2- 172.21.15.111:0/2628278947 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55b80cd9f0 0x7f55b81489e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:17.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.060+0000 7f55bfdff700 1 --2- 172.21.15.111:0/2628278947 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f55b81594c0 0x7f55b81f1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:17.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.060+0000 7f55bfdff700 1 -- 172.21.15.111:0/2628278947 >> 172.21.15.111:0/2628278947 conn(0x7f55b80cb6c0 msgr2=0x7f55b80b3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:17.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.060+0000 7f55bfdff700 1 -- 172.21.15.111:0/2628278947 shutdown_connections 2023-03-29T10:59:17.061 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.060+0000 7f55bfdff700 1 -- 172.21.15.111:0/2628278947 wait complete. 2023-03-29T10:59:17.062 INFO:teuthology.orchestra.run.smithi111.stderr:dumped all 2023-03-29T10:59:17.078 INFO:teuthology.orchestra.run.smithi111.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2023-03-29T10:59:16.148971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592734,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":71936,"kb_used_data":5120,"kb_used_omap":0,"kb_used_meta":66752,"kb_avail":754902784,"statfs":{"total":773094113280,"available":773020450816,"internally_reserved":0,"allocated":5242880,"data_stored":2971806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68354048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2366,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.336731"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313958+0000","last_change":"2023-03-29T10:59:12.324127+0000","last_active":"2023-03-29T10:59:13.313958+0000","last_peered":"2023-03-29T10:59:13.313958+0000","last_clean":"2023-03-29T10:59:13.313958+0000","last_became_active":"2023-03-29T10:59:12.323800+0000","last_became_peered":"2023-03-29T10:59:12.323800+0000","last_unstale":"2023-03-29T10:59:13.313958+0000","last_undegraded":"2023-03-29T10:59:13.313958+0000","last_fullsized":"2023-03-29T10:59:13.313958+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:42:57.025474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718453+0000","last_change":"2023-03-29T10:59:12.329201+0000","last_active":"2023-03-29T10:59:13.718453+0000","last_peered":"2023-03-29T10:59:13.718453+0000","last_clean":"2023-03-29T10:59:13.718453+0000","last_became_active":"2023-03-29T10:59:12.328922+0000","last_became_peered":"2023-03-29T10:59:12.328922+0000","last_unstale":"2023-03-29T10:59:13.718453+0000","last_undegraded":"2023-03-29T10:59:13.718453+0000","last_fullsized":"2023-03-29T10:59:13.718453+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:34:44.160250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718830+0000","last_change":"2023-03-29T10:59:12.325166+0000","last_active":"2023-03-29T10:59:13.718830+0000","last_peered":"2023-03-29T10:59:13.718830+0000","last_clean":"2023-03-29T10:59:13.718830+0000","last_became_active":"2023-03-29T10:59:12.324879+0000","last_became_peered":"2023-03-29T10:59:12.324879+0000","last_unstale":"2023-03-29T10:59:13.718830+0000","last_undegraded":"2023-03-29T10:59:13.718830+0000","last_fullsized":"2023-03-29T10:59:13.718830+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:31:56.427766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637378+0000","last_change":"2023-03-29T10:59:12.337300+0000","last_active":"2023-03-29T10:59:13.637378+0000","last_peered":"2023-03-29T10:59:13.637378+0000","last_clean":"2023-03-29T10:59:13.637378+0000","last_became_active":"2023-03-29T10:59:12.337030+0000","last_became_peered":"2023-03-29T10:59:12.337030+0000","last_unstale":"2023-03-29T10:59:13.637378+0000","last_undegraded":"2023-03-29T10:59:13.637378+0000","last_fullsized":"2023-03-29T10:59:13.637378+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:31:56.427766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718167+0000","last_change":"2023-03-29T10:59:12.329253+0000","last_active":"2023-03-29T10:59:13.718167+0000","last_peered":"2023-03-29T10:59:13.718167+0000","last_clean":"2023-03-29T10:59:13.718167+0000","last_became_active":"2023-03-29T10:59:12.328998+0000","last_became_peered":"2023-03-29T10:59:12.328998+0000","last_unstale":"2023-03-29T10:59:13.718167+0000","last_undegraded":"2023-03-29T10:59:13.718167+0000","last_fullsized":"2023-03-29T10:59:13.718167+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:05:53.919758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.522861+0000","last_change":"2023-03-29T10:59:12.332574+0000","last_active":"2023-03-29T10:59:13.522861+0000","last_peered":"2023-03-29T10:59:13.522861+0000","last_clean":"2023-03-29T10:59:13.522861+0000","last_became_active":"2023-03-29T10:59:12.332261+0000","last_became_peered":"2023-03-29T10:59:12.332261+0000","last_unstale":"2023-03-29T10:59:13.522861+0000","last_undegraded":"2023-03-29T10:59:13.522861+0000","last_fullsized":"2023-03-29T10:59:13.522861+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:05:53.919758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313825+0000","last_change":"2023-03-29T10:59:12.323148+0000","last_active":"2023-03-29T10:59:13.313825+0000","last_peered":"2023-03-29T10:59:13.313825+0000","last_clean":"2023-03-29T10:59:13.313825+0000","last_became_active":"2023-03-29T10:59:12.322854+0000","last_became_peered":"2023-03-29T10:59:12.322854+0000","last_unstale":"2023-03-29T10:59:13.313825+0000","last_undegraded":"2023-03-29T10:59:13.313825+0000","last_fullsized":"2023-03-29T10:59:13.313825+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:25:53.380937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637031+0000","last_change":"2023-03-29T10:59:12.336206+0000","last_active":"2023-03-29T10:59:13.637031+0000","last_peered":"2023-03-29T10:59:13.637031+0000","last_clean":"2023-03-29T10:59:13.637031+0000","last_became_active":"2023-03-29T10:59:12.334168+0000","last_became_peered":"2023-03-29T10:59:12.334168+0000","last_unstale":"2023-03-29T10:59:13.637031+0000","last_undegraded":"2023-03-29T10:59:13.637031+0000","last_fullsized":"2023-03-29T10:59:13.637031+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:32:53.291657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637458+0000","last_change":"2023-03-29T10:59:12.325862+0000","last_active":"2023-03-29T10:59:13.637458+0000","last_peered":"2023-03-29T10:59:13.637458+0000","last_clean":"2023-03-29T10:59:13.637458+0000","last_became_active":"2023-03-29T10:59:12.325492+0000","last_became_peered":"2023-03-29T10:59:12.325492+0000","last_unstale":"2023-03-29T10:59:13.637458+0000","last_undegraded":"2023-03-29T10:59:13.637458+0000","last_fullsized":"2023-03-29T10:59:13.637458+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:18:03.154980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637519+0000","last_change":"2023-03-29T10:59:12.334651+0000","last_active":"2023-03-29T10:59:13.637519+0000","last_peered":"2023-03-29T10:59:13.637519+0000","last_clean":"2023-03-29T10:59:13.637519+0000","last_became_active":"2023-03-29T10:59:12.334332+0000","last_became_peered":"2023-03-29T10:59:12.334332+0000","last_unstale":"2023-03-29T10:59:13.637519+0000","last_undegraded":"2023-03-29T10:59:13.637519+0000","last_fullsized":"2023-03-29T10:59:13.637519+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:21:44.320888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637210+0000","last_change":"2023-03-29T10:59:12.335223+0000","last_active":"2023-03-29T10:59:13.637210+0000","last_peered":"2023-03-29T10:59:13.637210+0000","last_clean":"2023-03-29T10:59:13.637210+0000","last_became_active":"2023-03-29T10:59:12.335037+0000","last_became_peered":"2023-03-29T10:59:12.335037+0000","last_unstale":"2023-03-29T10:59:13.637210+0000","last_undegraded":"2023-03-29T10:59:13.637210+0000","last_fullsized":"2023-03-29T10:59:13.637210+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:34:44.160250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718623+0000","last_change":"2023-03-29T10:59:12.311070+0000","last_active":"2023-03-29T10:59:13.718623+0000","last_peered":"2023-03-29T10:59:13.718623+0000","last_clean":"2023-03-29T10:59:13.718623+0000","last_became_active":"2023-03-29T10:59:12.310696+0000","last_became_peered":"2023-03-29T10:59:12.310696+0000","last_unstale":"2023-03-29T10:59:13.718623+0000","last_undegraded":"2023-03-29T10:59:13.718623+0000","last_fullsized":"2023-03-29T10:59:13.718623+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:29:07.728609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.308691+0000","last_change":"2023-03-29T10:59:12.336284+0000","last_active":"2023-03-29T10:59:13.308691+0000","last_peered":"2023-03-29T10:59:13.308691+0000","last_clean":"2023-03-29T10:59:13.308691+0000","last_became_active":"2023-03-29T10:59:12.335973+0000","last_became_peered":"2023-03-29T10:59:12.335973+0000","last_unstale":"2023-03-29T10:59:13.308691+0000","last_undegraded":"2023-03-29T10:59:13.308691+0000","last_fullsized":"2023-03-29T10:59:13.308691+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:13:18.091340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637124+0000","last_change":"2023-03-29T10:59:12.323063+0000","last_active":"2023-03-29T10:59:13.637124+0000","last_peered":"2023-03-29T10:59:13.637124+0000","last_clean":"2023-03-29T10:59:13.637124+0000","last_became_active":"2023-03-29T10:59:12.322888+0000","last_became_peered":"2023-03-29T10:59:12.322888+0000","last_unstale":"2023-03-29T10:59:13.637124+0000","last_undegraded":"2023-03-29T10:59:13.637124+0000","last_fullsized":"2023-03-29T10:59:13.637124+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:04:15.453097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.522854+0000","last_change":"2023-03-29T10:59:12.333923+0000","last_active":"2023-03-29T10:59:13.522854+0000","last_peered":"2023-03-29T10:59:13.522854+0000","last_clean":"2023-03-29T10:59:13.522854+0000","last_became_active":"2023-03-29T10:59:12.333348+0000","last_became_peered":"2023-03-29T10:59:12.333348+0000","last_unstale":"2023-03-29T10:59:13.522854+0000","last_undegraded":"2023-03-29T10:59:13.522854+0000","last_fullsized":"2023-03-29T10:59:13.522854+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:34:44.160250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718559+0000","last_change":"2023-03-29T10:59:12.329173+0000","last_active":"2023-03-29T10:59:13.718559+0000","last_peered":"2023-03-29T10:59:13.718559+0000","last_clean":"2023-03-29T10:59:13.718559+0000","last_became_active":"2023-03-29T10:59:12.328874+0000","last_became_peered":"2023-03-29T10:59:12.328874+0000","last_unstale":"2023-03-29T10:59:13.718559+0000","last_undegraded":"2023-03-29T10:59:13.718559+0000","last_fullsized":"2023-03-29T10:59:13.718559+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:44.599265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718877+0000","last_change":"2023-03-29T10:59:12.321393+0000","last_active":"2023-03-29T10:59:13.718877+0000","last_peered":"2023-03-29T10:59:13.718877+0000","last_clean":"2023-03-29T10:59:13.718877+0000","last_became_active":"2023-03-29T10:59:12.321116+0000","last_became_peered":"2023-03-29T10:59:12.321116+0000","last_unstale":"2023-03-29T10:59:13.718877+0000","last_undegraded":"2023-03-29T10:59:13.718877+0000","last_fullsized":"2023-03-29T10:59:13.718877+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:04:11.191595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719440+0000","last_change":"2023-03-29T10:59:12.327840+0000","last_active":"2023-03-29T10:59:13.719440+0000","last_peered":"2023-03-29T10:59:13.719440+0000","last_clean":"2023-03-29T10:59:13.719440+0000","last_became_active":"2023-03-29T10:59:12.327137+0000","last_became_peered":"2023-03-29T10:59:12.327137+0000","last_unstale":"2023-03-29T10:59:13.719440+0000","last_undegraded":"2023-03-29T10:59:13.719440+0000","last_fullsized":"2023-03-29T10:59:13.719440+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:21:44.320888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719024+0000","last_change":"2023-03-29T10:59:12.332453+0000","last_active":"2023-03-29T10:59:13.719024+0000","last_peered":"2023-03-29T10:59:13.719024+0000","last_clean":"2023-03-29T10:59:13.719024+0000","last_became_active":"2023-03-29T10:59:12.331831+0000","last_became_peered":"2023-03-29T10:59:12.331831+0000","last_unstale":"2023-03-29T10:59:13.719024+0000","last_undegraded":"2023-03-29T10:59:13.719024+0000","last_fullsized":"2023-03-29T10:59:13.719024+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:16:57.395398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.308707+0000","last_change":"2023-03-29T10:59:12.326835+0000","last_active":"2023-03-29T10:59:13.308707+0000","last_peered":"2023-03-29T10:59:13.308707+0000","last_clean":"2023-03-29T10:59:13.308707+0000","last_became_active":"2023-03-29T10:59:12.326682+0000","last_became_peered":"2023-03-29T10:59:12.326682+0000","last_unstale":"2023-03-29T10:59:13.308707+0000","last_undegraded":"2023-03-29T10:59:13.308707+0000","last_fullsized":"2023-03-29T10:59:13.308707+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:09:44.599265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.522902+0000","last_change":"2023-03-29T10:59:12.332900+0000","last_active":"2023-03-29T10:59:13.522902+0000","last_peered":"2023-03-29T10:59:13.522902+0000","last_clean":"2023-03-29T10:59:13.522902+0000","last_became_active":"2023-03-29T10:59:12.332634+0000","last_became_peered":"2023-03-29T10:59:12.332634+0000","last_unstale":"2023-03-29T10:59:13.522902+0000","last_undegraded":"2023-03-29T10:59:13.522902+0000","last_fullsized":"2023-03-29T10:59:13.522902+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:04:15.453097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637326+0000","last_change":"2023-03-29T10:59:12.333400+0000","last_active":"2023-03-29T10:59:13.637326+0000","last_peered":"2023-03-29T10:59:13.637326+0000","last_clean":"2023-03-29T10:59:13.637326+0000","last_became_active":"2023-03-29T10:59:12.333155+0000","last_became_peered":"2023-03-29T10:59:12.333155+0000","last_unstale":"2023-03-29T10:59:13.637326+0000","last_undegraded":"2023-03-29T10:59:13.637326+0000","last_fullsized":"2023-03-29T10:59:13.637326+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:14:43.981498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718771+0000","last_change":"2023-03-29T10:59:12.323414+0000","last_active":"2023-03-29T10:59:13.718771+0000","last_peered":"2023-03-29T10:59:13.718771+0000","last_clean":"2023-03-29T10:59:13.718771+0000","last_became_active":"2023-03-29T10:59:12.322807+0000","last_became_peered":"2023-03-29T10:59:12.322807+0000","last_unstale":"2023-03-29T10:59:13.718771+0000","last_undegraded":"2023-03-29T10:59:13.718771+0000","last_fullsized":"2023-03-29T10:59:13.718771+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:13:05.082148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718663+0000","last_change":"2023-03-29T10:59:12.322363+0000","last_active":"2023-03-29T10:59:13.718663+0000","last_peered":"2023-03-29T10:59:13.718663+0000","last_clean":"2023-03-29T10:59:13.718663+0000","last_became_active":"2023-03-29T10:59:12.322076+0000","last_became_peered":"2023-03-29T10:59:12.322076+0000","last_unstale":"2023-03-29T10:59:13.718663+0000","last_undegraded":"2023-03-29T10:59:13.718663+0000","last_fullsized":"2023-03-29T10:59:13.718663+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:35:58.000512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719420+0000","last_change":"2023-03-29T10:59:12.324550+0000","last_active":"2023-03-29T10:59:13.719420+0000","last_peered":"2023-03-29T10:59:13.719420+0000","last_clean":"2023-03-29T10:59:13.719420+0000","last_became_active":"2023-03-29T10:59:12.323215+0000","last_became_peered":"2023-03-29T10:59:12.323215+0000","last_unstale":"2023-03-29T10:59:13.719420+0000","last_undegraded":"2023-03-29T10:59:13.719420+0000","last_fullsized":"2023-03-29T10:59:13.719420+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:34:24.360304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.308817+0000","last_change":"2023-03-29T10:59:12.337050+0000","last_active":"2023-03-29T10:59:13.308817+0000","last_peered":"2023-03-29T10:59:13.308817+0000","last_clean":"2023-03-29T10:59:13.308817+0000","last_became_active":"2023-03-29T10:59:12.336800+0000","last_became_peered":"2023-03-29T10:59:12.336800+0000","last_unstale":"2023-03-29T10:59:13.308817+0000","last_undegraded":"2023-03-29T10:59:13.308817+0000","last_fullsized":"2023-03-29T10:59:13.308817+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:13:05.082148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718961+0000","last_change":"2023-03-29T10:59:12.320868+0000","last_active":"2023-03-29T10:59:13.718961+0000","last_peered":"2023-03-29T10:59:13.718961+0000","last_clean":"2023-03-29T10:59:13.718961+0000","last_became_active":"2023-03-29T10:59:12.320555+0000","last_became_peered":"2023-03-29T10:59:12.320555+0000","last_unstale":"2023-03-29T10:59:13.718961+0000","last_undegraded":"2023-03-29T10:59:13.718961+0000","last_fullsized":"2023-03-29T10:59:13.718961+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:04:15.453097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718346+0000","last_change":"2023-03-29T10:59:12.311066+0000","last_active":"2023-03-29T10:59:13.718346+0000","last_peered":"2023-03-29T10:59:13.718346+0000","last_clean":"2023-03-29T10:59:13.718346+0000","last_became_active":"2023-03-29T10:59:12.310698+0000","last_became_peered":"2023-03-29T10:59:12.310698+0000","last_unstale":"2023-03-29T10:59:13.718346+0000","last_undegraded":"2023-03-29T10:59:13.718346+0000","last_fullsized":"2023-03-29T10:59:13.718346+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:06:46.183216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718798+0000","last_change":"2023-03-29T10:59:12.324113+0000","last_active":"2023-03-29T10:59:13.718798+0000","last_peered":"2023-03-29T10:59:13.718798+0000","last_clean":"2023-03-29T10:59:13.718798+0000","last_became_active":"2023-03-29T10:59:12.323855+0000","last_became_peered":"2023-03-29T10:59:12.323855+0000","last_unstale":"2023-03-29T10:59:13.718798+0000","last_undegraded":"2023-03-29T10:59:13.718798+0000","last_fullsized":"2023-03-29T10:59:13.718798+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:07:41.583160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718727+0000","last_change":"2023-03-29T10:59:12.323918+0000","last_active":"2023-03-29T10:59:13.718727+0000","last_peered":"2023-03-29T10:59:13.718727+0000","last_clean":"2023-03-29T10:59:13.718727+0000","last_became_active":"2023-03-29T10:59:12.323637+0000","last_became_peered":"2023-03-29T10:59:12.323637+0000","last_unstale":"2023-03-29T10:59:13.718727+0000","last_undegraded":"2023-03-29T10:59:13.718727+0000","last_fullsized":"2023-03-29T10:59:13.718727+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:18:03.154980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.522752+0000","last_change":"2023-03-29T10:59:12.334069+0000","last_active":"2023-03-29T10:59:13.522752+0000","last_peered":"2023-03-29T10:59:13.522752+0000","last_clean":"2023-03-29T10:59:13.522752+0000","last_became_active":"2023-03-29T10:59:12.333665+0000","last_became_peered":"2023-03-29T10:59:12.333665+0000","last_unstale":"2023-03-29T10:59:13.522752+0000","last_undegraded":"2023-03-29T10:59:13.522752+0000","last_fullsized":"2023-03-29T10:59:13.522752+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:29:07.728609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.308505+0000","last_change":"2023-03-29T10:59:12.320236+0000","last_active":"2023-03-29T10:59:13.308505+0000","last_peered":"2023-03-29T10:59:13.308505+0000","last_clean":"2023-03-29T10:59:13.308505+0000","last_became_active":"2023-03-29T10:59:12.319962+0000","last_became_peered":"2023-03-29T10:59:12.319962+0000","last_unstale":"2023-03-29T10:59:13.308505+0000","last_undegraded":"2023-03-29T10:59:13.308505+0000","last_fullsized":"2023-03-29T10:59:13.308505+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:16:57.395398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313465+0000","last_change":"2023-03-29T10:59:12.333949+0000","last_active":"2023-03-29T10:59:13.313465+0000","last_peered":"2023-03-29T10:59:13.313465+0000","last_clean":"2023-03-29T10:59:13.313465+0000","last_became_active":"2023-03-29T10:59:12.333698+0000","last_became_peered":"2023-03-29T10:59:12.333698+0000","last_unstale":"2023-03-29T10:59:13.313465+0000","last_undegraded":"2023-03-29T10:59:13.313465+0000","last_fullsized":"2023-03-29T10:59:13.313465+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:48:31.147706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"42'4","reported_seq":15,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.523225+0000","last_change":"2023-03-29T10:59:11.838304+0000","last_active":"2023-03-29T10:59:13.523225+0000","last_peered":"2023-03-29T10:59:13.523225+0000","last_clean":"2023-03-29T10:59:13.523225+0000","last_became_active":"2023-03-29T10:59:11.838018+0000","last_became_peered":"2023-03-29T10:59:11.838018+0000","last_unstale":"2023-03-29T10:59:13.523225+0000","last_undegraded":"2023-03-29T10:59:13.523225+0000","last_fullsized":"2023-03-29T10:59:13.523225+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:00:07.668045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719316+0000","last_change":"2023-03-29T10:59:12.330616+0000","last_active":"2023-03-29T10:59:13.719316+0000","last_peered":"2023-03-29T10:59:13.719316+0000","last_clean":"2023-03-29T10:59:13.719316+0000","last_became_active":"2023-03-29T10:59:12.330329+0000","last_became_peered":"2023-03-29T10:59:12.330329+0000","last_unstale":"2023-03-29T10:59:13.719316+0000","last_undegraded":"2023-03-29T10:59:13.719316+0000","last_fullsized":"2023-03-29T10:59:13.719316+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:21:09.442824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719395+0000","last_change":"2023-03-29T10:59:11.840914+0000","last_active":"2023-03-29T10:59:13.719395+0000","last_peered":"2023-03-29T10:59:13.719395+0000","last_clean":"2023-03-29T10:59:13.719395+0000","last_became_active":"2023-03-29T10:59:11.840630+0000","last_became_peered":"2023-03-29T10:59:11.840630+0000","last_unstale":"2023-03-29T10:59:13.719395+0000","last_undegraded":"2023-03-29T10:59:13.719395+0000","last_fullsized":"2023-03-29T10:59:13.719395+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:40:10.721415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718452+0000","last_change":"2023-03-29T10:59:12.340614+0000","last_active":"2023-03-29T10:59:13.718452+0000","last_peered":"2023-03-29T10:59:13.718452+0000","last_clean":"2023-03-29T10:59:13.718452+0000","last_became_active":"2023-03-29T10:59:12.340354+0000","last_became_peered":"2023-03-29T10:59:12.340354+0000","last_unstale":"2023-03-29T10:59:13.718452+0000","last_undegraded":"2023-03-29T10:59:13.718452+0000","last_fullsized":"2023-03-29T10:59:13.718452+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:58:16.409418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313459+0000","last_change":"2023-03-29T10:59:11.838077+0000","last_active":"2023-03-29T10:59:13.313459+0000","last_peered":"2023-03-29T10:59:13.313459+0000","last_clean":"2023-03-29T10:59:13.313459+0000","last_became_active":"2023-03-29T10:59:11.837825+0000","last_became_peered":"2023-03-29T10:59:11.837825+0000","last_unstale":"2023-03-29T10:59:13.313459+0000","last_undegraded":"2023-03-29T10:59:13.313459+0000","last_fullsized":"2023-03-29T10:59:13.313459+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:22:23.341915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313295+0000","last_change":"2023-03-29T10:59:12.320564+0000","last_active":"2023-03-29T10:59:13.313295+0000","last_peered":"2023-03-29T10:59:13.313295+0000","last_clean":"2023-03-29T10:59:13.313295+0000","last_became_active":"2023-03-29T10:59:12.320300+0000","last_became_peered":"2023-03-29T10:59:12.320300+0000","last_unstale":"2023-03-29T10:59:13.313295+0000","last_undegraded":"2023-03-29T10:59:13.313295+0000","last_fullsized":"2023-03-29T10:59:13.313295+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:57:29.081200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313780+0000","last_change":"2023-03-29T10:59:11.826501+0000","last_active":"2023-03-29T10:59:13.313780+0000","last_peered":"2023-03-29T10:59:13.313780+0000","last_clean":"2023-03-29T10:59:13.313780+0000","last_became_active":"2023-03-29T10:59:11.826236+0000","last_became_peered":"2023-03-29T10:59:11.826236+0000","last_unstale":"2023-03-29T10:59:13.313780+0000","last_undegraded":"2023-03-29T10:59:13.313780+0000","last_fullsized":"2023-03-29T10:59:13.313780+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:13:17.086112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718851+0000","last_change":"2023-03-29T10:59:12.321197+0000","last_active":"2023-03-29T10:59:13.718851+0000","last_peered":"2023-03-29T10:59:13.718851+0000","last_clean":"2023-03-29T10:59:13.718851+0000","last_became_active":"2023-03-29T10:59:12.320920+0000","last_became_peered":"2023-03-29T10:59:12.320920+0000","last_unstale":"2023-03-29T10:59:13.718851+0000","last_undegraded":"2023-03-29T10:59:13.718851+0000","last_fullsized":"2023-03-29T10:59:13.718851+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:29:48.533623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.523004+0000","last_change":"2023-03-29T10:59:11.827680+0000","last_active":"2023-03-29T10:59:13.523004+0000","last_peered":"2023-03-29T10:59:13.523004+0000","last_clean":"2023-03-29T10:59:13.523004+0000","last_became_active":"2023-03-29T10:59:11.827408+0000","last_became_peered":"2023-03-29T10:59:11.827408+0000","last_unstale":"2023-03-29T10:59:13.523004+0000","last_undegraded":"2023-03-29T10:59:13.523004+0000","last_fullsized":"2023-03-29T10:59:13.523004+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:25:52.375709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637210+0000","last_change":"2023-03-29T10:59:12.330741+0000","last_active":"2023-03-29T10:59:13.637210+0000","last_peered":"2023-03-29T10:59:13.637210+0000","last_clean":"2023-03-29T10:59:13.637210+0000","last_became_active":"2023-03-29T10:59:12.330283+0000","last_became_peered":"2023-03-29T10:59:12.330283+0000","last_unstale":"2023-03-29T10:59:13.637210+0000","last_undegraded":"2023-03-29T10:59:13.637210+0000","last_fullsized":"2023-03-29T10:59:13.637210+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:56:05.521539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719190+0000","last_change":"2023-03-29T10:59:11.837742+0000","last_active":"2023-03-29T10:59:13.719190+0000","last_peered":"2023-03-29T10:59:13.719190+0000","last_clean":"2023-03-29T10:59:13.719190+0000","last_became_active":"2023-03-29T10:59:11.837390+0000","last_became_peered":"2023-03-29T10:59:11.837390+0000","last_unstale":"2023-03-29T10:59:13.719190+0000","last_undegraded":"2023-03-29T10:59:13.719190+0000","last_fullsized":"2023-03-29T10:59:13.719190+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:19:12.384738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637266+0000","last_change":"2023-03-29T10:59:12.335828+0000","last_active":"2023-03-29T10:59:13.637266+0000","last_peered":"2023-03-29T10:59:13.637266+0000","last_clean":"2023-03-29T10:59:13.637266+0000","last_became_active":"2023-03-29T10:59:12.331831+0000","last_became_peered":"2023-03-29T10:59:12.331831+0000","last_unstale":"2023-03-29T10:59:13.637266+0000","last_undegraded":"2023-03-29T10:59:13.637266+0000","last_fullsized":"2023-03-29T10:59:13.637266+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:29:48.533623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.717964+0000","last_change":"2023-03-29T10:59:11.830858+0000","last_active":"2023-03-29T10:59:13.717964+0000","last_peered":"2023-03-29T10:59:13.717964+0000","last_clean":"2023-03-29T10:59:13.717964+0000","last_became_active":"2023-03-29T10:59:11.830523+0000","last_became_peered":"2023-03-29T10:59:11.830523+0000","last_unstale":"2023-03-29T10:59:13.717964+0000","last_undegraded":"2023-03-29T10:59:13.717964+0000","last_fullsized":"2023-03-29T10:59:13.717964+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:50:48.449260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718878+0000","last_change":"2023-03-29T10:59:12.322950+0000","last_active":"2023-03-29T10:59:13.718878+0000","last_peered":"2023-03-29T10:59:13.718878+0000","last_clean":"2023-03-29T10:59:13.718878+0000","last_became_active":"2023-03-29T10:59:12.322642+0000","last_became_peered":"2023-03-29T10:59:12.322642+0000","last_unstale":"2023-03-29T10:59:13.718878+0000","last_undegraded":"2023-03-29T10:59:13.718878+0000","last_fullsized":"2023-03-29T10:59:13.718878+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:22:24.347143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637124+0000","last_change":"2023-03-29T10:59:11.835227+0000","last_active":"2023-03-29T10:59:13.637124+0000","last_peered":"2023-03-29T10:59:13.637124+0000","last_clean":"2023-03-29T10:59:13.637124+0000","last_became_active":"2023-03-29T10:59:11.834779+0000","last_became_peered":"2023-03-29T10:59:11.834779+0000","last_unstale":"2023-03-29T10:59:13.637124+0000","last_undegraded":"2023-03-29T10:59:13.637124+0000","last_fullsized":"2023-03-29T10:59:13.637124+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:07:59.525950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719216+0000","last_change":"2023-03-29T10:59:12.328010+0000","last_active":"2023-03-29T10:59:13.719216+0000","last_peered":"2023-03-29T10:59:13.719216+0000","last_clean":"2023-03-29T10:59:13.719216+0000","last_became_active":"2023-03-29T10:59:12.327520+0000","last_became_peered":"2023-03-29T10:59:12.327520+0000","last_unstale":"2023-03-29T10:59:13.719216+0000","last_undegraded":"2023-03-29T10:59:13.719216+0000","last_fullsized":"2023-03-29T10:59:13.719216+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:14:43.981498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719073+0000","last_change":"2023-03-29T10:59:11.838354+0000","last_active":"2023-03-29T10:59:13.719073+0000","last_peered":"2023-03-29T10:59:13.719073+0000","last_clean":"2023-03-29T10:59:13.719073+0000","last_became_active":"2023-03-29T10:59:11.838055+0000","last_became_peered":"2023-03-29T10:59:11.838055+0000","last_unstale":"2023-03-29T10:59:13.719073+0000","last_undegraded":"2023-03-29T10:59:13.719073+0000","last_fullsized":"2023-03-29T10:59:13.719073+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:52:54.550636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.314077+0000","last_change":"2023-03-29T10:59:12.317257+0000","last_active":"2023-03-29T10:59:13.314077+0000","last_peered":"2023-03-29T10:59:13.314077+0000","last_clean":"2023-03-29T10:59:13.314077+0000","last_became_active":"2023-03-29T10:59:12.316967+0000","last_became_peered":"2023-03-29T10:59:12.316967+0000","last_unstale":"2023-03-29T10:59:13.314077+0000","last_undegraded":"2023-03-29T10:59:13.314077+0000","last_fullsized":"2023-03-29T10:59:13.314077+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:00:08.673273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637638+0000","last_change":"2023-03-29T10:59:11.836706+0000","last_active":"2023-03-29T10:59:13.637638+0000","last_peered":"2023-03-29T10:59:13.637638+0000","last_clean":"2023-03-29T10:59:13.637638+0000","last_became_active":"2023-03-29T10:59:11.836419+0000","last_became_peered":"2023-03-29T10:59:11.836419+0000","last_unstale":"2023-03-29T10:59:13.637638+0000","last_undegraded":"2023-03-29T10:59:13.637638+0000","last_fullsized":"2023-03-29T10:59:13.637638+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:14:15.356259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.522995+0000","last_change":"2023-03-29T10:59:12.339181+0000","last_active":"2023-03-29T10:59:13.522995+0000","last_peered":"2023-03-29T10:59:13.522995+0000","last_clean":"2023-03-29T10:59:13.522995+0000","last_became_active":"2023-03-29T10:59:12.338919+0000","last_became_peered":"2023-03-29T10:59:12.338919+0000","last_unstale":"2023-03-29T10:59:13.522995+0000","last_undegraded":"2023-03-29T10:59:13.522995+0000","last_fullsized":"2023-03-29T10:59:13.522995+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:56:05.521539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"42'3","reported_seq":14,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.314087+0000","last_change":"2023-03-29T10:59:11.840807+0000","last_active":"2023-03-29T10:59:13.314087+0000","last_peered":"2023-03-29T10:59:13.314087+0000","last_clean":"2023-03-29T10:59:13.314087+0000","last_became_active":"2023-03-29T10:59:11.840428+0000","last_became_peered":"2023-03-29T10:59:11.840428+0000","last_unstale":"2023-03-29T10:59:13.314087+0000","last_undegraded":"2023-03-29T10:59:13.314087+0000","last_fullsized":"2023-03-29T10:59:13.314087+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:28:42.096962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313937+0000","last_change":"2023-03-29T10:59:12.324099+0000","last_active":"2023-03-29T10:59:13.313937+0000","last_peered":"2023-03-29T10:59:13.313937+0000","last_clean":"2023-03-29T10:59:13.313937+0000","last_became_active":"2023-03-29T10:59:12.323837+0000","last_became_peered":"2023-03-29T10:59:12.323837+0000","last_unstale":"2023-03-29T10:59:13.313937+0000","last_undegraded":"2023-03-29T10:59:13.313937+0000","last_fullsized":"2023-03-29T10:59:13.313937+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:18:30.372255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"42'6","reported_seq":17,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.308795+0000","last_change":"2023-03-29T10:59:11.838153+0000","last_active":"2023-03-29T10:59:13.308795+0000","last_peered":"2023-03-29T10:59:13.308795+0000","last_clean":"2023-03-29T10:59:13.308795+0000","last_became_active":"2023-03-29T10:59:11.837886+0000","last_became_peered":"2023-03-29T10:59:11.837886+0000","last_unstale":"2023-03-29T10:59:13.308795+0000","last_undegraded":"2023-03-29T10:59:13.308795+0000","last_fullsized":"2023-03-29T10:59:13.308795+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:52:55.411525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718359+0000","last_change":"2023-03-29T10:59:12.328294+0000","last_active":"2023-03-29T10:59:13.718359+0000","last_peered":"2023-03-29T10:59:13.718359+0000","last_clean":"2023-03-29T10:59:13.718359+0000","last_became_active":"2023-03-29T10:59:12.327976+0000","last_became_peered":"2023-03-29T10:59:12.327976+0000","last_unstale":"2023-03-29T10:59:13.718359+0000","last_undegraded":"2023-03-29T10:59:13.718359+0000","last_fullsized":"2023-03-29T10:59:13.718359+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:28:43.102190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.523197+0000","last_change":"2023-03-29T10:59:11.834754+0000","last_active":"2023-03-29T10:59:13.523197+0000","last_peered":"2023-03-29T10:59:13.523197+0000","last_clean":"2023-03-29T10:59:13.523197+0000","last_became_active":"2023-03-29T10:59:11.834488+0000","last_became_peered":"2023-03-29T10:59:11.834488+0000","last_unstale":"2023-03-29T10:59:13.523197+0000","last_undegraded":"2023-03-29T10:59:13.523197+0000","last_fullsized":"2023-03-29T10:59:13.523197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:07:59.525950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637230+0000","last_change":"2023-03-29T10:59:12.334159+0000","last_active":"2023-03-29T10:59:13.637230+0000","last_peered":"2023-03-29T10:59:13.637230+0000","last_clean":"2023-03-29T10:59:13.637230+0000","last_became_active":"2023-03-29T10:59:12.333911+0000","last_became_peered":"2023-03-29T10:59:12.333911+0000","last_unstale":"2023-03-29T10:59:13.637230+0000","last_undegraded":"2023-03-29T10:59:13.637230+0000","last_fullsized":"2023-03-29T10:59:13.637230+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:05:53.919758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313894+0000","last_change":"2023-03-29T10:59:11.835498+0000","last_active":"2023-03-29T10:59:13.313894+0000","last_peered":"2023-03-29T10:59:13.313894+0000","last_clean":"2023-03-29T10:59:13.313894+0000","last_became_active":"2023-03-29T10:59:11.835175+0000","last_became_peered":"2023-03-29T10:59:11.835175+0000","last_unstale":"2023-03-29T10:59:13.313894+0000","last_undegraded":"2023-03-29T10:59:13.313894+0000","last_fullsized":"2023-03-29T10:59:13.313894+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:07:40.577932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.522747+0000","last_change":"2023-03-29T10:59:12.328454+0000","last_active":"2023-03-29T10:59:13.522747+0000","last_peered":"2023-03-29T10:59:13.522747+0000","last_clean":"2023-03-29T10:59:13.522747+0000","last_became_active":"2023-03-29T10:59:12.328200+0000","last_became_peered":"2023-03-29T10:59:12.328200+0000","last_unstale":"2023-03-29T10:59:13.522747+0000","last_undegraded":"2023-03-29T10:59:13.522747+0000","last_fullsized":"2023-03-29T10:59:13.522747+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:06:46.183216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.522816+0000","last_change":"2023-03-29T10:59:11.835242+0000","last_active":"2023-03-29T10:59:13.522816+0000","last_peered":"2023-03-29T10:59:13.522816+0000","last_clean":"2023-03-29T10:59:13.522816+0000","last_became_active":"2023-03-29T10:59:11.834983+0000","last_became_peered":"2023-03-29T10:59:11.834983+0000","last_unstale":"2023-03-29T10:59:13.522816+0000","last_undegraded":"2023-03-29T10:59:13.522816+0000","last_fullsized":"2023-03-29T10:59:13.522816+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:18:29.367027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313957+0000","last_change":"2023-03-29T10:59:12.321082+0000","last_active":"2023-03-29T10:59:13.313957+0000","last_peered":"2023-03-29T10:59:13.313957+0000","last_clean":"2023-03-29T10:59:13.313957+0000","last_became_active":"2023-03-29T10:59:12.320808+0000","last_became_peered":"2023-03-29T10:59:12.320808+0000","last_unstale":"2023-03-29T10:59:13.313957+0000","last_undegraded":"2023-03-29T10:59:13.313957+0000","last_fullsized":"2023-03-29T10:59:13.313957+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:50:49.454488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"42'8","reported_seq":19,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637228+0000","last_change":"2023-03-29T10:59:11.835056+0000","last_active":"2023-03-29T10:59:13.637228+0000","last_peered":"2023-03-29T10:59:13.637228+0000","last_clean":"2023-03-29T10:59:13.637228+0000","last_became_active":"2023-03-29T10:59:11.834456+0000","last_became_peered":"2023-03-29T10:59:11.834456+0000","last_unstale":"2023-03-29T10:59:13.637228+0000","last_undegraded":"2023-03-29T10:59:13.637228+0000","last_fullsized":"2023-03-29T10:59:13.637228+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:00:07.668045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.308811+0000","last_change":"2023-03-29T10:59:12.316834+0000","last_active":"2023-03-29T10:59:13.308811+0000","last_peered":"2023-03-29T10:59:13.308811+0000","last_clean":"2023-03-29T10:59:13.308811+0000","last_became_active":"2023-03-29T10:59:12.316587+0000","last_became_peered":"2023-03-29T10:59:12.316587+0000","last_unstale":"2023-03-29T10:59:13.308811+0000","last_undegraded":"2023-03-29T10:59:13.308811+0000","last_fullsized":"2023-03-29T10:59:13.308811+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:57:29.081200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.308825+0000","last_change":"2023-03-29T10:59:11.839924+0000","last_active":"2023-03-29T10:59:13.308825+0000","last_peered":"2023-03-29T10:59:13.308825+0000","last_clean":"2023-03-29T10:59:13.308825+0000","last_became_active":"2023-03-29T10:59:11.839675+0000","last_became_peered":"2023-03-29T10:59:11.839675+0000","last_unstale":"2023-03-29T10:59:13.308825+0000","last_undegraded":"2023-03-29T10:59:13.308825+0000","last_fullsized":"2023-03-29T10:59:13.308825+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:45:50.136907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637503+0000","last_change":"2023-03-29T10:59:12.341367+0000","last_active":"2023-03-29T10:59:13.637503+0000","last_peered":"2023-03-29T10:59:13.637503+0000","last_clean":"2023-03-29T10:59:13.637503+0000","last_became_active":"2023-03-29T10:59:12.341098+0000","last_became_peered":"2023-03-29T10:59:12.341098+0000","last_unstale":"2023-03-29T10:59:13.637503+0000","last_undegraded":"2023-03-29T10:59:13.637503+0000","last_fullsized":"2023-03-29T10:59:13.637503+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:04:11.191595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719096+0000","last_change":"2023-03-29T10:59:11.840768+0000","last_active":"2023-03-29T10:59:13.719096+0000","last_peered":"2023-03-29T10:59:13.719096+0000","last_clean":"2023-03-29T10:59:13.719096+0000","last_became_active":"2023-03-29T10:59:11.840493+0000","last_became_peered":"2023-03-29T10:59:11.840493+0000","last_unstale":"2023-03-29T10:59:13.719096+0000","last_undegraded":"2023-03-29T10:59:13.719096+0000","last_fullsized":"2023-03-29T10:59:13.719096+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:40:07.850917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719184+0000","last_change":"2023-03-29T10:59:12.334169+0000","last_active":"2023-03-29T10:59:13.719184+0000","last_peered":"2023-03-29T10:59:13.719184+0000","last_clean":"2023-03-29T10:59:13.719184+0000","last_became_active":"2023-03-29T10:59:12.333918+0000","last_became_peered":"2023-03-29T10:59:12.333918+0000","last_unstale":"2023-03-29T10:59:13.719184+0000","last_undegraded":"2023-03-29T10:59:13.719184+0000","last_fullsized":"2023-03-29T10:59:13.719184+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:48:31.147706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"42'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718188+0000","last_change":"2023-03-29T10:59:11.833433+0000","last_active":"2023-03-29T10:59:13.718188+0000","last_peered":"2023-03-29T10:59:13.718188+0000","last_clean":"2023-03-29T10:59:13.718188+0000","last_became_active":"2023-03-29T10:59:11.833119+0000","last_became_peered":"2023-03-29T10:59:11.833119+0000","last_unstale":"2023-03-29T10:59:13.718188+0000","last_undegraded":"2023-03-29T10:59:13.718188+0000","last_fullsized":"2023-03-29T10:59:13.718188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:40:07.850917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719435+0000","last_change":"2023-03-29T10:59:12.324243+0000","last_active":"2023-03-29T10:59:13.719435+0000","last_peered":"2023-03-29T10:59:13.719435+0000","last_clean":"2023-03-29T10:59:13.719435+0000","last_became_active":"2023-03-29T10:59:12.323921+0000","last_became_peered":"2023-03-29T10:59:12.323921+0000","last_unstale":"2023-03-29T10:59:13.719435+0000","last_undegraded":"2023-03-29T10:59:13.719435+0000","last_fullsized":"2023-03-29T10:59:13.719435+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:13:18.091340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.523104+0000","last_change":"2023-03-29T10:59:11.840764+0000","last_active":"2023-03-29T10:59:13.523104+0000","last_peered":"2023-03-29T10:59:13.523104+0000","last_clean":"2023-03-29T10:59:13.523104+0000","last_became_active":"2023-03-29T10:59:11.840459+0000","last_became_peered":"2023-03-29T10:59:11.840459+0000","last_unstale":"2023-03-29T10:59:13.523104+0000","last_undegraded":"2023-03-29T10:59:13.523104+0000","last_fullsized":"2023-03-29T10:59:13.523104+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:58:54.085911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.522872+0000","last_change":"2023-03-29T10:59:12.339015+0000","last_active":"2023-03-29T10:59:13.522872+0000","last_peered":"2023-03-29T10:59:13.522872+0000","last_clean":"2023-03-29T10:59:13.522872+0000","last_became_active":"2023-03-29T10:59:12.338775+0000","last_became_peered":"2023-03-29T10:59:12.338775+0000","last_unstale":"2023-03-29T10:59:13.522872+0000","last_undegraded":"2023-03-29T10:59:13.522872+0000","last_fullsized":"2023-03-29T10:59:13.522872+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:58:16.409418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.523284+0000","last_change":"2023-03-29T10:59:11.826976+0000","last_active":"2023-03-29T10:59:13.523284+0000","last_peered":"2023-03-29T10:59:13.523284+0000","last_clean":"2023-03-29T10:59:13.523284+0000","last_became_active":"2023-03-29T10:59:11.826702+0000","last_became_peered":"2023-03-29T10:59:11.826702+0000","last_unstale":"2023-03-29T10:59:13.523284+0000","last_undegraded":"2023-03-29T10:59:13.523284+0000","last_fullsized":"2023-03-29T10:59:13.523284+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:14:15.356259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313293+0000","last_change":"2023-03-29T10:59:12.322297+0000","last_active":"2023-03-29T10:59:13.313293+0000","last_peered":"2023-03-29T10:59:13.313293+0000","last_clean":"2023-03-29T10:59:13.313293+0000","last_became_active":"2023-03-29T10:59:12.321962+0000","last_became_peered":"2023-03-29T10:59:12.321962+0000","last_unstale":"2023-03-29T10:59:13.313293+0000","last_undegraded":"2023-03-29T10:59:13.313293+0000","last_fullsized":"2023-03-29T10:59:13.313293+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:28:43.102190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718016+0000","last_change":"2023-03-29T10:59:11.840676+0000","last_active":"2023-03-29T10:59:13.718016+0000","last_peered":"2023-03-29T10:59:13.718016+0000","last_clean":"2023-03-29T10:59:13.718016+0000","last_became_active":"2023-03-29T10:59:11.840214+0000","last_became_peered":"2023-03-29T10:59:11.840214+0000","last_unstale":"2023-03-29T10:59:13.718016+0000","last_undegraded":"2023-03-29T10:59:13.718016+0000","last_fullsized":"2023-03-29T10:59:13.718016+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:46:14.252695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.308496+0000","last_change":"2023-03-29T10:59:12.317597+0000","last_active":"2023-03-29T10:59:13.308496+0000","last_peered":"2023-03-29T10:59:13.308496+0000","last_clean":"2023-03-29T10:59:13.308496+0000","last_became_active":"2023-03-29T10:59:12.317340+0000","last_became_peered":"2023-03-29T10:59:12.317340+0000","last_unstale":"2023-03-29T10:59:13.308496+0000","last_undegraded":"2023-03-29T10:59:13.308496+0000","last_fullsized":"2023-03-29T10:59:13.308496+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T22:21:09.442824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637194+0000","last_change":"2023-03-29T10:59:11.837124+0000","last_active":"2023-03-29T10:59:13.637194+0000","last_peered":"2023-03-29T10:59:13.637194+0000","last_clean":"2023-03-29T10:59:13.637194+0000","last_became_active":"2023-03-29T10:59:11.836884+0000","last_became_peered":"2023-03-29T10:59:11.836884+0000","last_unstale":"2023-03-29T10:59:13.637194+0000","last_undegraded":"2023-03-29T10:59:13.637194+0000","last_fullsized":"2023-03-29T10:59:13.637194+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:58:54.085911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637079+0000","last_change":"2023-03-29T10:59:12.326052+0000","last_active":"2023-03-29T10:59:13.637079+0000","last_peered":"2023-03-29T10:59:13.637079+0000","last_clean":"2023-03-29T10:59:13.637079+0000","last_became_active":"2023-03-29T10:59:12.325717+0000","last_became_peered":"2023-03-29T10:59:12.325717+0000","last_unstale":"2023-03-29T10:59:13.637079+0000","last_undegraded":"2023-03-29T10:59:13.637079+0000","last_fullsized":"2023-03-29T10:59:13.637079+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:20:37.878273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313485+0000","last_change":"2023-03-29T10:59:11.835546+0000","last_active":"2023-03-29T10:59:13.313485+0000","last_peered":"2023-03-29T10:59:13.313485+0000","last_clean":"2023-03-29T10:59:13.313485+0000","last_became_active":"2023-03-29T10:59:11.835013+0000","last_became_peered":"2023-03-29T10:59:11.835013+0000","last_unstale":"2023-03-29T10:59:13.313485+0000","last_undegraded":"2023-03-29T10:59:13.313485+0000","last_fullsized":"2023-03-29T10:59:13.313485+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:34:23.355075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719137+0000","last_change":"2023-03-29T10:59:12.339213+0000","last_active":"2023-03-29T10:59:13.719137+0000","last_peered":"2023-03-29T10:59:13.719137+0000","last_clean":"2023-03-29T10:59:13.719137+0000","last_became_active":"2023-03-29T10:59:12.339024+0000","last_became_peered":"2023-03-29T10:59:12.339024+0000","last_unstale":"2023-03-29T10:59:13.719137+0000","last_undegraded":"2023-03-29T10:59:13.719137+0000","last_fullsized":"2023-03-29T10:59:13.719137+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:56:05.521539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637356+0000","last_change":"2023-03-29T10:59:11.840280+0000","last_active":"2023-03-29T10:59:13.637356+0000","last_peered":"2023-03-29T10:59:13.637356+0000","last_clean":"2023-03-29T10:59:13.637356+0000","last_became_active":"2023-03-29T10:59:11.839982+0000","last_became_peered":"2023-03-29T10:59:11.839982+0000","last_unstale":"2023-03-29T10:59:13.637356+0000","last_undegraded":"2023-03-29T10:59:13.637356+0000","last_fullsized":"2023-03-29T10:59:13.637356+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:25:52.375709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-29T10:59:12.310697+0000","last_change":"2023-03-29T10:57:22.318583+0000","last_active":"2023-03-29T10:59:12.310697+0000","last_peered":"2023-03-29T10:59:12.310697+0000","last_clean":"2023-03-29T10:59:12.310697+0000","last_became_active":"2023-03-29T10:57:22.318223+0000","last_became_peered":"2023-03-29T10:57:22.318223+0000","last_unstale":"2023-03-29T10:59:12.310697+0000","last_undegraded":"2023-03-29T10:59:12.310697+0000","last_fullsized":"2023-03-29T10:59:12.310697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:31:03.349241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719354+0000","last_change":"2023-03-29T10:59:12.330220+0000","last_active":"2023-03-29T10:59:13.719354+0000","last_peered":"2023-03-29T10:59:13.719354+0000","last_clean":"2023-03-29T10:59:13.719354+0000","last_became_active":"2023-03-29T10:59:12.329936+0000","last_became_peered":"2023-03-29T10:59:12.329936+0000","last_unstale":"2023-03-29T10:59:13.719354+0000","last_undegraded":"2023-03-29T10:59:13.719354+0000","last_fullsized":"2023-03-29T10:59:13.719354+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:57:29.081200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"43'1","reported_seq":12,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:14.027623+0000","last_change":"2023-03-29T10:59:11.841525+0000","last_active":"2023-03-29T10:59:14.027623+0000","last_peered":"2023-03-29T10:59:14.027623+0000","last_clean":"2023-03-29T10:59:14.027623+0000","last_became_active":"2023-03-29T10:59:11.841261+0000","last_became_peered":"2023-03-29T10:59:11.841261+0000","last_unstale":"2023-03-29T10:59:14.027623+0000","last_undegraded":"2023-03-29T10:59:14.027623+0000","last_fullsized":"2023-03-29T10:59:14.027623+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:42:56.020246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":30,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-29T10:59:12.305983+0000","last_change":"2023-03-29T10:57:22.320330+0000","last_active":"2023-03-29T10:59:12.305983+0000","last_peered":"2023-03-29T10:59:12.305983+0000","last_clean":"2023-03-29T10:59:12.305983+0000","last_became_active":"2023-03-29T10:57:22.320011+0000","last_became_peered":"2023-03-29T10:57:22.320011+0000","last_unstale":"2023-03-29T10:59:12.305983+0000","last_undegraded":"2023-03-29T10:59:12.305983+0000","last_fullsized":"2023-03-29T10:59:12.305983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:47:01.099069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637384+0000","last_change":"2023-03-29T10:59:12.325157+0000","last_active":"2023-03-29T10:59:13.637384+0000","last_peered":"2023-03-29T10:59:13.637384+0000","last_clean":"2023-03-29T10:59:13.637384+0000","last_became_active":"2023-03-29T10:59:12.324927+0000","last_became_peered":"2023-03-29T10:59:12.324927+0000","last_unstale":"2023-03-29T10:59:13.637384+0000","last_undegraded":"2023-03-29T10:59:13.637384+0000","last_fullsized":"2023-03-29T10:59:13.637384+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:22:01.039112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.314064+0000","last_change":"2023-03-29T10:59:11.837671+0000","last_active":"2023-03-29T10:59:13.314064+0000","last_peered":"2023-03-29T10:59:13.314064+0000","last_clean":"2023-03-29T10:59:13.314064+0000","last_became_active":"2023-03-29T10:59:11.837354+0000","last_became_peered":"2023-03-29T10:59:11.837354+0000","last_unstale":"2023-03-29T10:59:13.314064+0000","last_undegraded":"2023-03-29T10:59:13.314064+0000","last_fullsized":"2023-03-29T10:59:13.314064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T18:48:30.142478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-29T10:59:12.305312+0000","last_change":"2023-03-29T10:57:22.467148+0000","last_active":"2023-03-29T10:59:12.305312+0000","last_peered":"2023-03-29T10:59:12.305312+0000","last_clean":"2023-03-29T10:59:12.305312+0000","last_became_active":"2023-03-29T10:57:22.466822+0000","last_became_peered":"2023-03-29T10:57:22.466822+0000","last_unstale":"2023-03-29T10:59:12.305312+0000","last_undegraded":"2023-03-29T10:59:12.305312+0000","last_fullsized":"2023-03-29T10:59:12.305312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T16:53:23.007205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.637591+0000","last_change":"2023-03-29T10:59:12.334749+0000","last_active":"2023-03-29T10:59:13.637591+0000","last_peered":"2023-03-29T10:59:13.637591+0000","last_clean":"2023-03-29T10:59:13.637591+0000","last_became_active":"2023-03-29T10:59:12.334494+0000","last_became_peered":"2023-03-29T10:59:12.334494+0000","last_unstale":"2023-03-29T10:59:13.637591+0000","last_undegraded":"2023-03-29T10:59:13.637591+0000","last_fullsized":"2023-03-29T10:59:13.637591+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:06:46.183216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"42'2","reported_seq":13,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.718494+0000","last_change":"2023-03-29T10:59:11.825421+0000","last_active":"2023-03-29T10:59:13.718494+0000","last_peered":"2023-03-29T10:59:13.718494+0000","last_clean":"2023-03-29T10:59:13.718494+0000","last_became_active":"2023-03-29T10:59:11.825124+0000","last_became_peered":"2023-03-29T10:59:11.825124+0000","last_unstale":"2023-03-29T10:59:13.718494+0000","last_undegraded":"2023-03-29T10:59:13.718494+0000","last_fullsized":"2023-03-29T10:59:13.718494+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:39:48.277218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":30,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-29T10:59:12.305875+0000","last_change":"2023-03-29T10:57:22.321523+0000","last_active":"2023-03-29T10:59:12.305875+0000","last_peered":"2023-03-29T10:59:12.305875+0000","last_clean":"2023-03-29T10:59:12.305875+0000","last_became_active":"2023-03-29T10:57:22.321183+0000","last_became_peered":"2023-03-29T10:57:22.321183+0000","last_unstale":"2023-03-29T10:59:12.305875+0000","last_undegraded":"2023-03-29T10:59:12.305875+0000","last_fullsized":"2023-03-29T10:59:12.305875+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:35:11.664657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":186,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-29T10:59:12.305133+0000","last_change":"2023-03-29T10:57:18.176665+0000","last_active":"2023-03-29T10:59:12.305133+0000","last_peered":"2023-03-29T10:59:12.305133+0000","last_clean":"2023-03-29T10:59:12.305133+0000","last_became_active":"2023-03-29T10:57:18.175739+0000","last_became_peered":"2023-03-29T10:57:18.175739+0000","last_unstale":"2023-03-29T10:59:12.305133+0000","last_undegraded":"2023-03-29T10:59:12.305133+0000","last_fullsized":"2023-03-29T10:59:12.305133+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:16.275989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:16.275989+0000","last_clean_scrub_stamp":"2023-03-29T10:57:16.275989+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:06:32.872344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313983+0000","last_change":"2023-03-29T10:59:12.314644+0000","last_active":"2023-03-29T10:59:13.313983+0000","last_peered":"2023-03-29T10:59:13.313983+0000","last_clean":"2023-03-29T10:59:13.313983+0000","last_became_active":"2023-03-29T10:59:12.314361+0000","last_became_peered":"2023-03-29T10:59:12.314361+0000","last_unstale":"2023-03-29T10:59:13.313983+0000","last_undegraded":"2023-03-29T10:59:13.313983+0000","last_fullsized":"2023-03-29T10:59:13.313983+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T11:08:00.531178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313157+0000","last_change":"2023-03-29T10:59:11.833778+0000","last_active":"2023-03-29T10:59:13.313157+0000","last_peered":"2023-03-29T10:59:13.313157+0000","last_clean":"2023-03-29T10:59:13.313157+0000","last_became_active":"2023-03-29T10:59:11.833478+0000","last_became_peered":"2023-03-29T10:59:11.833478+0000","last_unstale":"2023-03-29T10:59:13.313157+0000","last_undegraded":"2023-03-29T10:59:13.313157+0000","last_fullsized":"2023-03-29T10:59:13.313157+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:52:55.411525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":33,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-29T10:59:12.305230+0000","last_change":"2023-03-29T10:57:22.322607+0000","last_active":"2023-03-29T10:59:12.305230+0000","last_peered":"2023-03-29T10:59:12.305230+0000","last_clean":"2023-03-29T10:59:12.305230+0000","last_became_active":"2023-03-29T10:57:22.321613+0000","last_became_peered":"2023-03-29T10:57:22.321613+0000","last_unstale":"2023-03-29T10:59:12.305230+0000","last_undegraded":"2023-03-29T10:59:12.305230+0000","last_fullsized":"2023-03-29T10:59:12.305230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T20:35:11.664657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.717916+0000","last_change":"2023-03-29T10:59:12.318928+0000","last_active":"2023-03-29T10:59:13.717916+0000","last_peered":"2023-03-29T10:59:13.717916+0000","last_clean":"2023-03-29T10:59:13.717916+0000","last_became_active":"2023-03-29T10:59:12.318485+0000","last_became_peered":"2023-03-29T10:59:12.318485+0000","last_unstale":"2023-03-29T10:59:13.717916+0000","last_undegraded":"2023-03-29T10:59:13.717916+0000","last_fullsized":"2023-03-29T10:59:13.717916+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:20:37.878273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313161+0000","last_change":"2023-03-29T10:59:11.840495+0000","last_active":"2023-03-29T10:59:13.313161+0000","last_peered":"2023-03-29T10:59:13.313161+0000","last_clean":"2023-03-29T10:59:13.313161+0000","last_became_active":"2023-03-29T10:59:11.840245+0000","last_became_peered":"2023-03-29T10:59:11.840245+0000","last_unstale":"2023-03-29T10:59:13.313161+0000","last_undegraded":"2023-03-29T10:59:13.313161+0000","last_fullsized":"2023-03-29T10:59:13.313161+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T19:52:57.622991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-29T10:59:12.306604+0000","last_change":"2023-03-29T10:57:22.315905+0000","last_active":"2023-03-29T10:59:12.306604+0000","last_peered":"2023-03-29T10:59:12.306604+0000","last_clean":"2023-03-29T10:59:12.306604+0000","last_became_active":"2023-03-29T10:57:22.315603+0000","last_became_peered":"2023-03-29T10:57:22.315603+0000","last_unstale":"2023-03-29T10:59:12.306604+0000","last_undegraded":"2023-03-29T10:59:12.306604+0000","last_fullsized":"2023-03-29T10:59:12.306604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T10:59:44.030013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.523131+0000","last_change":"2023-03-29T10:59:12.332368+0000","last_active":"2023-03-29T10:59:13.523131+0000","last_peered":"2023-03-29T10:59:13.523131+0000","last_clean":"2023-03-29T10:59:13.523131+0000","last_became_active":"2023-03-29T10:59:12.332085+0000","last_became_peered":"2023-03-29T10:59:12.332085+0000","last_unstale":"2023-03-29T10:59:13.523131+0000","last_undegraded":"2023-03-29T10:59:13.523131+0000","last_fullsized":"2023-03-29T10:59:13.523131+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T17:26:51.059416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.719288+0000","last_change":"2023-03-29T10:59:11.840268+0000","last_active":"2023-03-29T10:59:13.719288+0000","last_peered":"2023-03-29T10:59:13.719288+0000","last_clean":"2023-03-29T10:59:13.719288+0000","last_became_active":"2023-03-29T10:59:11.839947+0000","last_became_peered":"2023-03-29T10:59:11.839947+0000","last_unstale":"2023-03-29T10:59:13.719288+0000","last_undegraded":"2023-03-29T10:59:13.719288+0000","last_fullsized":"2023-03-29T10:59:13.719288+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T21:46:14.252695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":30,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-29T10:59:12.309709+0000","last_change":"2023-03-29T10:57:22.319554+0000","last_active":"2023-03-29T10:59:12.309709+0000","last_peered":"2023-03-29T10:59:12.309709+0000","last_clean":"2023-03-29T10:59:12.309709+0000","last_became_active":"2023-03-29T10:57:22.319198+0000","last_became_peered":"2023-03-29T10:57:22.319198+0000","last_unstale":"2023-03-29T10:59:12.309709+0000","last_undegraded":"2023-03-29T10:59:12.309709+0000","last_fullsized":"2023-03-29T10:59:12.309709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T15:47:01.099069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.523002+0000","last_change":"2023-03-29T10:59:12.332711+0000","last_active":"2023-03-29T10:59:13.523002+0000","last_peered":"2023-03-29T10:59:13.523002+0000","last_clean":"2023-03-29T10:59:13.523002+0000","last_became_active":"2023-03-29T10:59:12.332410+0000","last_became_peered":"2023-03-29T10:59:12.332410+0000","last_unstale":"2023-03-29T10:59:13.523002+0000","last_undegraded":"2023-03-29T10:59:13.523002+0000","last_fullsized":"2023-03-29T10:59:13.523002+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:11.809363+0000","last_clean_scrub_stamp":"2023-03-29T10:59:11.809363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T14:20:37.878273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-03-29T10:59:13.313910+0000","last_change":"2023-03-29T10:59:11.840781+0000","last_active":"2023-03-29T10:59:13.313910+0000","last_peered":"2023-03-29T10:59:13.313910+0000","last_clean":"2023-03-29T10:59:13.313910+0000","last_became_active":"2023-03-29T10:59:11.840480+0000","last_became_peered":"2023-03-29T10:59:11.840480+0000","last_unstale":"2023-03-29T10:59:13.313910+0000","last_undegraded":"2023-03-29T10:59:13.313910+0000","last_fullsized":"2023-03-29T10:59:13.313910+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:59:10.804134+0000","last_clean_scrub_stamp":"2023-03-29T10:59:10.804134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T13:35:56.995284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":29,"reported_epoch":42,"state":"active+clean","last_fresh":"2023-03-29T10:59:12.314367+0000","last_change":"2023-03-29T10:57:22.322319+0000","last_active":"2023-03-29T10:59:12.314367+0000","last_peered":"2023-03-29T10:59:12.314367+0000","last_clean":"2023-03-29T10:59:12.314367+0000","last_became_active":"2023-03-29T10:57:22.321943+0000","last_became_peered":"2023-03-29T10:57:22.321943+0000","last_unstale":"2023-03-29T10:59:12.314367+0000","last_undegraded":"2023-03-29T10:59:12.314367+0000","last_fullsized":"2023-03-29T10:59:12.314367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-03-29T10:57:21.302443+0000","last_clean_scrub_stamp":"2023-03-29T10:57:21.302443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-03-30T12:58:04.969453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2366,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9848,"kb_used_data":1072,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94361992,"statfs":{"total":96636764160,"available":96626679808,"internally_reserved":0,"allocated":1097728,"data_stored":813905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 29 10:58:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.2909999999999999,"5min":1.2909999999999999,"15min":1.2909999999999999},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed Mar 29 10:58:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.496},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.69099999999999995}]},{"osd":2,"last update":"Wed Mar 29 10:58:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.53600000000000003}]},{"osd":3,"last update":"Wed Mar 29 10:58:14 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.60199999999999998}]},{"osd":4,"last update":"Wed Mar 29 10:58:14 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.52000000000000002}]},{"osd":5,"last update":"Wed Mar 29 10:58:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.626},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.73199999999999998}]},{"osd":6,"last update":"Wed Mar 29 10:58:14 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.65900000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8632,"kb_used_data":496,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363208,"statfs":{"total":96636764160,"available":96627924992,"internally_reserved":0,"allocated":507904,"data_stored":224119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.79900000000000004}]},{"osd":1,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.70899999999999996}]},{"osd":2,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.755}]},{"osd":3,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.73399999999999999}]},{"osd":4,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.82799999999999996}]},{"osd":5,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.77900000000000003}]},{"osd":7,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.81299999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8756,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363084,"statfs":{"total":96636764160,"available":96627798016,"internally_reserved":0,"allocated":503808,"data_stored":223517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 29 10:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.38},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.35699999999999998}]},{"osd":2,"last update":"Wed Mar 29 10:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.42499999999999999}]},{"osd":3,"last update":"Wed Mar 29 10:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.41599999999999998}]},{"osd":4,"last update":"Wed Mar 29 10:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.623}]},{"osd":5,"last update":"Wed Mar 29 10:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.48899999999999999}]},{"osd":6,"last update":"Wed Mar 29 10:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.57599999999999996}]},{"osd":7,"last update":"Wed Mar 29 10:58:15 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.056,"5min":2.056,"15min":2.056},"last":0.59099999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9844,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94361996,"statfs":{"total":96636764160,"available":96626683904,"internally_reserved":0,"allocated":1093632,"data_stored":813234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.40100000000000002}]},{"osd":2,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.55400000000000005}]},{"osd":3,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.42299999999999999}]},{"osd":4,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.57699999999999996}]},{"osd":5,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":4.4610000000000003,"5min":4.4610000000000003,"15min":4.4610000000000003},"last":0.748},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.484,"5min":1.484,"15min":1.484},"last":0.60699999999999998}]},{"osd":6,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":4.4820000000000002,"5min":4.4820000000000002,"15min":4.4820000000000002},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":4.4989999999999997,"5min":4.4989999999999997,"15min":4.4989999999999997},"last":0.71899999999999997}]},{"osd":7,"last update":"Wed Mar 29 10:58:17 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.63600000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8636,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363204,"statfs":{"total":96636764160,"available":96627920896,"internally_reserved":0,"allocated":512000,"data_stored":224414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.5609999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":0.47899999999999998}]},{"osd":1,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":2.3570000000000002,"5min":2.3570000000000002,"15min":2.3570000000000002},"last":0.40799999999999997}]},{"osd":3,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":2.6200000000000001,"5min":2.6200000000000001,"15min":2.6200000000000001},"last":0.439},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.54900000000000004}]},{"osd":4,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":0.46999999999999997}]},{"osd":5,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.2999999999999998,"5min":2.2999999999999998,"15min":2.2999999999999998},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.4940000000000002,"5min":2.4940000000000002,"15min":2.4940000000000002},"last":0.65300000000000002}]},{"osd":6,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.5899999999999999,"5min":2.5899999999999999,"15min":2.5899999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.63500000000000001}]},{"osd":7,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.528,"5min":2.528,"15min":2.528},"last":0.61899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8764,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363076,"statfs":{"total":96636764160,"available":96627789824,"internally_reserved":0,"allocated":512000,"data_stored":224089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.29099999999999998}]},{"osd":1,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.312}]},{"osd":2,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.42699999999999999}]},{"osd":4,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.46999999999999997}]},{"osd":5,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.63200000000000001}]},{"osd":6,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.56899999999999995}]},{"osd":7,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.58899999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8764,"kb_used_data":500,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363076,"statfs":{"total":96636764160,"available":96627789824,"internally_reserved":0,"allocated":512000,"data_stored":225011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.68700000000000006}]},{"osd":1,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.73799999999999999}]},{"osd":2,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.72299999999999998}]},{"osd":3,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":2.528,"5min":2.528,"15min":2.528},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.76200000000000001}]},{"osd":5,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.47599999999999998}]},{"osd":6,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.53700000000000003}]},{"osd":7,"last update":"Wed Mar 29 10:58:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.48799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8692,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363148,"statfs":{"total":96636764160,"available":96627863552,"internally_reserved":0,"allocated":503808,"data_stored":223517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Mar 29 10:58:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.753}]},{"osd":1,"last update":"Wed Mar 29 10:58:16 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.72099999999999997}]},{"osd":2,"last update":"Wed Mar 29 10:58:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.628},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.66700000000000004}]},{"osd":3,"last update":"Wed Mar 29 10:58:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.3719999999999999,"5min":2.3719999999999999,"15min":2.3719999999999999},"last":0.69899999999999995}]},{"osd":4,"last update":"Wed Mar 29 10:58:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.5,"5min":2.5,"15min":2.5},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.82999999999999996}]},{"osd":6,"last update":"Wed Mar 29 10:58:16 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":2.4750000000000001,"5min":2.4750000000000001,"15min":2.4750000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.5880000000000001,"5min":2.5880000000000001,"15min":2.5880000000000001},"last":0.85699999999999998}]},{"osd":7,"last update":"Wed Mar 29 10:58:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.80400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-03-29T10:59:17.093 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-03-29T10:59:17.093 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-03-29T10:59:17.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.306+0000 7f06fddff700 1 Processor -- start 2023-03-29T10:59:17.307 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.306+0000 7f06fddff700 1 -- start start 2023-03-29T10:59:17.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.307+0000 7f06fddff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f06f80cd790 0x7f06f80ca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:17.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.307+0000 7f06fddff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f06f80b1d50 0x7f06f806b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:17.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.308+0000 7f06fddff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f806a5f0 con 0x7f06f80cdb60 2023-03-29T10:59:17.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.308+0000 7f06fddff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06f806a840 con 0x7f06f80cd790 2023-03-29T10:59:17.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.308+0000 7f06fddff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06f806ab20 con 0x7f06f80b1d50 2023-03-29T10:59:17.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.308+0000 7f06f77fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f06f80cd790 0x7f06f80ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:17.308 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.308+0000 7f06f77fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f06f80cd790 0x7f06f80ca210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40406/0 (socket says 172.21.15.111:40406) 2023-03-29T10:59:17.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.308+0000 7f06f77fe700 1 -- 172.21.15.111:0/2852114294 learned_addr learned my addr 172.21.15.111:0/2852114294 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:17.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.308+0000 7f06f7fff700 1 --2- 172.21.15.111:0/2852114294 >> v2:172.21.15.119:3300/0 conn(0x7f06f80b1d50 0x7f06f806b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:17.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.308+0000 7f06f77fe700 1 -- 172.21.15.111:0/2852114294 >> v2:172.21.15.119:3300/0 conn(0x7f06f80b1d50 msgr2=0x7f06f806b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:17.309 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.308+0000 7f06f77fe700 1 --2- 172.21.15.111:0/2852114294 >> v2:172.21.15.119:3300/0 conn(0x7f06f80b1d50 0x7f06f806b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:17.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.308+0000 7f06f77fe700 1 -- 172.21.15.111:0/2852114294 >> v1:172.21.15.119:6789/0 conn(0x7f06f80cdb60 legacy=0x7f06f80b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:17.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.308+0000 7f06f77fe700 1 -- 172.21.15.111:0/2852114294 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f806ae00 con 0x7f06f80cd790 2023-03-29T10:59:17.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.309+0000 7f06f77fe700 1 --2- 172.21.15.111:0/2852114294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06f80cd790 0x7f06f80ca210 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f06e80098f0 tx=0x7f06e8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c130fadde89f757 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:17.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.309+0000 7f06f6ffd700 1 -- 172.21.15.111:0/2852114294 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8004230 con 0x7f06f80cd790 2023-03-29T10:59:17.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.309+0000 7f06f6ffd700 1 -- 172.21.15.111:0/2852114294 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e8004390 con 0x7f06f80cd790 2023-03-29T10:59:17.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.309+0000 7f06f6ffd700 1 -- 172.21.15.111:0/2852114294 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e80036a0 con 0x7f06f80cd790 2023-03-29T10:59:17.310 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.309+0000 7f06fddff700 1 -- 172.21.15.111:0/2852114294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06f80cd790 msgr2=0x7f06f80ca210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:17.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.309+0000 7f06fddff700 1 --2- 172.21.15.111:0/2852114294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06f80cd790 0x7f06f80ca210 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f06e80098f0 tx=0x7f06e8009c00 comp rx=0 tx=0).stop 2023-03-29T10:59:17.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.309+0000 7f06fddff700 1 -- 172.21.15.111:0/2852114294 shutdown_connections 2023-03-29T10:59:17.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.309+0000 7f06fddff700 1 --2- 172.21.15.111:0/2852114294 >> v2:172.21.15.119:3300/0 conn(0x7f06f80b1d50 0x7f06f806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:17.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.309+0000 7f06fddff700 1 --2- 172.21.15.111:0/2852114294 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06f80cd790 0x7f06f80ca210 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:17.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.309+0000 7f06fddff700 1 -- 172.21.15.111:0/2852114294 >> 172.21.15.111:0/2852114294 conn(0x7f06f806e980 msgr2=0x7f06f80cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:17.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.309+0000 7f06fddff700 1 -- 172.21.15.111:0/2852114294 shutdown_connections 2023-03-29T10:59:17.311 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.310+0000 7f06fddff700 1 -- 172.21.15.111:0/2852114294 wait complete. 2023-03-29T10:59:17.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.311+0000 7f06fddff700 1 Processor -- start 2023-03-29T10:59:17.312 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.311+0000 7f06fddff700 1 -- start start 2023-03-29T10:59:17.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.312+0000 7f06fddff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06f80b1d50 0x7f06f80ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:17.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.312+0000 7f06fddff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f80cd790 0x7f06f815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:17.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.312+0000 7f06fddff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06f80cdb60 0x7f06f80b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:17.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.312+0000 7f06fddff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f80c86b0 con 0x7f06f80b1d50 2023-03-29T10:59:17.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.312+0000 7f06fddff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f80c7d80 con 0x7f06f80cd790 2023-03-29T10:59:17.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.312+0000 7f06fddff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f80c8060 con 0x7f06f80cdb60 2023-03-29T10:59:17.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.312+0000 7f06f77fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06f80b1d50 0x7f06f80ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:17.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.312+0000 7f06f77fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06f80b1d50 0x7f06f80ade00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40418/0 (socket says 172.21.15.111:40418) 2023-03-29T10:59:17.313 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.312+0000 7f06f77fe700 1 -- 172.21.15.111:0/1167627840 learned_addr learned my addr 172.21.15.111:0/1167627840 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:17.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.313+0000 7f06effff700 1 --2- 172.21.15.111:0/1167627840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f80cd790 0x7f06f815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:17.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.313+0000 7f06f7fff700 1 --2- 172.21.15.111:0/1167627840 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06f80cdb60 0x7f06f80b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:17.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.313+0000 7f06f77fe700 1 -- 172.21.15.111:0/1167627840 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06f80cdb60 msgr2=0x7f06f80b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:17.315 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.313+0000 7f06f77fe700 1 --2- 172.21.15.111:0/1167627840 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06f80cdb60 0x7f06f80b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:17.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.313+0000 7f06f77fe700 1 -- 172.21.15.111:0/1167627840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f80cd790 msgr2=0x7f06f815dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:17.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.313+0000 7f06f77fe700 1 --2- 172.21.15.111:0/1167627840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f80cd790 0x7f06f815dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:17.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.313+0000 7f06f77fe700 1 -- 172.21.15.111:0/1167627840 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e8009550 con 0x7f06f80b1d50 2023-03-29T10:59:17.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.313+0000 7f06f7fff700 1 --2- 172.21.15.111:0/1167627840 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06f80cdb60 0x7f06f80b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:59:17.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.313+0000 7f06f77fe700 1 --2- 172.21.15.111:0/1167627840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06f80b1d50 0x7f06f80ade00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f06f806f200 tx=0x7f06e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:17.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.313+0000 7f06f4ff9700 1 -- 172.21.15.111:0/1167627840 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8011630 con 0x7f06f80b1d50 2023-03-29T10:59:17.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.313+0000 7f06f4ff9700 1 -- 172.21.15.111:0/1167627840 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e8011790 con 0x7f06f80b1d50 2023-03-29T10:59:17.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.313+0000 7f06fddff700 1 -- 172.21.15.111:0/1167627840 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f80ce000 con 0x7f06f80b1d50 2023-03-29T10:59:17.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.313+0000 7f06f4ff9700 1 -- 172.21.15.111:0/1167627840 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e8011b40 con 0x7f06f80b1d50 2023-03-29T10:59:17.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.314+0000 7f06fddff700 1 -- 172.21.15.111:0/1167627840 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f806b0e0 con 0x7f06f80b1d50 2023-03-29T10:59:17.316 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.314+0000 7f06fddff700 1 -- 172.21.15.111:0/1167627840 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f815ca20 con 0x7f06f80b1d50 2023-03-29T10:59:17.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.315+0000 7f06f4ff9700 1 -- 172.21.15.111:0/1167627840 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f06e8011ca0 con 0x7f06f80b1d50 2023-03-29T10:59:17.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.316+0000 7f06f4ff9700 1 --2- 172.21.15.111:0/1167627840 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f06d8069de0 0x7f06d806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:17.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.316+0000 7f06f4ff9700 1 -- 172.21.15.111:0/1167627840 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f06e80899e0 con 0x7f06f80b1d50 2023-03-29T10:59:17.317 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.317+0000 7f06effff700 1 --2- 172.21.15.111:0/1167627840 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f06d8069de0 0x7f06d806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:17.318 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.317+0000 7f06effff700 1 --2- 172.21.15.111:0/1167627840 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f06d8069de0 0x7f06d806c290 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f06f80b0c60 tx=0x7f06e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:17.321 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.320+0000 7f06f4ff9700 1 -- 172.21.15.111:0/1167627840 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f06e8055140 con 0x7f06f80b1d50 2023-03-29T10:59:17.590 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.590+0000 7f06fddff700 1 -- 172.21.15.111:0/1167627840 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f06f806c510 con 0x7f06f80b1d50 2023-03-29T10:59:17.595 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.594+0000 7f06f4ff9700 1 -- 172.21.15.111:0/1167627840 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f06e8058760 con 0x7f06f80b1d50 2023-03-29T10:59:17.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.597+0000 7f06fddff700 1 -- 172.21.15.111:0/1167627840 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f06d8069de0 msgr2=0x7f06d806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:17.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.597+0000 7f06fddff700 1 --2- 172.21.15.111:0/1167627840 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f06d8069de0 0x7f06d806c290 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f06f80b0c60 tx=0x7f06e0008040 comp rx=0 tx=0).stop 2023-03-29T10:59:17.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.597+0000 7f06fddff700 1 -- 172.21.15.111:0/1167627840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06f80b1d50 msgr2=0x7f06f80ade00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:17.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.597+0000 7f06fddff700 1 --2- 172.21.15.111:0/1167627840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06f80b1d50 0x7f06f80ade00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f06f806f200 tx=0x7f06e8004750 comp rx=0 tx=0).stop 2023-03-29T10:59:17.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.597+0000 7f06fddff700 1 -- 172.21.15.111:0/1167627840 shutdown_connections 2023-03-29T10:59:17.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.597+0000 7f06fddff700 1 --2- 172.21.15.111:0/1167627840 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f06d8069de0 0x7f06d806c290 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:17.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.597+0000 7f06fddff700 1 --2- 172.21.15.111:0/1167627840 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f06f80b1d50 0x7f06f80ade00 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:17.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.597+0000 7f06fddff700 1 --2- 172.21.15.111:0/1167627840 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f06f80cdb60 0x7f06f80b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:17.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.597+0000 7f06fddff700 1 --2- 172.21.15.111:0/1167627840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06f80cd790 0x7f06f815dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:17.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.597+0000 7f06fddff700 1 -- 172.21.15.111:0/1167627840 >> 172.21.15.111:0/1167627840 conn(0x7f06f806e980 msgr2=0x7f06f8160a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:17.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.597+0000 7f06fddff700 1 -- 172.21.15.111:0/1167627840 shutdown_connections 2023-03-29T10:59:17.598 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.597+0000 7f06fddff700 1 -- 172.21.15.111:0/1167627840 wait complete. 2023-03-29T10:59:17.599 INFO:teuthology.orchestra.run.smithi111.stderr:updated caps for client.0 2023-03-29T10:59:17.615 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-03-29T10:59:17.798 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.797+0000 7f0079bff700 1 Processor -- start 2023-03-29T10:59:17.798 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.797+0000 7f0079bff700 1 -- start start 2023-03-29T10:59:17.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.798+0000 7f0079bff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f00740b0dd0 0x7f00740c9290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:17.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.798+0000 7f0079bff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00740b16b0 con 0x7f0074143f10 2023-03-29T10:59:17.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.799+0000 7f0079bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00740c9c10 con 0x7f00740c97d0 2023-03-29T10:59:17.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.799+0000 7f0079bff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f007415ac50 con 0x7f00740b0dd0 2023-03-29T10:59:17.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.799+0000 7f00737fe700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f0074143f10 0x7f00740b06c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:38246/0 (socket says 172.21.15.111:38246) 2023-03-29T10:59:17.800 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.799+0000 7f00737fe700 1 -- 172.21.15.111:0/3463046990 learned_addr learned my addr 172.21.15.111:0/3463046990 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:59:17.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.799+0000 7f0072ffd700 1 --2- 172.21.15.111:0/3463046990 >> v2:172.21.15.111:3300/0 conn(0x7f00740b0dd0 0x7f00740c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:17.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.799+0000 7f0072ffd700 1 -- 172.21.15.111:0/3463046990 >> v1:172.21.15.119:6789/0 conn(0x7f00740c97d0 legacy=0x7f00740c2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:17.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.799+0000 7f0072ffd700 1 -- 172.21.15.111:0/3463046990 >> v1:172.21.15.111:6789/0 conn(0x7f0074143f10 legacy=0x7f00740b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:17.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.799+0000 7f0072ffd700 1 -- 172.21.15.111:0/3463046990 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007406d970 con 0x7f00740b0dd0 2023-03-29T10:59:17.801 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.799+0000 7f00727fc700 1 -- 172.21.15.111:0/3463046990 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3124231432 0 0) 0x7f00740b16b0 con 0x7f0074143f10 2023-03-29T10:59:17.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.800+0000 7f0072ffd700 1 --2- 172.21.15.111:0/3463046990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00740b0dd0 0x7f00740c9290 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0068009840 tx=0x7f0068009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1a60d0eb2c4206 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:17.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.800+0000 7f00727fc700 1 -- 172.21.15.111:0/3463046990 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068011830 con 0x7f00740b0dd0 2023-03-29T10:59:17.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.800+0000 7f00727fc700 1 -- 172.21.15.111:0/3463046990 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0068011990 con 0x7f00740b0dd0 2023-03-29T10:59:17.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.800+0000 7f00727fc700 1 -- 172.21.15.111:0/3463046990 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068011d20 con 0x7f00740b0dd0 2023-03-29T10:59:17.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.800+0000 7f0079bff700 1 -- 172.21.15.111:0/3463046990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00740b0dd0 msgr2=0x7f00740c9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:17.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.800+0000 7f0079bff700 1 --2- 172.21.15.111:0/3463046990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00740b0dd0 0x7f00740c9290 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0068009840 tx=0x7f0068009b50 comp rx=0 tx=0).stop 2023-03-29T10:59:17.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.800+0000 7f0079bff700 1 -- 172.21.15.111:0/3463046990 shutdown_connections 2023-03-29T10:59:17.802 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.800+0000 7f0079bff700 1 --2- 172.21.15.111:0/3463046990 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f00740b0dd0 0x7f00740c9290 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:17.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.800+0000 7f0079bff700 1 -- 172.21.15.111:0/3463046990 >> 172.21.15.111:0/3463046990 conn(0x7f00740cb6c0 msgr2=0x7f00740cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:17.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.800+0000 7f0079bff700 1 -- 172.21.15.111:0/3463046990 shutdown_connections 2023-03-29T10:59:17.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.801+0000 7f0079bff700 1 -- 172.21.15.111:0/3463046990 wait complete. 2023-03-29T10:59:17.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.802+0000 7f0079bff700 1 Processor -- start 2023-03-29T10:59:17.803 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.802+0000 7f0079bff700 1 -- start start 2023-03-29T10:59:17.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.803+0000 7f0079bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00740b0dd0 0x7f00740aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:17.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.803+0000 7f0079bff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f00740c97d0 0x7f00740aaa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:17.804 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.803+0000 7f0079bff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0074143f10 0x7f00740c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.803+0000 7f0079bff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00740c87d0 con 0x7f0074143f10 2023-03-29T10:59:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.803+0000 7f0079bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007414aac0 con 0x7f00740b0dd0 2023-03-29T10:59:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.803+0000 7f0079bff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007414ada0 con 0x7f00740c97d0 2023-03-29T10:59:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.803+0000 7f0073fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0074143f10 0x7f00740c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.804+0000 7f0073fff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0074143f10 0x7f00740c7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:40448/0 (socket says 172.21.15.111:40448) 2023-03-29T10:59:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.804+0000 7f0073fff700 1 -- 172.21.15.111:0/346727025 learned_addr learned my addr 172.21.15.111:0/346727025 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.804+0000 7f00737fe700 1 --2- 172.21.15.111:0/346727025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00740b0dd0 0x7f00740aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.804+0000 7f0072ffd700 1 --2- 172.21.15.111:0/346727025 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f00740c97d0 0x7f00740aaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.804+0000 7f0073fff700 1 -- 172.21.15.111:0/346727025 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f00740c97d0 msgr2=0x7f00740aaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.804+0000 7f0073fff700 1 --2- 172.21.15.111:0/346727025 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f00740c97d0 0x7f00740aaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.804+0000 7f0073fff700 1 -- 172.21.15.111:0/346727025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00740b0dd0 msgr2=0x7f00740aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.804+0000 7f0073fff700 1 --2- 172.21.15.111:0/346727025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00740b0dd0 0x7f00740aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.804+0000 7f0073fff700 1 -- 172.21.15.111:0/346727025 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00680094a0 con 0x7f0074143f10 2023-03-29T10:59:17.805 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.804+0000 7f0073fff700 1 --2- 172.21.15.111:0/346727025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0074143f10 0x7f00740c7a30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f006c00cc50 tx=0x7f006c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:17.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.804+0000 7f005bfff700 1 -- 172.21.15.111:0/346727025 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c00ad80 con 0x7f0074143f10 2023-03-29T10:59:17.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.804+0000 7f005bfff700 1 -- 172.21.15.111:0/346727025 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c00ce50 con 0x7f0074143f10 2023-03-29T10:59:17.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.804+0000 7f0079bff700 1 -- 172.21.15.111:0/346727025 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f007406df30 con 0x7f0074143f10 2023-03-29T10:59:17.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.804+0000 7f005bfff700 1 -- 172.21.15.111:0/346727025 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c003c40 con 0x7f0074143f10 2023-03-29T10:59:17.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.805+0000 7f0079bff700 1 -- 172.21.15.111:0/346727025 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007406dc50 con 0x7f0074143f10 2023-03-29T10:59:17.806 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.805+0000 7f0079bff700 1 -- 172.21.15.111:0/346727025 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007406e210 con 0x7f0074143f10 2023-03-29T10:59:17.807 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.806+0000 7f005bfff700 1 -- 172.21.15.111:0/346727025 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f006c0050d0 con 0x7f0074143f10 2023-03-29T10:59:17.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.807+0000 7f005bfff700 1 --2- 172.21.15.111:0/346727025 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0060069de0 0x7f006006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:17.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.807+0000 7f005bfff700 1 -- 172.21.15.111:0/346727025 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f006c088290 con 0x7f0074143f10 2023-03-29T10:59:17.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.807+0000 7f00737fe700 1 --2- 172.21.15.111:0/346727025 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0060069de0 0x7f006006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:17.808 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.808+0000 7f00737fe700 1 --2- 172.21.15.111:0/346727025 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0060069de0 0x7f006006c290 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0064007fd0 tx=0x7f0064007c20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:17.812 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:17.811+0000 7f005bfff700 1 -- 172.21.15.111:0/346727025 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f006c053960 con 0x7f0074143f10 2023-03-29T10:59:18.053 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.053+0000 7f0079bff700 1 -- 172.21.15.111:0/346727025 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0074074430 con 0x7f0074143f10 2023-03-29T10:59:18.054 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.053+0000 7f005bfff700 1 -- 172.21.15.111:0/346727025 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4683 (secure 0 0 0) 0x7f006c056f80 con 0x7f0074143f10 2023-03-29T10:59:18.054 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:59:18.054 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5021,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.111:6835/1004725536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6834","nonce":1004725536},{"type":"v1","addr":"172.21.15.111:6835","nonce":1004725536}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5341,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2981418361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":2981418361},{"type":"v1","addr":"172.21.15.119:6835","nonce":2981418361}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5574,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/4270377263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":4270377263},{"type":"v1","addr":"172.21.15.119:6834","nonce":4270377263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-03-29T10:59:12.298455+0000","modified":"2023-03-29T10:59:15.307952+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5580},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5580":{"gid":5580,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.111:6837/3450732660","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:6836","nonce":3450732660},{"type":"v1","addr":"172.21.15.111:6837","nonce":3450732660}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-03-29T10:59:18.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.056+0000 7f0079bff700 1 -- 172.21.15.111:0/346727025 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0060069de0 msgr2=0x7f006006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:18.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.056+0000 7f0079bff700 1 --2- 172.21.15.111:0/346727025 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0060069de0 0x7f006006c290 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0064007fd0 tx=0x7f0064007c20 comp rx=0 tx=0).stop 2023-03-29T10:59:18.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.056+0000 7f0079bff700 1 -- 172.21.15.111:0/346727025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0074143f10 msgr2=0x7f00740c7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:18.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.056+0000 7f0079bff700 1 --2- 172.21.15.111:0/346727025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0074143f10 0x7f00740c7a30 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f006c00cc50 tx=0x7f006c00a340 comp rx=0 tx=0).stop 2023-03-29T10:59:18.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.056+0000 7f0079bff700 1 -- 172.21.15.111:0/346727025 shutdown_connections 2023-03-29T10:59:18.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.056+0000 7f0079bff700 1 --2- 172.21.15.111:0/346727025 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f0060069de0 0x7f006006c290 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.056+0000 7f0079bff700 1 --2- 172.21.15.111:0/346727025 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f0074143f10 0x7f00740c7a30 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.056+0000 7f0079bff700 1 --2- 172.21.15.111:0/346727025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00740b0dd0 0x7f00740aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.056+0000 7f0079bff700 1 --2- 172.21.15.111:0/346727025 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f00740c97d0 0x7f00740aaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.056+0000 7f0079bff700 1 -- 172.21.15.111:0/346727025 >> 172.21.15.111:0/346727025 conn(0x7f00740cb6c0 msgr2=0x7f007415cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:18.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.056+0000 7f0079bff700 1 -- 172.21.15.111:0/346727025 shutdown_connections 2023-03-29T10:59:18.057 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.056+0000 7f0079bff700 1 -- 172.21.15.111:0/346727025 wait complete. 2023-03-29T10:59:18.057 INFO:teuthology.orchestra.run.smithi111.stderr:dumped fsmap epoch 17 2023-03-29T10:59:18.477 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-03-29T10:59:12.298455+0000', 'modified': '2023-03-29T10:59:15.307952+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5580}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5580': {'gid': 5580, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.111:6837/3450732660', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.111:6836', 'nonce': 3450732660}, {'type': 'v1', 'addr': '172.21.15.111:6837', 'nonce': 3450732660}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-03-29T10:59:18.477 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-03-29T10:59:18.477 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-03-29T10:59:18.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.663+0000 7f141d9ff700 1 Processor -- start 2023-03-29T10:59:18.664 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.663+0000 7f141d9ff700 1 -- start start 2023-03-29T10:59:18.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.664+0000 7f141d9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f14180b87a0 0x7f14180cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.664+0000 7f141d9ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1418160d40 0x7f14180d1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.664+0000 7f141d9ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14180cdc60 con 0x7f14181601e0 2023-03-29T10:59:18.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.664+0000 7f141d9ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14180d02d0 con 0x7f1418160d40 2023-03-29T10:59:18.665 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.664+0000 7f141d9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14180d0cb0 con 0x7f14180b87a0 2023-03-29T10:59:18.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.665+0000 7f14177fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1418160d40 0x7f14180d1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.665+0000 7f14177fe700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7f1418160d40 0x7f14180d1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:37354/0 (socket says 172.21.15.111:37354) 2023-03-29T10:59:18.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.665+0000 7f14177fe700 1 -- 172.21.15.111:0/2219349632 learned_addr learned my addr 172.21.15.111:0/2219349632 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:18.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.665+0000 7f1416ffd700 1 --2- 172.21.15.111:0/2219349632 >> v2:172.21.15.119:3300/0 conn(0x7f14180b87a0 0x7f14180cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.665+0000 7f14177fe700 1 -- 172.21.15.111:0/2219349632 >> v2:172.21.15.119:3300/0 conn(0x7f14180b87a0 msgr2=0x7f14180cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:18.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.665+0000 7f14177fe700 1 --2- 172.21.15.111:0/2219349632 >> v2:172.21.15.119:3300/0 conn(0x7f14180b87a0 0x7f14180cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.665+0000 7f14177fe700 1 -- 172.21.15.111:0/2219349632 >> v1:172.21.15.119:6789/0 conn(0x7f14181601e0 legacy=0x7f1418160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:18.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.665+0000 7f14177fe700 1 -- 172.21.15.111:0/2219349632 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14180cd6a0 con 0x7f1418160d40 2023-03-29T10:59:18.667 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.665+0000 7f14177fe700 1 --2- 172.21.15.111:0/2219349632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1418160d40 0x7f14180d1100 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f14180b4ac0 tx=0x7f141000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e4b1c1ef140f6d server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:18.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.665+0000 7f1415ffb700 1 -- 172.21.15.111:0/2219349632 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141000e070 con 0x7f1418160d40 2023-03-29T10:59:18.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.665+0000 7f1415ffb700 1 -- 172.21.15.111:0/2219349632 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1410003a20 con 0x7f1418160d40 2023-03-29T10:59:18.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.665+0000 7f1415ffb700 1 -- 172.21.15.111:0/2219349632 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1410003dd0 con 0x7f1418160d40 2023-03-29T10:59:18.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.666+0000 7f141d9ff700 1 -- 172.21.15.111:0/2219349632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1418160d40 msgr2=0x7f14180d1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:18.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.666+0000 7f141d9ff700 1 --2- 172.21.15.111:0/2219349632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1418160d40 0x7f14180d1100 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f14180b4ac0 tx=0x7f141000b330 comp rx=0 tx=0).stop 2023-03-29T10:59:18.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.666+0000 7f141d9ff700 1 -- 172.21.15.111:0/2219349632 shutdown_connections 2023-03-29T10:59:18.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.666+0000 7f141d9ff700 1 --2- 172.21.15.111:0/2219349632 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f1418160d40 0x7f14180d1100 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.666+0000 7f141d9ff700 1 --2- 172.21.15.111:0/2219349632 >> v2:172.21.15.119:3300/0 conn(0x7f14180b87a0 0x7f14180cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.666+0000 7f141d9ff700 1 -- 172.21.15.111:0/2219349632 >> 172.21.15.111:0/2219349632 conn(0x7f1418071b10 msgr2=0x7f1418071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:18.668 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.666+0000 7f141d9ff700 1 -- 172.21.15.111:0/2219349632 shutdown_connections 2023-03-29T10:59:18.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.666+0000 7f141d9ff700 1 -- 172.21.15.111:0/2219349632 wait complete. 2023-03-29T10:59:18.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.668+0000 7f141d9ff700 1 Processor -- start 2023-03-29T10:59:18.669 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.669+0000 7f141d9ff700 1 -- start start 2023-03-29T10:59:18.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.670+0000 7f141d9ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14180b87a0 0x7f1418150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.670+0000 7f141d9ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14181601e0 0x7f1418154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.670+0000 7f141d9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1418160d40 0x7f1418152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.670+0000 7f141d9ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14180cba80 con 0x7f14181601e0 2023-03-29T10:59:18.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.670+0000 7f141d9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1418153160 con 0x7f1418160d40 2023-03-29T10:59:18.671 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.670+0000 7f141d9ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1418153440 con 0x7f14180b87a0 2023-03-29T10:59:18.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.670+0000 7f14167fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14181601e0 0x7f1418154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.670+0000 7f14167fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14181601e0 0x7f1418154200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:37366/0 (socket says 172.21.15.111:37366) 2023-03-29T10:59:18.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.670+0000 7f14167fc700 1 -- 172.21.15.111:0/894874888 learned_addr learned my addr 172.21.15.111:0/894874888 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:18.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.670+0000 7f14177fe700 1 --2- 172.21.15.111:0/894874888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1418160d40 0x7f1418152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.670+0000 7f1416ffd700 1 --2- 172.21.15.111:0/894874888 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14180b87a0 0x7f1418150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.670+0000 7f14167fc700 1 -- 172.21.15.111:0/894874888 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14180b87a0 msgr2=0x7f1418150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:18.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.670+0000 7f14167fc700 1 --2- 172.21.15.111:0/894874888 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14180b87a0 0x7f1418150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.670+0000 7f14167fc700 1 -- 172.21.15.111:0/894874888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1418160d40 msgr2=0x7f1418152610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:18.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.670+0000 7f14167fc700 1 --2- 172.21.15.111:0/894874888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1418160d40 0x7f1418152610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.672 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.670+0000 7f14167fc700 1 -- 172.21.15.111:0/894874888 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1410009b30 con 0x7f14181601e0 2023-03-29T10:59:18.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.671+0000 7f14167fc700 1 --2- 172.21.15.111:0/894874888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14181601e0 0x7f1418154200 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f140c00c4d0 tx=0x7f140c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:18.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.671+0000 7f13fffff700 1 -- 172.21.15.111:0/894874888 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c011790 con 0x7f14181601e0 2023-03-29T10:59:18.673 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.671+0000 7f13fffff700 1 -- 172.21.15.111:0/894874888 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c0118f0 con 0x7f14181601e0 2023-03-29T10:59:18.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.671+0000 7f13fffff700 1 -- 172.21.15.111:0/894874888 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c011ca0 con 0x7f14181601e0 2023-03-29T10:59:18.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.671+0000 7f141d9ff700 1 -- 172.21.15.111:0/894874888 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14180d1640 con 0x7f14181601e0 2023-03-29T10:59:18.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.671+0000 7f141d9ff700 1 -- 172.21.15.111:0/894874888 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14180cd980 con 0x7f14181601e0 2023-03-29T10:59:18.674 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.673+0000 7f13fffff700 1 -- 172.21.15.111:0/894874888 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f140c01c460 con 0x7f14181601e0 2023-03-29T10:59:18.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.674+0000 7f13fffff700 1 --2- 172.21.15.111:0/894874888 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1404069de0 0x7f140406c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.674+0000 7f13fffff700 1 -- 172.21.15.111:0/894874888 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f140c087ee0 con 0x7f14181601e0 2023-03-29T10:59:18.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.674+0000 7f1416ffd700 1 --2- 172.21.15.111:0/894874888 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1404069de0 0x7f140406c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.675 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.675+0000 7f1416ffd700 1 --2- 172.21.15.111:0/894874888 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1404069de0 0x7f140406c290 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1408006b20 tx=0x7f1408008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:18.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.676+0000 7f14312f6b80 1 Processor -- start 2023-03-29T10:59:18.676 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.676+0000 7f14312f6b80 1 -- start start 2023-03-29T10:59:18.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.677+0000 7f14312f6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a1a060 0x557270a1a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.677+0000 7f14312f6b80 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a59c50 0x557270a401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.677+0000 7f14312f6b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1b6e0 0x557270a406e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.677+0000 7f14312f6b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x55727099b960 con 0x557270a1b6e0 2023-03-29T10:59:18.678 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.677+0000 7f14312f6b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270925120 con 0x557270a1a060 2023-03-29T10:59:18.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.677+0000 7f14312f6b80 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x557270977690 con 0x557270a59c50 2023-03-29T10:59:18.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.677+0000 7f14177fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1b6e0 0x557270a406e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.677+0000 7f14177fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1b6e0 0x557270a406e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:37374/0 (socket says 172.21.15.111:37374) 2023-03-29T10:59:18.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.677+0000 7f14177fe700 1 -- 172.21.15.111:0/1083913822 learned_addr learned my addr 172.21.15.111:0/1083913822 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:18.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.677+0000 7f1416ffd700 1 --2- 172.21.15.111:0/1083913822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a1a060 0x557270a1a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.677+0000 7f14167fc700 1 --2- 172.21.15.111:0/1083913822 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a59c50 0x557270a401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.677+0000 7f14177fe700 1 -- 172.21.15.111:0/1083913822 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a59c50 msgr2=0x557270a401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:18.679 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.677+0000 7f14177fe700 1 --2- 172.21.15.111:0/1083913822 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a59c50 0x557270a401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.678+0000 7f14177fe700 1 -- 172.21.15.111:0/1083913822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a1a060 msgr2=0x557270a1a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:18.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.678+0000 7f14177fe700 1 --2- 172.21.15.111:0/1083913822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a1a060 0x557270a1a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.678+0000 7f14177fe700 1 -- 172.21.15.111:0/1083913822 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557270764540 con 0x557270a1b6e0 2023-03-29T10:59:18.680 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.678+0000 7f14177fe700 1 --2- 172.21.15.111:0/1083913822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1b6e0 0x557270a406e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1418072760 tx=0x7f1410010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2484df128d8819 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:18.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.678+0000 7f13fd7fa700 1 -- 172.21.15.111:0/1083913822 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1410003e10 con 0x557270a1b6e0 2023-03-29T10:59:18.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.678+0000 7f13fd7fa700 1 -- 172.21.15.111:0/1083913822 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141000b940 con 0x557270a1b6e0 2023-03-29T10:59:18.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.678+0000 7f13fd7fa700 1 -- 172.21.15.111:0/1083913822 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141000bcf0 con 0x557270a1b6e0 2023-03-29T10:59:18.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.679+0000 7f14312f6b80 1 -- 172.21.15.111:0/1083913822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1b6e0 msgr2=0x557270a406e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:18.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.679+0000 7f14312f6b80 1 --2- 172.21.15.111:0/1083913822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1b6e0 0x557270a406e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1418072760 tx=0x7f1410010000 comp rx=0 tx=0).stop 2023-03-29T10:59:18.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.679+0000 7f14312f6b80 1 -- 172.21.15.111:0/1083913822 shutdown_connections 2023-03-29T10:59:18.684 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.679+0000 7f14312f6b80 1 --2- 172.21.15.111:0/1083913822 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1b6e0 0x557270a406e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.679+0000 7f14312f6b80 1 --2- 172.21.15.111:0/1083913822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a1a060 0x557270a1a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.679+0000 7f14312f6b80 1 --2- 172.21.15.111:0/1083913822 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a59c50 0x557270a401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.679+0000 7f14312f6b80 1 -- 172.21.15.111:0/1083913822 >> 172.21.15.111:0/1083913822 conn(0x557270a12ac0 msgr2=0x557270a18030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:18.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.679+0000 7f14312f6b80 1 -- 172.21.15.111:0/1083913822 shutdown_connections 2023-03-29T10:59:18.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.679+0000 7f14312f6b80 1 -- 172.21.15.111:0/1083913822 wait complete. 2023-03-29T10:59:18.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.680+0000 7f14312f6b80 1 -- start start 2023-03-29T10:59:18.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.680+0000 7f14312f6b80 1 Processor -- start 2023-03-29T10:59:18.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.681+0000 7f14312f6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a1a060 0x557270a626b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.685 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.681+0000 7f14312f6b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1b6e0 0x557270a64f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.681+0000 7f14312f6b80 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a59c50 0x557270a65460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.681+0000 7f14312f6b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270970960 con 0x557270a1b6e0 2023-03-29T10:59:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.681+0000 7f14312f6b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270765140 con 0x557270a1a060 2023-03-29T10:59:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.681+0000 7f14312f6b80 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x557270a73d40 con 0x557270a59c50 2023-03-29T10:59:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.681+0000 7f14312f6b80 10 client.0 _ll_register_callbacks cb 0x55727090e1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-03-29T10:59:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.681+0000 7f14167fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1b6e0 0x557270a64f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.681+0000 7f14167fc700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1b6e0 0x557270a64f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:37382/0 (socket says 172.21.15.111:37382) 2023-03-29T10:59:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.681+0000 7f14167fc700 1 -- 172.21.15.111:0/365776522 learned_addr learned my addr 172.21.15.111:0/365776522 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:18.686 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.681+0000 7f14177fe700 1 --2- 172.21.15.111:0/365776522 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a59c50 0x557270a65460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.681+0000 7f1416ffd700 1 --2- 172.21.15.111:0/365776522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a1a060 0x557270a626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.681+0000 7f14167fc700 1 -- 172.21.15.111:0/365776522 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a59c50 msgr2=0x557270a65460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:18.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.681+0000 7f14167fc700 1 --2- 172.21.15.111:0/365776522 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a59c50 0x557270a65460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.681+0000 7f14167fc700 1 -- 172.21.15.111:0/365776522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a1a060 msgr2=0x557270a626b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:18.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.681+0000 7f14167fc700 1 --2- 172.21.15.111:0/365776522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a1a060 0x557270a626b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.681+0000 7f14167fc700 1 -- 172.21.15.111:0/365776522 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14180cd6a0 con 0x557270a1b6e0 2023-03-29T10:59:18.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.682+0000 7f14167fc700 1 --2- 172.21.15.111:0/365776522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1b6e0 0x557270a64f20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f140c00cb30 tx=0x7f140c011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261d146ae5db10ad server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:18.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.682+0000 7f13f67fc700 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-03-29T10:59:18.687 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.682+0000 7f13f67fc700 1 -- 172.21.15.111:0/365776522 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c05f080 con 0x557270a1b6e0 2023-03-29T10:59:18.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.682+0000 7f14312f6b80 1 -- 172.21.15.111:0/365776522 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557270a100d0 con 0x557270a1b6e0 2023-03-29T10:59:18.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.682+0000 7f13f67fc700 1 -- 172.21.15.111:0/365776522 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f140c06abb0 con 0x557270a1b6e0 2023-03-29T10:59:18.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.682+0000 7f13f67fc700 1 -- 172.21.15.111:0/365776522 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f140c060ea0 con 0x557270a1b6e0 2023-03-29T10:59:18.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.682+0000 7f14312f6b80 10 client.5772 fetch_fsmap learned FSMap version 17 2023-03-29T10:59:18.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.682+0000 7f14312f6b80 1 -- 172.21.15.111:0/365776522 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55727079eb10 con 0x557270a1b6e0 2023-03-29T10:59:18.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.682+0000 7f13f67fc700 1 -- 172.21.15.111:0/365776522 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557270a100d0 con 0x557270a1b6e0 2023-03-29T10:59:18.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.682+0000 7f13f67fc700 1 -- 172.21.15.111:0/365776522 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f140c00fb30 con 0x557270a1b6e0 2023-03-29T10:59:18.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.682+0000 7f14312f6b80 10 client.5772 fetch_fsmap finished waiting for FSMap version 17 2023-03-29T10:59:18.688 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.682+0000 7f14312f6b80 10 client.5772 resolve_mds: resolved name 'c' to daemon mds.c 2023-03-29T10:59:18.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.682+0000 7f14312f6b80 20 client.5772 populate_metadata read hostname 'smithi111' 2023-03-29T10:59:18.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.682+0000 7f14312f6b80 1 --2- 172.21.15.111:0/365776522 >> [v2:172.21.15.111:6836/3450732660,v1:172.21.15.111:6837/3450732660] conn(0x557270a636b0 0x557270a72a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.682+0000 7f14312f6b80 4 client.5772 mds_command: new command op to 5580 tid=0[{"prefix": "get_command_descriptions"}] 2023-03-29T10:59:18.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.682+0000 7f14312f6b80 1 -- 172.21.15.111:0/365776522 --> [v2:172.21.15.111:6836/3450732660,v1:172.21.15.111:6837/3450732660] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572709ed4c0 con 0x557270a636b0 2023-03-29T10:59:18.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.683+0000 7f1416ffd700 1 --2- 172.21.15.111:0/365776522 >> [v2:172.21.15.111:6836/3450732660,v1:172.21.15.111:6837/3450732660] conn(0x557270a636b0 0x557270a72a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.683+0000 7f1416ffd700 1 --2- 172.21.15.111:0/365776522 >> [v2:172.21.15.111:6836/3450732660,v1:172.21.15.111:6837/3450732660] conn(0x557270a636b0 0x557270a72a00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5989aae35471b78 server_cookie=5bbac5e397424499 in_seq=0 out_seq=0 2023-03-29T10:59:18.689 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.683+0000 7f13f67fc700 10 client.5772 ms_handle_connect on v2:172.21.15.111:6836/3450732660 2023-03-29T10:59:18.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.684+0000 7f13f67fc700 1 -- 172.21.15.111:0/365776522 <== mds.0 v2:172.21.15.111:6836/3450732660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18278 (crc 0 0 0) 0x5572709ed4c0 con 0x557270a636b0 2023-03-29T10:59:18.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.684+0000 7f13f67fc700 10 client.5772 handle_command_reply: tid=0 2023-03-29T10:59:18.690 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.684+0000 7f14312f6b80 1 client.5772 shutdown 2023-03-29T10:59:18.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.684+0000 7f14312f6b80 1 -- 172.21.15.111:0/365776522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1b6e0 msgr2=0x557270a64f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:18.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.684+0000 7f14312f6b80 1 --2- 172.21.15.111:0/365776522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1b6e0 0x557270a64f20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f140c00cb30 tx=0x7f140c011e80 comp rx=0 tx=0).stop 2023-03-29T10:59:18.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.684+0000 7f14312f6b80 1 -- 172.21.15.111:0/365776522 shutdown_connections 2023-03-29T10:59:18.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.684+0000 7f14312f6b80 1 --2- 172.21.15.111:0/365776522 >> [v2:172.21.15.111:6836/3450732660,v1:172.21.15.111:6837/3450732660] conn(0x557270a636b0 0x557270a72a00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.684+0000 7f14312f6b80 1 --2- 172.21.15.111:0/365776522 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a59c50 0x557270a65460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.684+0000 7f14312f6b80 1 --2- 172.21.15.111:0/365776522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a1a060 0x557270a626b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.684+0000 7f13f67fc700 0 client.5772 ms_handle_reset on v2:172.21.15.111:6836/3450732660 2023-03-29T10:59:18.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.684+0000 7f14312f6b80 1 --2- 172.21.15.111:0/365776522 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1b6e0 0x557270a64f20 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.691 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.684+0000 7f14312f6b80 1 -- 172.21.15.111:0/365776522 >> 172.21.15.111:0/365776522 conn(0x557270a12ac0 msgr2=0x557270a177f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:18.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.684+0000 7f14312f6b80 1 -- 172.21.15.111:0/365776522 shutdown_connections 2023-03-29T10:59:18.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.684+0000 7f14312f6b80 1 -- 172.21.15.111:0/365776522 wait complete. 2023-03-29T10:59:18.692 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.685+0000 7f14312f6b80 20 client.5772 trim_cache size 0 max 16384 2023-03-29T10:59:18.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.702+0000 7f14312f6b80 1 Processor -- start 2023-03-29T10:59:18.703 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.703+0000 7f14312f6b80 1 -- start start 2023-03-29T10:59:18.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.703+0000 7f14312f6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a1a060 0x557270a75460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.703+0000 7f14312f6b80 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a1b6e0 0x557270a63c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.703+0000 7f14312f6b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a59c50 0x557270a62100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.703+0000 7f14312f6b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270a1f7a0 con 0x557270a59c50 2023-03-29T10:59:18.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.703+0000 7f14312f6b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270a5fb70 con 0x557270a1a060 2023-03-29T10:59:18.704 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.703+0000 7f14312f6b80 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x557270a73b40 con 0x557270a1b6e0 2023-03-29T10:59:18.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.704+0000 7f14177fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a59c50 0x557270a62100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.704+0000 7f14177fe700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a59c50 0x557270a62100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:37394/0 (socket says 172.21.15.111:37394) 2023-03-29T10:59:18.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.704+0000 7f14177fe700 1 -- 172.21.15.111:0/2579418720 learned_addr learned my addr 172.21.15.111:0/2579418720 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:18.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.704+0000 7f1416ffd700 1 --2- 172.21.15.111:0/2579418720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a1a060 0x557270a75460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.706 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.704+0000 7f14167fc700 1 --2- 172.21.15.111:0/2579418720 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a1b6e0 0x557270a63c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.704+0000 7f14177fe700 1 -- 172.21.15.111:0/2579418720 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a1b6e0 msgr2=0x557270a63c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:18.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.704+0000 7f14177fe700 1 --2- 172.21.15.111:0/2579418720 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a1b6e0 0x557270a63c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.704+0000 7f14177fe700 1 -- 172.21.15.111:0/2579418720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a1a060 msgr2=0x557270a75460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:18.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.704+0000 7f14177fe700 1 --2- 172.21.15.111:0/2579418720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a1a060 0x557270a75460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.704+0000 7f14177fe700 1 -- 172.21.15.111:0/2579418720 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557270a44b00 con 0x557270a59c50 2023-03-29T10:59:18.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.704+0000 7f14177fe700 1 --2- 172.21.15.111:0/2579418720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a59c50 0x557270a62100 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1410009e50 tx=0x7f1410011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e84b0c9d8fcf82 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:18.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.704+0000 7f13fdffb700 1 -- 172.21.15.111:0/2579418720 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141000bcb0 con 0x557270a59c50 2023-03-29T10:59:18.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.705+0000 7f13fdffb700 1 -- 172.21.15.111:0/2579418720 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141000be10 con 0x557270a59c50 2023-03-29T10:59:18.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.705+0000 7f13fdffb700 1 -- 172.21.15.111:0/2579418720 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1410004270 con 0x557270a59c50 2023-03-29T10:59:18.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.705+0000 7f14312f6b80 1 -- 172.21.15.111:0/2579418720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a59c50 msgr2=0x557270a62100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:18.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.705+0000 7f14312f6b80 1 --2- 172.21.15.111:0/2579418720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a59c50 0x557270a62100 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1410009e50 tx=0x7f1410011000 comp rx=0 tx=0).stop 2023-03-29T10:59:18.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.705+0000 7f14312f6b80 1 -- 172.21.15.111:0/2579418720 shutdown_connections 2023-03-29T10:59:18.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.705+0000 7f14312f6b80 1 --2- 172.21.15.111:0/2579418720 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a59c50 0x557270a62100 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.705+0000 7f14312f6b80 1 --2- 172.21.15.111:0/2579418720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a1a060 0x557270a75460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.705+0000 7f14312f6b80 1 --2- 172.21.15.111:0/2579418720 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a1b6e0 0x557270a63c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.705+0000 7f14312f6b80 1 -- 172.21.15.111:0/2579418720 >> 172.21.15.111:0/2579418720 conn(0x557270a12ac0 msgr2=0x557270a45400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:18.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.705+0000 7f14312f6b80 1 -- 172.21.15.111:0/2579418720 shutdown_connections 2023-03-29T10:59:18.708 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.705+0000 7f14312f6b80 1 -- 172.21.15.111:0/2579418720 wait complete. 2023-03-29T10:59:18.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.706+0000 7f14312f6b80 1 -- start start 2023-03-29T10:59:18.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.706+0000 7f14312f6b80 1 Processor -- start 2023-03-29T10:59:18.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.707+0000 7f14312f6b80 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1a060 0x557270a61000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.707+0000 7f14312f6b80 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a1b6e0 0x557270a62100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.707+0000 7f14312f6b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a59c50 0x557270a62640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.707+0000 7f14312f6b80 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270a1c890 con 0x557270a1a060 2023-03-29T10:59:18.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.707+0000 7f14312f6b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557270a653b0 con 0x557270a59c50 2023-03-29T10:59:18.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.707+0000 7f14312f6b80 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x557270a65920 con 0x557270a1b6e0 2023-03-29T10:59:18.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.707+0000 7f14312f6b80 10 client.0 _ll_register_callbacks cb 0x7f140c056b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-03-29T10:59:18.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.707+0000 7f1416ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1a060 0x557270a61000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.712 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.707+0000 7f1416ffd700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1a060 0x557270a61000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:37396/0 (socket says 172.21.15.111:37396) 2023-03-29T10:59:18.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.707+0000 7f1416ffd700 1 -- 172.21.15.111:0/3104073754 learned_addr learned my addr 172.21.15.111:0/3104073754 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:18.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.707+0000 7f14177fe700 1 --2- 172.21.15.111:0/3104073754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a59c50 0x557270a62640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.707+0000 7f14167fc700 1 --2- 172.21.15.111:0/3104073754 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a1b6e0 0x557270a62100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.708+0000 7f1416ffd700 1 -- 172.21.15.111:0/3104073754 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a1b6e0 msgr2=0x557270a62100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:18.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.708+0000 7f1416ffd700 1 --2- 172.21.15.111:0/3104073754 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a1b6e0 0x557270a62100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.708+0000 7f1416ffd700 1 -- 172.21.15.111:0/3104073754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a59c50 msgr2=0x557270a62640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:18.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.708+0000 7f1416ffd700 1 --2- 172.21.15.111:0/3104073754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a59c50 0x557270a62640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.708+0000 7f1416ffd700 1 -- 172.21.15.111:0/3104073754 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557270764540 con 0x557270a1a060 2023-03-29T10:59:18.713 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.708+0000 7f14167fc700 1 --2- 172.21.15.111:0/3104073754 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a1b6e0 0x557270a62100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-03-29T10:59:18.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.708+0000 7f1416ffd700 1 --2- 172.21.15.111:0/3104073754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1a060 0x557270a61000 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1408006f10 tx=0x7f14080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51ce2f7a2810902 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:18.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.708+0000 7f13f67fc700 10 client.0 ms_handle_connect on v2:172.21.15.111:3300/0 2023-03-29T10:59:18.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.708+0000 7f13f67fc700 1 -- 172.21.15.111:0/3104073754 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140802bcd0 con 0x557270a1a060 2023-03-29T10:59:18.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.708+0000 7f14312f6b80 1 -- 172.21.15.111:0/3104073754 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f141000b940 con 0x557270a1a060 2023-03-29T10:59:18.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.708+0000 7f13f67fc700 1 -- 172.21.15.111:0/3104073754 <== mon.0 v2:172.21.15.111:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1408028070 con 0x557270a1a060 2023-03-29T10:59:18.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.708+0000 7f13f67fc700 1 -- 172.21.15.111:0/3104073754 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f140802be30 con 0x557270a1a060 2023-03-29T10:59:18.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.708+0000 7f14312f6b80 10 client.5778 fetch_fsmap learned FSMap version 17 2023-03-29T10:59:18.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.708+0000 7f14312f6b80 1 -- 172.21.15.111:0/3104073754 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55727091da60 con 0x557270a1a060 2023-03-29T10:59:18.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.709+0000 7f13f67fc700 1 -- 172.21.15.111:0/3104073754 <== mon.0 v2:172.21.15.111:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141000b940 con 0x557270a1a060 2023-03-29T10:59:18.714 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.709+0000 7f13f67fc700 1 -- 172.21.15.111:0/3104073754 <== mon.0 v2:172.21.15.111:3300/0 5 ==== fsmap(e 17) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1408034470 con 0x557270a1a060 2023-03-29T10:59:18.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.709+0000 7f14312f6b80 10 client.5778 fetch_fsmap finished waiting for FSMap version 17 2023-03-29T10:59:18.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.709+0000 7f14312f6b80 10 client.5778 resolve_mds: resolved name 'c' to daemon mds.c 2023-03-29T10:59:18.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.709+0000 7f14312f6b80 20 client.5778 populate_metadata read hostname 'smithi111' 2023-03-29T10:59:18.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.709+0000 7f14312f6b80 1 --2- 172.21.15.111:0/3104073754 >> [v2:172.21.15.111:6836/3450732660,v1:172.21.15.111:6837/3450732660] conn(0x557270a1bd40 0x557270a1c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.709+0000 7f14312f6b80 4 client.5778 mds_command: new command op to 5580 tid=0[{"prefix": "status"}] 2023-03-29T10:59:18.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.709+0000 7f14312f6b80 1 -- 172.21.15.111:0/3104073754 --> [v2:172.21.15.111:6836/3450732660,v1:172.21.15.111:6837/3450732660] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557270969aa0 con 0x557270a1bd40 2023-03-29T10:59:18.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.709+0000 7f14167fc700 1 --2- 172.21.15.111:0/3104073754 >> [v2:172.21.15.111:6836/3450732660,v1:172.21.15.111:6837/3450732660] conn(0x557270a1bd40 0x557270a1c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.715 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.709+0000 7f14167fc700 1 --2- 172.21.15.111:0/3104073754 >> [v2:172.21.15.111:6836/3450732660,v1:172.21.15.111:6837/3450732660] conn(0x557270a1bd40 0x557270a1c110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e03897a7c5fcd59 server_cookie=3b01fd47132f4b59 in_seq=0 out_seq=0 2023-03-29T10:59:18.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.709+0000 7f13f67fc700 10 client.5778 ms_handle_connect on v2:172.21.15.111:6836/3450732660 2023-03-29T10:59:18.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.710+0000 7f13f67fc700 1 -- 172.21.15.111:0/3104073754 <== mds.0 v2:172.21.15.111:6836/3450732660 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x557270969aa0 con 0x557270a1bd40 2023-03-29T10:59:18.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.710+0000 7f13f67fc700 10 client.5778 handle_command_reply: tid=0 2023-03-29T10:59:18.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.710+0000 7f14312f6b80 1 client.5778 shutdown 2023-03-29T10:59:18.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.710+0000 7f14312f6b80 1 -- 172.21.15.111:0/3104073754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1a060 msgr2=0x557270a61000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:18.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.710+0000 7f14312f6b80 1 --2- 172.21.15.111:0/3104073754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1a060 0x557270a61000 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1408006f10 tx=0x7f14080047c0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.710+0000 7f14312f6b80 1 -- 172.21.15.111:0/3104073754 shutdown_connections 2023-03-29T10:59:18.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.710+0000 7f14312f6b80 1 --2- 172.21.15.111:0/3104073754 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x557270a1a060 0x557270a61000 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.716 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.710+0000 7f14312f6b80 1 --2- 172.21.15.111:0/3104073754 >> [v2:172.21.15.111:6836/3450732660,v1:172.21.15.111:6837/3450732660] conn(0x557270a1bd40 0x557270a1c110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.710+0000 7f14312f6b80 1 --2- 172.21.15.111:0/3104073754 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557270a59c50 0x557270a62640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.710+0000 7f14312f6b80 1 --2- 172.21.15.111:0/3104073754 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x557270a1b6e0 0x557270a62100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.710+0000 7f14312f6b80 1 -- 172.21.15.111:0/3104073754 >> 172.21.15.111:0/3104073754 conn(0x557270a12ac0 msgr2=0x557270a60780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:18.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.710+0000 7f13f67fc700 0 client.5778 ms_handle_reset on v2:172.21.15.111:6836/3450732660 2023-03-29T10:59:18.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.710+0000 7f14312f6b80 1 -- 172.21.15.111:0/3104073754 shutdown_connections 2023-03-29T10:59:18.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.710+0000 7f14312f6b80 1 -- 172.21.15.111:0/3104073754 wait complete. 2023-03-29T10:59:18.717 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.710+0000 7f14312f6b80 20 client.5778 trim_cache size 0 max 16384 2023-03-29T10:59:18.718 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-03-29T10:59:18.718 INFO:teuthology.orchestra.run.smithi111.stdout: "cluster_fsid": "c8526f71-9b58-4b2e-978f-708199ec8aef", 2023-03-29T10:59:18.718 INFO:teuthology.orchestra.run.smithi111.stdout: "whoami": 0, 2023-03-29T10:59:18.718 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 5580, 2023-03-29T10:59:18.718 INFO:teuthology.orchestra.run.smithi111.stdout: "want_state": "up:active", 2023-03-29T10:59:18.718 INFO:teuthology.orchestra.run.smithi111.stdout: "state": "up:active", 2023-03-29T10:59:18.718 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs", 2023-03-29T10:59:18.718 INFO:teuthology.orchestra.run.smithi111.stdout: "rank_uptime": 6.4101557619999996, 2023-03-29T10:59:18.718 INFO:teuthology.orchestra.run.smithi111.stdout: "mdsmap_epoch": 17, 2023-03-29T10:59:18.719 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch": 43, 2023-03-29T10:59:18.719 INFO:teuthology.orchestra.run.smithi111.stdout: "osdmap_epoch_barrier": 43, 2023-03-29T10:59:18.719 INFO:teuthology.orchestra.run.smithi111.stdout: "uptime": 14.690860081 2023-03-29T10:59:18.719 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-03-29T10:59:18.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.711+0000 7f141d9ff700 1 -- 172.21.15.111:0/894874888 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1404069de0 msgr2=0x7f140406c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:18.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.711+0000 7f141d9ff700 1 --2- 172.21.15.111:0/894874888 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1404069de0 0x7f140406c290 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1408006b20 tx=0x7f1408008040 comp rx=0 tx=0).stop 2023-03-29T10:59:18.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.711+0000 7f141d9ff700 1 -- 172.21.15.111:0/894874888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14181601e0 msgr2=0x7f1418154200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:18.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.711+0000 7f141d9ff700 1 --2- 172.21.15.111:0/894874888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14181601e0 0x7f1418154200 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f140c00c4d0 tx=0x7f140c00c7e0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.711+0000 7f141d9ff700 1 -- 172.21.15.111:0/894874888 shutdown_connections 2023-03-29T10:59:18.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.711+0000 7f141d9ff700 1 --2- 172.21.15.111:0/894874888 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f1404069de0 0x7f140406c290 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.711+0000 7f141d9ff700 1 --2- 172.21.15.111:0/894874888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1418160d40 0x7f1418152610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.720 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.711+0000 7f141d9ff700 1 --2- 172.21.15.111:0/894874888 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f14180b87a0 0x7f1418150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.711+0000 7f141d9ff700 1 --2- 172.21.15.111:0/894874888 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f14181601e0 0x7f1418154200 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.711+0000 7f141d9ff700 1 -- 172.21.15.111:0/894874888 >> 172.21.15.111:0/894874888 conn(0x7f1418071b10 msgr2=0x7f1418071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:18.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.711+0000 7f141d9ff700 1 -- 172.21.15.111:0/894874888 shutdown_connections 2023-03-29T10:59:18.721 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.711+0000 7f141d9ff700 1 -- 172.21.15.111:0/894874888 wait complete. 2023-03-29T10:59:18.728 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-03-29T10:59:18.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.910+0000 7fd53c1ff700 1 Processor -- start 2023-03-29T10:59:18.911 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.910+0000 7fd53c1ff700 1 -- start start 2023-03-29T10:59:18.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.911+0000 7fd53c1ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd5340cf970 0x7fd5340cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.911+0000 7fd53c1ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5340cdc10 con 0x7fd5340b70f0 2023-03-29T10:59:18.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.911+0000 7fd53c1ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5340b6c60 con 0x7fd5340cfd40 2023-03-29T10:59:18.912 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.911+0000 7fd53c1ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5340d08d0 con 0x7fd5340cf970 2023-03-29T10:59:18.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.911+0000 7fd539f9b700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fd5340cf970 0x7fd5340cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.912+0000 7fd53a79c700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7fd5340b70f0 0x7fd5340b74f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:53956/0 (socket says 172.21.15.111:53956) 2023-03-29T10:59:18.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.912+0000 7fd53a79c700 1 -- 172.21.15.111:0/301956576 learned_addr learned my addr 172.21.15.111:0/301956576 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:59:18.913 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.912+0000 7fd539f9b700 1 -- 172.21.15.111:0/301956576 >> v1:172.21.15.119:6789/0 conn(0x7fd5340cfd40 legacy=0x7fd5340d01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:18.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.912+0000 7fd539f9b700 1 -- 172.21.15.111:0/301956576 >> v1:172.21.15.111:6789/0 conn(0x7fd5340b70f0 legacy=0x7fd5340b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:18.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.912+0000 7fd539f9b700 1 -- 172.21.15.111:0/301956576 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5340cd650 con 0x7fd5340cf970 2023-03-29T10:59:18.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.912+0000 7fd538f99700 1 -- 172.21.15.111:0/301956576 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 909490167 0 0) 0x7fd5340cdc10 con 0x7fd5340b70f0 2023-03-29T10:59:18.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.912+0000 7fd539f9b700 1 --2- 172.21.15.111:0/301956576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5340cf970 0x7fd5340cf110 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd530009840 tx=0x7fd530009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8894e0c6aa10e485 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:18.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.912+0000 7fd538f99700 1 -- 172.21.15.111:0/301956576 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd530011830 con 0x7fd5340cf970 2023-03-29T10:59:18.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.913+0000 7fd538f99700 1 -- 172.21.15.111:0/301956576 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd530011990 con 0x7fd5340cf970 2023-03-29T10:59:18.914 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.913+0000 7fd538f99700 1 -- 172.21.15.111:0/301956576 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd530011d20 con 0x7fd5340cf970 2023-03-29T10:59:18.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.913+0000 7fd53c1ff700 1 -- 172.21.15.111:0/301956576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5340cf970 msgr2=0x7fd5340cf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:18.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.913+0000 7fd53c1ff700 1 --2- 172.21.15.111:0/301956576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5340cf970 0x7fd5340cf110 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd530009840 tx=0x7fd530009b50 comp rx=0 tx=0).stop 2023-03-29T10:59:18.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.913+0000 7fd53c1ff700 1 -- 172.21.15.111:0/301956576 shutdown_connections 2023-03-29T10:59:18.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.913+0000 7fd53c1ff700 1 --2- 172.21.15.111:0/301956576 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5340cf970 0x7fd5340cf110 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.913+0000 7fd53c1ff700 1 -- 172.21.15.111:0/301956576 >> 172.21.15.111:0/301956576 conn(0x7fd534071b10 msgr2=0x7fd534071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:18.915 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.913+0000 7fd53c1ff700 1 -- 172.21.15.111:0/301956576 shutdown_connections 2023-03-29T10:59:18.916 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.913+0000 7fd53c1ff700 1 -- 172.21.15.111:0/301956576 wait complete. 2023-03-29T10:59:18.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.915+0000 7fd53c1ff700 1 Processor -- start 2023-03-29T10:59:18.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.915+0000 7fd53c1ff700 1 -- start start 2023-03-29T10:59:18.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.916+0000 7fd53c1ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5340b70f0 0x7fd53415af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.916+0000 7fd53c1ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5340cf970 0x7fd5340c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.916+0000 7fd53c1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5340cfd40 0x7fd5340c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.916+0000 7fd53c1ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5340c7ff0 con 0x7fd5340cf970 2023-03-29T10:59:18.918 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.916+0000 7fd53c1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5340caf90 con 0x7fd5340cfd40 2023-03-29T10:59:18.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.916+0000 7fd53c1ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5340ca1d0 con 0x7fd5340b70f0 2023-03-29T10:59:18.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.916+0000 7fd53979a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5340cf970 0x7fd5340c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.916+0000 7fd53979a700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5340cf970 0x7fd5340c7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:37404/0 (socket says 172.21.15.111:37404) 2023-03-29T10:59:18.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.916+0000 7fd53979a700 1 -- 172.21.15.111:0/208179521 learned_addr learned my addr 172.21.15.111:0/208179521 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:18.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.916+0000 7fd53a79c700 1 --2- 172.21.15.111:0/208179521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5340cfd40 0x7fd5340c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.916+0000 7fd539f9b700 1 --2- 172.21.15.111:0/208179521 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5340b70f0 0x7fd53415af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.916+0000 7fd53979a700 1 -- 172.21.15.111:0/208179521 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5340b70f0 msgr2=0x7fd53415af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:18.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.916+0000 7fd53979a700 1 --2- 172.21.15.111:0/208179521 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5340b70f0 0x7fd53415af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.919 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.917+0000 7fd53979a700 1 -- 172.21.15.111:0/208179521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5340cfd40 msgr2=0x7fd5340c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:18.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.917+0000 7fd53979a700 1 --2- 172.21.15.111:0/208179521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5340cfd40 0x7fd5340c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:18.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.917+0000 7fd53979a700 1 -- 172.21.15.111:0/208179521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5300094a0 con 0x7fd5340cf970 2023-03-29T10:59:18.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.917+0000 7fd53a79c700 1 --2- 172.21.15.111:0/208179521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5340cfd40 0x7fd5340c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:59:18.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.917+0000 7fd53979a700 1 --2- 172.21.15.111:0/208179521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5340cf970 0x7fd5340c7480 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd52400c480 tx=0x7fd52400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:18.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.917+0000 7fd52a7fc700 1 -- 172.21.15.111:0/208179521 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5240052d0 con 0x7fd5340cf970 2023-03-29T10:59:18.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.917+0000 7fd52a7fc700 1 -- 172.21.15.111:0/208179521 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd524005430 con 0x7fd5340cf970 2023-03-29T10:59:18.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.917+0000 7fd52a7fc700 1 -- 172.21.15.111:0/208179521 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5240038c0 con 0x7fd5340cf970 2023-03-29T10:59:18.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.917+0000 7fd53c1ff700 1 -- 172.21.15.111:0/208179521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5340b68f0 con 0x7fd5340cf970 2023-03-29T10:59:18.920 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.917+0000 7fd53c1ff700 1 -- 172.21.15.111:0/208179521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5340cd930 con 0x7fd5340cf970 2023-03-29T10:59:18.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.918+0000 7fd53c1ff700 1 -- 172.21.15.111:0/208179521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5340cf650 con 0x7fd5340cf970 2023-03-29T10:59:18.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.919+0000 7fd52a7fc700 1 -- 172.21.15.111:0/208179521 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fd524003a20 con 0x7fd5340cf970 2023-03-29T10:59:18.921 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.920+0000 7fd52a7fc700 1 --2- 172.21.15.111:0/208179521 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd518069de0 0x7fd51806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:18.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.920+0000 7fd52a7fc700 1 -- 172.21.15.111:0/208179521 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd5240874d0 con 0x7fd5340cf970 2023-03-29T10:59:18.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.920+0000 7fd539f9b700 1 --2- 172.21.15.111:0/208179521 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd518069de0 0x7fd51806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:18.922 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.921+0000 7fd539f9b700 1 --2- 172.21.15.111:0/208179521 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd518069de0 0x7fd51806c290 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd5300090b0 tx=0x7fd53001c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:18.924 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:18.924+0000 7fd52a7fc700 1 -- 172.21.15.111:0/208179521 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fd524052c40 con 0x7fd5340cf970 2023-03-29T10:59:19.167 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:19.167+0000 7fd53c1ff700 1 -- 172.21.15.111:0/208179521 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd534074430 con 0x7fd5340cf970 2023-03-29T10:59:19.168 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:19.167+0000 7fd52a7fc700 1 -- 172.21.15.111:0/208179521 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd524056260 con 0x7fd5340cf970 2023-03-29T10:59:19.168 INFO:teuthology.orchestra.run.smithi111.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-03-29T10:59:19.170 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:19.170+0000 7fd53c1ff700 1 -- 172.21.15.111:0/208179521 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd518069de0 msgr2=0x7fd51806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:19.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:19.170+0000 7fd53c1ff700 1 --2- 172.21.15.111:0/208179521 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd518069de0 0x7fd51806c290 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd5300090b0 tx=0x7fd53001c040 comp rx=0 tx=0).stop 2023-03-29T10:59:19.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:19.170+0000 7fd53c1ff700 1 -- 172.21.15.111:0/208179521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5340cf970 msgr2=0x7fd5340c7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:19.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:19.170+0000 7fd53c1ff700 1 --2- 172.21.15.111:0/208179521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5340cf970 0x7fd5340c7480 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd52400c480 tx=0x7fd52400c790 comp rx=0 tx=0).stop 2023-03-29T10:59:19.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:19.170+0000 7fd53c1ff700 1 -- 172.21.15.111:0/208179521 shutdown_connections 2023-03-29T10:59:19.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:19.170+0000 7fd53c1ff700 1 --2- 172.21.15.111:0/208179521 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fd518069de0 0x7fd51806c290 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:19.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:19.170+0000 7fd53c1ff700 1 --2- 172.21.15.111:0/208179521 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5340cfd40 0x7fd5340c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:19.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:19.170+0000 7fd53c1ff700 1 --2- 172.21.15.111:0/208179521 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fd5340b70f0 0x7fd53415af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:19.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:19.170+0000 7fd53c1ff700 1 --2- 172.21.15.111:0/208179521 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fd5340cf970 0x7fd5340c7480 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:19.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:19.170+0000 7fd53c1ff700 1 -- 172.21.15.111:0/208179521 >> 172.21.15.111:0/208179521 conn(0x7fd534071b10 msgr2=0x7fd534072d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:19.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:19.170+0000 7fd53c1ff700 1 -- 172.21.15.111:0/208179521 shutdown_connections 2023-03-29T10:59:19.171 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:19.170+0000 7fd53c1ff700 1 -- 172.21.15.111:0/208179521 wait complete. 2023-03-29T10:59:19.188 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-03-29T10:59:19.188 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-03-29T10:59:19.188 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi111.front.sepia.ceph.com 2023-03-29T10:59:19.188 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-03-29T10:59:19.189 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-03-29T10:59:19.189 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-03-29T10:59:19.189 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-03-29T10:59:19.189 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-03-29T10:59:19.189 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-03-29T10:59:19.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:59:19.189 DEBUG:teuthology.orchestra.run.smithi111:> ip addr 2023-03-29T10:59:19.230 INFO:teuthology.orchestra.run.smithi111.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-03-29T10:59:19.231 INFO:teuthology.orchestra.run.smithi111.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-03-29T10:59:19.231 INFO:teuthology.orchestra.run.smithi111.stdout: inet 127.0.0.1/8 scope host lo 2023-03-29T10:59:19.231 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:59:19.231 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 ::1/128 scope host 2023-03-29T10:59:19.231 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:59:19.231 INFO:teuthology.orchestra.run.smithi111.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-29T10:59:19.231 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:24 brd ff:ff:ff:ff:ff:ff 2023-03-29T10:59:19.231 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f0 2023-03-29T10:59:19.231 INFO:teuthology.orchestra.run.smithi111.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-29T10:59:19.231 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:d8:a1:25 brd ff:ff:ff:ff:ff:ff 2023-03-29T10:59:19.231 INFO:teuthology.orchestra.run.smithi111.stdout: altname enp7s0f1 2023-03-29T10:59:19.231 INFO:teuthology.orchestra.run.smithi111.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-03-29T10:59:19.231 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2e brd ff:ff:ff:ff:ff:ff 2023-03-29T10:59:19.231 INFO:teuthology.orchestra.run.smithi111.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-03-29T10:59:19.231 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 0c:c4:7a:8f:cc:2f brd ff:ff:ff:ff:ff:ff 2023-03-29T10:59:19.231 INFO:teuthology.orchestra.run.smithi111.stdout: inet 172.21.15.111/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-03-29T10:59:19.231 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft 42258sec preferred_lft 42258sec 2023-03-29T10:59:19.231 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::ec4:7aff:fe8f:cc2f/64 scope link 2023-03-29T10:59:19.232 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:59:19.232 INFO:teuthology.orchestra.run.smithi111.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-03-29T10:59:19.232 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 8a:68:29:ab:22:51 brd ff:ff:ff:ff:ff:ff 2023-03-29T10:59:19.232 INFO:teuthology.orchestra.run.smithi111.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-03-29T10:59:19.232 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:59:19.232 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::54c9:aeff:fe97:250b/64 scope link 2023-03-29T10:59:19.232 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:59:19.232 INFO:teuthology.orchestra.run.smithi111.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-03-29T10:59:19.232 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether f2:0a:47:b9:bd:43 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-03-29T10:59:19.232 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::f00a:47ff:feb9:bd43/64 scope link 2023-03-29T10:59:19.232 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:59:19.232 INFO:teuthology.orchestra.run.smithi111.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-03-29T10:59:19.232 INFO:teuthology.orchestra.run.smithi111.stdout: link/ether 8a:68:29:ab:22:51 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-03-29T10:59:19.232 INFO:teuthology.orchestra.run.smithi111.stdout: inet6 fe80::8868:29ff:feab:2251/64 scope link 2023-03-29T10:59:19.232 INFO:teuthology.orchestra.run.smithi111.stdout: valid_lft forever preferred_lft forever 2023-03-29T10:59:19.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:59:19.233 DEBUG:teuthology.orchestra.run.smithi111:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-03-29T10:59:19.330 INFO:teuthology.orchestra.run.smithi111.stdout:1 2023-03-29T10:59:19.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:59:19.333 DEBUG:teuthology.orchestra.run.smithi111:> route 2023-03-29T10:59:19.376 INFO:teuthology.orchestra.run.smithi111.stdout:Kernel IP routing table 2023-03-29T10:59:19.376 INFO:teuthology.orchestra.run.smithi111.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-03-29T10:59:19.376 INFO:teuthology.orchestra.run.smithi111.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-03-29T10:59:19.376 INFO:teuthology.orchestra.run.smithi111.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-03-29T10:59:19.377 INFO:teuthology.orchestra.run.smithi111.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-03-29T10:59:19.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:59:19.377 DEBUG:teuthology.orchestra.run.smithi111:> (cd / && exec bash -c ' 2023-03-29T10:59:19.377 DEBUG:teuthology.orchestra.run.smithi111:> set -e 2023-03-29T10:59:19.377 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-03-29T10:59:19.377 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-03-29T10:59:19.377 DEBUG:teuthology.orchestra.run.smithi111:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-03-29T10:59:19.377 DEBUG:teuthology.orchestra.run.smithi111:> ') 2023-03-29T10:59:19.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:59:19.567 DEBUG:teuthology.orchestra.run.smithi111:> ip netns list 2023-03-29T10:59:19.608 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-03-29T10:59:19.608 INFO:teuthology.orchestra.run.smithi111.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-03-29T10:59:19.608 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-03-29T10:59:19.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:59:19.608 DEBUG:teuthology.orchestra.run.smithi111:> sudo ip link set brx.0 up 2023-03-29T10:59:19.711 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-03-29T10:59:19.711 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-29T10:59:19.711 DEBUG:teuthology.orchestra.run.smithi111:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-03-29T10:59:19.752 INFO:teuthology.orchestra.run.smithi111.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-03-29T10:59:19.752 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-03-29T10:59:19.752 DEBUG:teuthology.orchestra.run.smithi111:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-03-29T10:59:19.832 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2023-03-29T10:59:19.935 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-03-29T10:59:19.935 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-03-29T10:59:19.997 INFO:teuthology.orchestra.run.smithi111.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-03-29T10:59:19.998 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-03-29T10:59:19.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:59:19.999 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2023-03-29T10:59:20.039 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-03-29T10:59:20.039 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-03-29T10:59:20.084 DEBUG:teuthology.orchestra.run.smithi111:> sudo modprobe fuse 2023-03-29T10:59:20.146 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-03-29T10:59:20.146 DEBUG:teuthology.orchestra.run.smithi111:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-03-29T10:59:20.210 INFO:teuthology.orchestra.run.smithi111.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-03-29T10:59:20.212 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-03-29T10:59:20.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:59:20.213 DEBUG:teuthology.orchestra.run.smithi111:> ls /sys/fs/fuse/connections 2023-03-29T10:59:20.240 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:2023-03-29T10:59:20.239+0000 7fbeffdb75c0 -1 init, newargv = 0x55f6fb303ba0 newargc=15 2023-03-29T10:59:20.240 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[130136]: starting ceph client 2023-03-29T10:59:20.247 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[130136]: starting fuse 2023-03-29T10:59:20.257 INFO:teuthology.orchestra.run.smithi111.stdout:42 2023-03-29T10:59:20.257 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2023-03-29T10:59:20.258 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-03-29T10:59:20.258 DEBUG:teuthology.orchestra.run.smithi111:> import glob 2023-03-29T10:59:20.258 DEBUG:teuthology.orchestra.run.smithi111:> import re 2023-03-29T10:59:20.258 DEBUG:teuthology.orchestra.run.smithi111:> import os 2023-03-29T10:59:20.258 DEBUG:teuthology.orchestra.run.smithi111:> import subprocess 2023-03-29T10:59:20.258 DEBUG:teuthology.orchestra.run.smithi111:> 2023-03-29T10:59:20.258 DEBUG:teuthology.orchestra.run.smithi111:> def _find_admin_socket(client_name): 2023-03-29T10:59:20.258 DEBUG:teuthology.orchestra.run.smithi111:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-03-29T10:59:20.258 DEBUG:teuthology.orchestra.run.smithi111:> files = glob.glob(asok_path) 2023-03-29T10:59:20.258 DEBUG:teuthology.orchestra.run.smithi111:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-03-29T10:59:20.258 DEBUG:teuthology.orchestra.run.smithi111:> 2023-03-29T10:59:20.258 DEBUG:teuthology.orchestra.run.smithi111:> # Given a non-glob path, it better be there 2023-03-29T10:59:20.258 DEBUG:teuthology.orchestra.run.smithi111:> if "*" not in asok_path: 2023-03-29T10:59:20.258 DEBUG:teuthology.orchestra.run.smithi111:> assert(len(files) == 1) 2023-03-29T10:59:20.258 DEBUG:teuthology.orchestra.run.smithi111:> return files[0] 2023-03-29T10:59:20.259 DEBUG:teuthology.orchestra.run.smithi111:> 2023-03-29T10:59:20.259 DEBUG:teuthology.orchestra.run.smithi111:> for f in files: 2023-03-29T10:59:20.259 DEBUG:teuthology.orchestra.run.smithi111:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-03-29T10:59:20.259 DEBUG:teuthology.orchestra.run.smithi111:> if os.path.exists("/proc/{0}".format(pid)): 2023-03-29T10:59:20.259 DEBUG:teuthology.orchestra.run.smithi111:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-03-29T10:59:20.259 DEBUG:teuthology.orchestra.run.smithi111:> contents = proc_f.read() 2023-03-29T10:59:20.259 DEBUG:teuthology.orchestra.run.smithi111:> if mountpoint in contents: 2023-03-29T10:59:20.259 DEBUG:teuthology.orchestra.run.smithi111:> return f 2023-03-29T10:59:20.259 DEBUG:teuthology.orchestra.run.smithi111:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-03-29T10:59:20.259 DEBUG:teuthology.orchestra.run.smithi111:> 2023-03-29T10:59:20.259 DEBUG:teuthology.orchestra.run.smithi111:> print(_find_admin_socket("client.0")) 2023-03-29T10:59:20.259 DEBUG:teuthology.orchestra.run.smithi111:> ' 2023-03-29T10:59:20.434 INFO:teuthology.orchestra.run.smithi111.stdout:/var/run/ceph/ceph-client.0.130136.asok 2023-03-29T10:59:20.612 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.130136.asok 2023-03-29T10:59:20.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:59:20.613 DEBUG:teuthology.orchestra.run.smithi111:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.130136.asok status 2023-03-29T10:59:20.828 INFO:teuthology.orchestra.run.smithi111.stdout:{ 2023-03-29T10:59:20.829 INFO:teuthology.orchestra.run.smithi111.stdout: "metadata": { 2023-03-29T10:59:20.829 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_sha1": "f03d69a8741c605ae6711afe2c9991ac46095f7d", 2023-03-29T10:59:20.829 INFO:teuthology.orchestra.run.smithi111.stdout: "ceph_version": "ceph version 18.0.0-3066-gf03d69a8 (f03d69a8741c605ae6711afe2c9991ac46095f7d) reef (dev)", 2023-03-29T10:59:20.829 INFO:teuthology.orchestra.run.smithi111.stdout: "entity_id": "0", 2023-03-29T10:59:20.829 INFO:teuthology.orchestra.run.smithi111.stdout: "hostname": "smithi111", 2023-03-29T10:59:20.829 INFO:teuthology.orchestra.run.smithi111.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-03-29T10:59:20.829 INFO:teuthology.orchestra.run.smithi111.stdout: "pid": "130136", 2023-03-29T10:59:20.830 INFO:teuthology.orchestra.run.smithi111.stdout: "root": "/" 2023-03-29T10:59:20.830 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:20.830 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_count": 0, 2023-03-29T10:59:20.830 INFO:teuthology.orchestra.run.smithi111.stdout: "dentry_pinned_count": 0, 2023-03-29T10:59:20.830 INFO:teuthology.orchestra.run.smithi111.stdout: "id": 5793, 2023-03-29T10:59:20.830 INFO:teuthology.orchestra.run.smithi111.stdout: "inst": { 2023-03-29T10:59:20.830 INFO:teuthology.orchestra.run.smithi111.stdout: "name": { 2023-03-29T10:59:20.830 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "client", 2023-03-29T10:59:20.830 INFO:teuthology.orchestra.run.smithi111.stdout: "num": 5793 2023-03-29T10:59:20.830 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:20.830 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2023-03-29T10:59:20.830 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2023-03-29T10:59:20.830 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2023-03-29T10:59:20.831 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 2722392590 2023-03-29T10:59:20.831 INFO:teuthology.orchestra.run.smithi111.stdout: } 2023-03-29T10:59:20.831 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:20.831 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": { 2023-03-29T10:59:20.831 INFO:teuthology.orchestra.run.smithi111.stdout: "type": "v1", 2023-03-29T10:59:20.831 INFO:teuthology.orchestra.run.smithi111.stdout: "addr": "192.168.0.1:0", 2023-03-29T10:59:20.831 INFO:teuthology.orchestra.run.smithi111.stdout: "nonce": 2722392590 2023-03-29T10:59:20.831 INFO:teuthology.orchestra.run.smithi111.stdout: }, 2023-03-29T10:59:20.831 INFO:teuthology.orchestra.run.smithi111.stdout: "inst_str": "client.5793 192.168.0.1:0/2722392590", 2023-03-29T10:59:20.831 INFO:teuthology.orchestra.run.smithi111.stdout: "addr_str": "192.168.0.1:0/2722392590", 2023-03-29T10:59:20.832 INFO:teuthology.orchestra.run.smithi111.stdout: "inode_count": 1, 2023-03-29T10:59:20.832 INFO:teuthology.orchestra.run.smithi111.stdout: "mds_epoch": 17, 2023-03-29T10:59:20.832 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch": 43, 2023-03-29T10:59:20.832 INFO:teuthology.orchestra.run.smithi111.stdout: "osd_epoch_barrier": 0, 2023-03-29T10:59:20.832 INFO:teuthology.orchestra.run.smithi111.stdout: "blocklisted": false, 2023-03-29T10:59:20.832 INFO:teuthology.orchestra.run.smithi111.stdout: "fs_name": "cephfs" 2023-03-29T10:59:20.832 INFO:teuthology.orchestra.run.smithi111.stdout:} 2023-03-29T10:59:20.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:59:20.842 DEBUG:teuthology.orchestra.run.smithi111:> stat --file-system '--printf=%T 2023-03-29T10:59:20.842 DEBUG:teuthology.orchestra.run.smithi111:> ' -- /home/ubuntu/cephtest/mnt.0 2023-03-29T10:59:20.883 INFO:teuthology.orchestra.run.smithi111.stdout:fuseblk 2023-03-29T10:59:20.883 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-03-29T10:59:20.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:59:20.883 DEBUG:teuthology.orchestra.run.smithi111:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-03-29T10:59:20.986 DEBUG:teuthology.orchestra.run.smithi111:> mktemp -d --suffix=xfstests-dev 2023-03-29T10:59:21.059 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.CzuMqE4LuJxfstests-dev 2023-03-29T10:59:21.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:59:21.059 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.CzuMqE4LuJxfstests-dev) 2023-03-29T10:59:21.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:59:21.124 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.CzuMqE4LuJxfstests-dev) 2023-03-29T10:59:21.171 INFO:teuthology.orchestra.run.smithi111.stderr:Cloning into '/tmp/tmp.CzuMqE4LuJxfstests-dev'... 2023-03-29T10:59:21.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:59:21.962 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2023-03-29T10:59:21.982 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T10:59:22.010 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T10:59:22.011 DEBUG:teuthology.orchestra.run.smithi111:> mktemp -d --suffix=test 2023-03-29T10:59:22.092 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.6dtVtxiLHFtest 2023-03-29T10:59:22.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T10:59:22.093 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2023-03-29T10:59:22.190 DEBUG:teuthology.orchestra.run.smithi111:> mktemp -d --suffix=scratch 2023-03-29T10:59:22.270 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.KcLS4LFr4Iscratch 2023-03-29T10:59:22.271 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "centos" and "8". 2023-03-29T10:59:22.271 DEBUG:teuthology.orchestra.run.smithi111:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc indent libtool lvm2 make psmisc quota sed xfsdump xfsprogs libacl-devel libattr-devel libaio-devel libuuid-devel xfsprogs-devel python2 sqlite xfsprogs-qa-devel 2023-03-29T10:59:23.256 INFO:teuthology.orchestra.run.smithi111.stdout:Last metadata expiration check: 0:05:06 ago on Wed 29 Mar 2023 10:54:17 AM UTC. 2023-03-29T10:59:24.378 INFO:teuthology.orchestra.run.smithi111.stdout:Package acl-2.2.53-1.el8.x86_64 is already installed. 2023-03-29T10:59:24.379 INFO:teuthology.orchestra.run.smithi111.stdout:Package attr-2.4.48-3.el8.x86_64 is already installed. 2023-03-29T10:59:24.380 INFO:teuthology.orchestra.run.smithi111.stdout:Package automake-1.16.1-7.el8.noarch is already installed. 2023-03-29T10:59:24.381 INFO:teuthology.orchestra.run.smithi111.stdout:Package bc-1.07.1-5.el8.x86_64 is already installed. 2023-03-29T10:59:24.382 INFO:teuthology.orchestra.run.smithi111.stdout:Package dbench-4.0-20.el8.x86_64 is already installed. 2023-03-29T10:59:24.384 INFO:teuthology.orchestra.run.smithi111.stdout:Package e2fsprogs-1.45.6-5.el8.x86_64 is already installed. 2023-03-29T10:59:24.386 INFO:teuthology.orchestra.run.smithi111.stdout:Package gawk-4.2.1-4.el8.x86_64 is already installed. 2023-03-29T10:59:24.387 INFO:teuthology.orchestra.run.smithi111.stdout:Package gcc-8.5.0-17.el8.x86_64 is already installed. 2023-03-29T10:59:24.389 INFO:teuthology.orchestra.run.smithi111.stdout:Package libtool-2.4.6-25.el8.x86_64 is already installed. 2023-03-29T10:59:24.390 INFO:teuthology.orchestra.run.smithi111.stdout:Package lvm2-8:2.03.14-6.el8.x86_64 is already installed. 2023-03-29T10:59:24.391 INFO:teuthology.orchestra.run.smithi111.stdout:Package make-1:4.2.1-11.el8.x86_64 is already installed. 2023-03-29T10:59:24.392 INFO:teuthology.orchestra.run.smithi111.stdout:Package psmisc-23.1-5.el8.x86_64 is already installed. 2023-03-29T10:59:24.393 INFO:teuthology.orchestra.run.smithi111.stdout:Package quota-1:4.04-14.el8.x86_64 is already installed. 2023-03-29T10:59:24.394 INFO:teuthology.orchestra.run.smithi111.stdout:Package sed-4.5-5.el8.x86_64 is already installed. 2023-03-29T10:59:24.395 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-03-29T10:59:24.396 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-03-29T10:59:24.396 INFO:teuthology.orchestra.run.smithi111.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-03-29T10:59:24.398 INFO:teuthology.orchestra.run.smithi111.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-03-29T10:59:24.399 INFO:teuthology.orchestra.run.smithi111.stdout:Package libaio-devel-0.3.112-1.el8.x86_64 is already installed. 2023-03-29T10:59:24.399 INFO:teuthology.orchestra.run.smithi111.stdout:Package libuuid-devel-2.32.1-41.el8.x86_64 is already installed. 2023-03-29T10:59:24.400 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsprogs-devel-5.0.0-10.el8.x86_64 is already installed. 2023-03-29T10:59:24.402 INFO:teuthology.orchestra.run.smithi111.stdout:Package sqlite-3.26.0-15.el8.x86_64 is already installed. 2023-03-29T10:59:24.406 INFO:teuthology.orchestra.run.smithi111.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-03-29T10:59:24.533 INFO:teuthology.orchestra.run.smithi111.stdout:Dependencies resolved. 2023-03-29T10:59:24.537 INFO:teuthology.orchestra.run.smithi111.stdout:============================================================================================== 2023-03-29T10:59:24.537 INFO:teuthology.orchestra.run.smithi111.stdout: Package Arch Version Repository Size 2023-03-29T10:59:24.537 INFO:teuthology.orchestra.run.smithi111.stdout:============================================================================================== 2023-03-29T10:59:24.537 INFO:teuthology.orchestra.run.smithi111.stdout:Installing: 2023-03-29T10:59:24.537 INFO:teuthology.orchestra.run.smithi111.stdout: dump x86_64 1:0.4-0.36.b46.el8 CentOS-BaseOS 164 k 2023-03-29T10:59:24.537 INFO:teuthology.orchestra.run.smithi111.stdout: fio x86_64 3.19-4.el8 CentOS-AppStream 538 k 2023-03-29T10:59:24.537 INFO:teuthology.orchestra.run.smithi111.stdout: indent x86_64 2.2.12-4.el8 epel 185 k 2023-03-29T10:59:24.537 INFO:teuthology.orchestra.run.smithi111.stdout: python2 x86_64 2.7.18-12.module_el8+299+aa6e9afa CentOS-AppStream 110 k 2023-03-29T10:59:24.537 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrading: 2023-03-29T10:59:24.537 INFO:teuthology.orchestra.run.smithi111.stdout: cpp x86_64 8.5.0-18.el8 CentOS-BaseOS 10 M 2023-03-29T10:59:24.537 INFO:teuthology.orchestra.run.smithi111.stdout: device-mapper x86_64 8:1.02.181-9.el8 CentOS-BaseOS 378 k 2023-03-29T10:59:24.537 INFO:teuthology.orchestra.run.smithi111.stdout: device-mapper-event x86_64 8:1.02.181-9.el8 CentOS-BaseOS 272 k 2023-03-29T10:59:24.538 INFO:teuthology.orchestra.run.smithi111.stdout: device-mapper-event-libs x86_64 8:1.02.181-9.el8 CentOS-BaseOS 271 k 2023-03-29T10:59:24.538 INFO:teuthology.orchestra.run.smithi111.stdout: device-mapper-libs x86_64 8:1.02.181-9.el8 CentOS-BaseOS 410 k 2023-03-29T10:59:24.538 INFO:teuthology.orchestra.run.smithi111.stdout: gcc x86_64 8.5.0-18.el8 CentOS-BaseOS 23 M 2023-03-29T10:59:24.538 INFO:teuthology.orchestra.run.smithi111.stdout: gcc-gdb-plugin x86_64 8.5.0-18.el8 CentOS-AppStream 120 k 2023-03-29T10:59:24.538 INFO:teuthology.orchestra.run.smithi111.stdout: gcc-plugin-annobin x86_64 8.5.0-18.el8 CentOS-BaseOS 35 k 2023-03-29T10:59:24.538 INFO:teuthology.orchestra.run.smithi111.stdout: libgcc x86_64 8.5.0-18.el8 CentOS-BaseOS 81 k 2023-03-29T10:59:24.538 INFO:teuthology.orchestra.run.smithi111.stdout: libgomp x86_64 8.5.0-18.el8 CentOS-BaseOS 208 k 2023-03-29T10:59:24.538 INFO:teuthology.orchestra.run.smithi111.stdout: lvm2 x86_64 8:2.03.14-9.el8 CentOS-BaseOS 1.7 M 2023-03-29T10:59:24.538 INFO:teuthology.orchestra.run.smithi111.stdout: lvm2-libs x86_64 8:2.03.14-9.el8 CentOS-BaseOS 1.2 M 2023-03-29T10:59:24.538 INFO:teuthology.orchestra.run.smithi111.stdout: sqlite x86_64 3.26.0-17.el8 CentOS-BaseOS 668 k 2023-03-29T10:59:24.538 INFO:teuthology.orchestra.run.smithi111.stdout: sqlite-libs x86_64 3.26.0-17.el8 CentOS-BaseOS 581 k 2023-03-29T10:59:24.538 INFO:teuthology.orchestra.run.smithi111.stdout:Installing dependencies: 2023-03-29T10:59:24.538 INFO:teuthology.orchestra.run.smithi111.stdout: daxctl-libs x86_64 71.1-4.el8 CentOS-BaseOS 42 k 2023-03-29T10:59:24.538 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemblk x86_64 1.6.1-1.el8 CentOS-AppStream 92 k 2023-03-29T10:59:24.538 INFO:teuthology.orchestra.run.smithi111.stdout: ndctl-libs x86_64 71.1-4.el8 CentOS-BaseOS 80 k 2023-03-29T10:59:24.538 INFO:teuthology.orchestra.run.smithi111.stdout: python2-libs x86_64 2.7.18-12.module_el8+299+aa6e9afa CentOS-AppStream 6.5 M 2023-03-29T10:59:24.538 INFO:teuthology.orchestra.run.smithi111.stdout: python2-pip-wheel noarch 9.0.3-19.module_el8+299+aa6e9afa CentOS-AppStream 892 k 2023-03-29T10:59:24.538 INFO:teuthology.orchestra.run.smithi111.stdout: python2-setuptools-wheel noarch 39.0.1-13.module_el8+299+aa6e9afa CentOS-AppStream 287 k 2023-03-29T10:59:24.539 INFO:teuthology.orchestra.run.smithi111.stdout: python2-wheel-wheel noarch 1:0.31.1-3.module_el8+299+aa6e9afa CentOS-AppStream 44 k 2023-03-29T10:59:24.539 INFO:teuthology.orchestra.run.smithi111.stdout: rmt x86_64 2:1.5.3-13.el8 CentOS-BaseOS 51 k 2023-03-29T10:59:24.539 INFO:teuthology.orchestra.run.smithi111.stdout:Installing weak dependencies: 2023-03-29T10:59:24.539 INFO:teuthology.orchestra.run.smithi111.stdout: python2-pip noarch 9.0.3-19.module_el8+299+aa6e9afa CentOS-AppStream 1.8 M 2023-03-29T10:59:24.539 INFO:teuthology.orchestra.run.smithi111.stdout: python2-setuptools noarch 39.0.1-13.module_el8+299+aa6e9afa CentOS-AppStream 702 k 2023-03-29T10:59:24.539 INFO:teuthology.orchestra.run.smithi111.stdout:Enabling module streams: 2023-03-29T10:59:24.539 INFO:teuthology.orchestra.run.smithi111.stdout: python27 2.7 2023-03-29T10:59:24.539 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:59:24.539 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction Summary 2023-03-29T10:59:24.539 INFO:teuthology.orchestra.run.smithi111.stdout:============================================================================================== 2023-03-29T10:59:24.539 INFO:teuthology.orchestra.run.smithi111.stdout:Install 14 Packages 2023-03-29T10:59:24.539 INFO:teuthology.orchestra.run.smithi111.stdout:Upgrade 14 Packages 2023-03-29T10:59:24.539 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:59:24.540 INFO:teuthology.orchestra.run.smithi111.stdout:Total download size: 51 M 2023-03-29T10:59:24.540 INFO:teuthology.orchestra.run.smithi111.stdout:Downloading Packages: 2023-03-29T10:59:24.671 INFO:teuthology.orchestra.run.smithi111.stdout:(1/28): libpmemblk-1.6.1-1.el8.x86_64.rpm 727 kB/s | 92 kB 00:00 2023-03-29T10:59:24.695 INFO:teuthology.orchestra.run.smithi111.stdout:(2/28): python2-2.7.18-12.module_el8+299+aa6e9a 730 kB/s | 110 kB 00:00 2023-03-29T10:59:24.738 INFO:teuthology.orchestra.run.smithi111.stdout:(3/28): fio-3.19-4.el8.x86_64.rpm 2.7 MB/s | 538 kB 00:00 2023-03-29T10:59:24.813 INFO:teuthology.orchestra.run.smithi111.stdout:(4/28): python2-pip-wheel-9.0.3-19.module_el8+2 12 MB/s | 892 kB 00:00 2023-03-29T10:59:24.888 INFO:teuthology.orchestra.run.smithi111.stdout:(5/28): python2-setuptools-39.0.1-13.module_el8 9.1 MB/s | 702 kB 00:00 2023-03-29T10:59:24.939 INFO:teuthology.orchestra.run.smithi111.stdout:(6/28): python2-setuptools-wheel-39.0.1-13.modu 5.6 MB/s | 287 kB 00:00 2023-03-29T10:59:25.022 INFO:teuthology.orchestra.run.smithi111.stdout:(7/28): python2-libs-2.7.18-12.module_el8+299+a 18 MB/s | 6.5 MB 00:00 2023-03-29T10:59:25.048 INFO:teuthology.orchestra.run.smithi111.stdout:(8/28): python2-wheel-wheel-0.31.1-3.module_el8 410 kB/s | 44 kB 00:00 2023-03-29T10:59:25.073 INFO:teuthology.orchestra.run.smithi111.stdout:(9/28): daxctl-libs-71.1-4.el8.x86_64.rpm 837 kB/s | 42 kB 00:00 2023-03-29T10:59:25.099 INFO:teuthology.orchestra.run.smithi111.stdout:(10/28): dump-0.4-0.36.b46.el8.x86_64.rpm 3.2 MB/s | 164 kB 00:00 2023-03-29T10:59:25.124 INFO:teuthology.orchestra.run.smithi111.stdout:(11/28): ndctl-libs-71.1-4.el8.x86_64.rpm 1.6 MB/s | 80 kB 00:00 2023-03-29T10:59:25.150 INFO:teuthology.orchestra.run.smithi111.stdout:(12/28): rmt-1.5.3-13.el8.x86_64.rpm 1.0 MB/s | 51 kB 00:00 2023-03-29T10:59:25.174 INFO:teuthology.orchestra.run.smithi111.stdout:(13/28): gcc-gdb-plugin-8.5.0-18.el8.x86_64.rpm 4.7 MB/s | 120 kB 00:00 2023-03-29T10:59:25.216 INFO:teuthology.orchestra.run.smithi111.stdout:(14/28): python2-pip-9.0.3-19.module_el8+299+aa 3.5 MB/s | 1.8 MB 00:00 2023-03-29T10:59:25.283 INFO:teuthology.orchestra.run.smithi111.stdout:(15/28): device-mapper-1.02.181-9.el8.x86_64.rp 5.5 MB/s | 378 kB 00:00 2023-03-29T10:59:25.375 INFO:teuthology.orchestra.run.smithi111.stdout:(16/28): device-mapper-event-1.02.181-9.el8.x86 2.9 MB/s | 272 kB 00:00 2023-03-29T10:59:25.401 INFO:teuthology.orchestra.run.smithi111.stdout:(17/28): indent-2.2.12-4.el8.x86_64.rpm 669 kB/s | 185 kB 00:00 2023-03-29T10:59:25.518 INFO:teuthology.orchestra.run.smithi111.stdout:(18/28): cpp-8.5.0-18.el8.x86_64.rpm 30 MB/s | 10 MB 00:00 2023-03-29T10:59:25.543 INFO:teuthology.orchestra.run.smithi111.stdout:(19/28): device-mapper-event-libs-1.02.181-9.el 1.6 MB/s | 271 kB 00:00 2023-03-29T10:59:25.577 INFO:teuthology.orchestra.run.smithi111.stdout:(20/28): device-mapper-libs-1.02.181-9.el8.x86_ 2.3 MB/s | 410 kB 00:00 2023-03-29T10:59:25.602 INFO:teuthology.orchestra.run.smithi111.stdout:(21/28): gcc-plugin-annobin-8.5.0-18.el8.x86_64 604 kB/s | 35 kB 00:00 2023-03-29T10:59:25.628 INFO:teuthology.orchestra.run.smithi111.stdout:(22/28): libgcc-8.5.0-18.el8.x86_64.rpm 1.6 MB/s | 81 kB 00:00 2023-03-29T10:59:25.653 INFO:teuthology.orchestra.run.smithi111.stdout:(23/28): libgomp-8.5.0-18.el8.x86_64.rpm 4.0 MB/s | 208 kB 00:00 2023-03-29T10:59:25.761 INFO:teuthology.orchestra.run.smithi111.stdout:(24/28): lvm2-libs-2.03.14-9.el8.x86_64.rpm 11 MB/s | 1.2 MB 00:00 2023-03-29T10:59:25.812 INFO:teuthology.orchestra.run.smithi111.stdout:(25/28): sqlite-3.26.0-17.el8.x86_64.rpm 13 MB/s | 668 kB 00:00 2023-03-29T10:59:25.870 INFO:teuthology.orchestra.run.smithi111.stdout:(26/28): sqlite-libs-3.26.0-17.el8.x86_64.rpm 9.8 MB/s | 581 kB 00:00 2023-03-29T10:59:26.012 INFO:teuthology.orchestra.run.smithi111.stdout:(27/28): lvm2-2.03.14-9.el8.x86_64.rpm 4.3 MB/s | 1.7 MB 00:00 2023-03-29T10:59:26.304 INFO:teuthology.orchestra.run.smithi111.stdout:(28/28): gcc-8.5.0-18.el8.x86_64.rpm 30 MB/s | 23 MB 00:00 2023-03-29T10:59:26.307 INFO:teuthology.orchestra.run.smithi111.stdout:-------------------------------------------------------------------------------- 2023-03-29T10:59:26.307 INFO:teuthology.orchestra.run.smithi111.stdout:Total 29 MB/s | 51 MB 00:01 2023-03-29T10:59:27.088 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction check 2023-03-29T10:59:27.222 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction check succeeded. 2023-03-29T10:59:27.222 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction test 2023-03-29T10:59:27.787 INFO:teuthology.orchestra.run.smithi111.stdout:Transaction test succeeded. 2023-03-29T10:59:27.792 INFO:teuthology.orchestra.run.smithi111.stdout:Running transaction 2023-03-29T10:59:29.246 INFO:teuthology.orchestra.run.smithi111.stdout: Preparing : 1/1 2023-03-29T10:59:29.313 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libgcc-8.5.0-18.el8.x86_64 1/1 2023-03-29T10:59:29.465 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : libgcc-8.5.0-18.el8.x86_64 1/42 2023-03-29T10:59:29.580 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libgcc-8.5.0-18.el8.x86_64 1/42 2023-03-29T10:59:29.850 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : device-mapper-libs-8:1.02.181-9.el8.x86_64 2/42 2023-03-29T10:59:30.105 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : device-mapper-8:1.02.181-9.el8.x86_64 3/42 2023-03-29T10:59:30.364 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : device-mapper-event-libs-8:1.02.181-9.el8.x86_64 4/42 2023-03-29T10:59:30.572 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : sqlite-libs-3.26.0-17.el8.x86_64 5/42 2023-03-29T10:59:30.723 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : daxctl-libs-71.1-4.el8.x86_64 6/42 2023-03-29T10:59:30.815 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: daxctl-libs-71.1-4.el8.x86_64 6/42 2023-03-29T10:59:30.999 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : ndctl-libs-71.1-4.el8.x86_64 7/42 2023-03-29T10:59:31.082 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: ndctl-libs-71.1-4.el8.x86_64 7/42 2023-03-29T10:59:31.248 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : libpmemblk-1.6.1-1.el8.x86_64 8/42 2023-03-29T10:59:31.329 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libpmemblk-1.6.1-1.el8.x86_64 8/42 2023-03-29T10:59:31.532 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : device-mapper-event-8:1.02.181-9.el8.x86_64 9/42 2023-03-29T10:59:31.874 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: device-mapper-event-8:1.02.181-9.el8.x86_64 9/42 2023-03-29T10:59:32.156 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : lvm2-libs-8:2.03.14-9.el8.x86_64 10/42 2023-03-29T10:59:32.382 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : libgomp-8.5.0-18.el8.x86_64 11/42 2023-03-29T10:59:33.373 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libgomp-8.5.0-18.el8.x86_64 11/42 2023-03-29T10:59:33.714 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : cpp-8.5.0-18.el8.x86_64 12/42 2023-03-29T10:59:35.523 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cpp-8.5.0-18.el8.x86_64 12/42 2023-03-29T10:59:35.839 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : gcc-8.5.0-18.el8.x86_64 13/42 2023-03-29T10:59:35.897 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: gcc-8.5.0-18.el8.x86_64 13/42 2023-03-29T10:59:36.087 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : rmt-2:1.5.3-13.el8.x86_64 14/42 2023-03-29T10:59:36.246 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python2-setuptools-wheel-39.0.1-13.module_el8+299+ 15/42 2023-03-29T10:59:37.749 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python2-pip-wheel-9.0.3-19.module_el8+299+aa6e9afa 16/42 2023-03-29T10:59:38.591 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python2-libs-2.7.18-12.module_el8+299+aa6e9afa.x86 17/42 2023-03-29T10:59:39.032 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python2-pip-9.0.3-19.module_el8+299+aa6e9afa.noarc 18/42 2023-03-29T10:59:39.279 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python2-setuptools-39.0.1-13.module_el8+299+aa6e9a 19/42 2023-03-29T10:59:39.486 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python2-2.7.18-12.module_el8+299+aa6e9afa.x86_64 20/42 2023-03-29T10:59:39.528 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: python2-2.7.18-12.module_el8+299+aa6e9afa.x86_64 20/42 2023-03-29T10:59:39.738 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : dump-1:0.4-0.36.b46.el8.x86_64 21/42 2023-03-29T10:59:39.955 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : gcc-gdb-plugin-8.5.0-18.el8.x86_64 22/42 2023-03-29T10:59:40.037 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: gcc-gdb-plugin-8.5.0-18.el8.x86_64 22/42 2023-03-29T10:59:40.206 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : gcc-plugin-annobin-8.5.0-18.el8.x86_64 23/42 2023-03-29T10:59:40.475 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: gcc-plugin-annobin-8.5.0-18.el8.x86_64 23/42 2023-03-29T10:59:40.475 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : lvm2-8:2.03.14-9.el8.x86_64 24/42 2023-03-29T10:59:40.475 INFO:teuthology.orchestra.run.smithi111.stdout:warning: /etc/lvm/lvm.conf created as /etc/lvm/lvm.conf.rpmnew 2023-03-29T10:59:40.475 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:59:41.112 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: lvm2-8:2.03.14-9.el8.x86_64 24/42 2023-03-29T10:59:41.432 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : fio-3.19-4.el8.x86_64 25/42 2023-03-29T10:59:41.732 INFO:teuthology.orchestra.run.smithi111.stdout: Upgrading : sqlite-3.26.0-17.el8.x86_64 26/42 2023-03-29T10:59:41.959 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : indent-2.2.12-4.el8.x86_64 27/42 2023-03-29T10:59:42.141 INFO:teuthology.orchestra.run.smithi111.stdout: Installing : python2-wheel-wheel-1:0.31.1-3.module_el8+299+aa6e 28/42 2023-03-29T10:59:42.160 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: lvm2-8:2.03.14-6.el8.x86_64 29/42 2023-03-29T10:59:42.161 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : lvm2-8:2.03.14-6.el8.x86_64 29/42 2023-03-29T10:59:42.425 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: lvm2-8:2.03.14-6.el8.x86_64 29/42 2023-03-29T10:59:42.648 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : lvm2-libs-8:2.03.14-6.el8.x86_64 30/42 2023-03-29T10:59:42.661 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: device-mapper-event-8:1.02.181-6.el8.x86_64 31/42 2023-03-29T10:59:42.857 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : device-mapper-event-8:1.02.181-6.el8.x86_64 31/42 2023-03-29T10:59:43.082 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : device-mapper-event-libs-8:1.02.181-6.el8.x86_64 32/42 2023-03-29T10:59:43.082 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : gcc-gdb-plugin-8.5.0-17.el8.x86_64 33/42 2023-03-29T10:59:43.349 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: gcc-gdb-plugin-8.5.0-17.el8.x86_64 33/42 2023-03-29T10:59:43.489 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : gcc-plugin-annobin-8.5.0-17.el8.x86_64 34/42 2023-03-29T10:59:43.511 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: gcc-8.5.0-17.el8.x86_64 35/42 2023-03-29T10:59:43.807 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : gcc-8.5.0-17.el8.x86_64 35/42 2023-03-29T10:59:44.040 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : device-mapper-libs-8:1.02.181-6.el8.x86_64 36/42 2023-03-29T10:59:44.239 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : device-mapper-8:1.02.181-6.el8.x86_64 37/42 2023-03-29T10:59:44.414 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : sqlite-3.26.0-15.el8.x86_64 38/42 2023-03-29T10:59:44.580 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : sqlite-libs-3.26.0-15.el8.x86_64 39/42 2023-03-29T10:59:44.595 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: cpp-8.5.0-17.el8.x86_64 40/42 2023-03-29T10:59:44.748 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : cpp-8.5.0-17.el8.x86_64 40/42 2023-03-29T10:59:44.748 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : libgcc-8.5.0-17.el8.x86_64 41/42 2023-03-29T10:59:44.973 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libgcc-8.5.0-17.el8.x86_64 41/42 2023-03-29T10:59:44.986 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libgomp-8.5.0-17.el8.x86_64 42/42 2023-03-29T10:59:44.986 INFO:teuthology.orchestra.run.smithi111.stdout: Cleanup : libgomp-8.5.0-17.el8.x86_64 42/42 2023-03-29T10:59:46.186 INFO:teuthology.orchestra.run.smithi111.stdout: Running scriptlet: libgomp-8.5.0-17.el8.x86_64 42/42 2023-03-29T10:59:46.186 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : fio-3.19-4.el8.x86_64 1/42 2023-03-29T10:59:46.186 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libpmemblk-1.6.1-1.el8.x86_64 2/42 2023-03-29T10:59:46.186 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python2-2.7.18-12.module_el8+299+aa6e9afa.x86_64 3/42 2023-03-29T10:59:46.186 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python2-libs-2.7.18-12.module_el8+299+aa6e9afa.x86 4/42 2023-03-29T10:59:46.186 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python2-pip-9.0.3-19.module_el8+299+aa6e9afa.noarc 5/42 2023-03-29T10:59:46.186 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python2-pip-wheel-9.0.3-19.module_el8+299+aa6e9afa 6/42 2023-03-29T10:59:46.186 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python2-setuptools-39.0.1-13.module_el8+299+aa6e9a 7/42 2023-03-29T10:59:46.186 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python2-setuptools-wheel-39.0.1-13.module_el8+299+ 8/42 2023-03-29T10:59:46.186 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : python2-wheel-wheel-1:0.31.1-3.module_el8+299+aa6e 9/42 2023-03-29T10:59:46.187 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : daxctl-libs-71.1-4.el8.x86_64 10/42 2023-03-29T10:59:46.187 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : dump-1:0.4-0.36.b46.el8.x86_64 11/42 2023-03-29T10:59:46.187 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : ndctl-libs-71.1-4.el8.x86_64 12/42 2023-03-29T10:59:46.187 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : rmt-2:1.5.3-13.el8.x86_64 13/42 2023-03-29T10:59:46.187 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : indent-2.2.12-4.el8.x86_64 14/42 2023-03-29T10:59:46.187 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gcc-gdb-plugin-8.5.0-18.el8.x86_64 15/42 2023-03-29T10:59:46.187 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gcc-gdb-plugin-8.5.0-17.el8.x86_64 16/42 2023-03-29T10:59:46.187 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cpp-8.5.0-18.el8.x86_64 17/42 2023-03-29T10:59:46.187 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : cpp-8.5.0-17.el8.x86_64 18/42 2023-03-29T10:59:46.187 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : device-mapper-8:1.02.181-9.el8.x86_64 19/42 2023-03-29T10:59:46.187 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : device-mapper-8:1.02.181-6.el8.x86_64 20/42 2023-03-29T10:59:46.187 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : device-mapper-event-8:1.02.181-9.el8.x86_64 21/42 2023-03-29T10:59:46.187 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : device-mapper-event-8:1.02.181-6.el8.x86_64 22/42 2023-03-29T10:59:46.187 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : device-mapper-event-libs-8:1.02.181-9.el8.x86_64 23/42 2023-03-29T10:59:46.187 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : device-mapper-event-libs-8:1.02.181-6.el8.x86_64 24/42 2023-03-29T10:59:46.187 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : device-mapper-libs-8:1.02.181-9.el8.x86_64 25/42 2023-03-29T10:59:46.187 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : device-mapper-libs-8:1.02.181-6.el8.x86_64 26/42 2023-03-29T10:59:46.187 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gcc-8.5.0-18.el8.x86_64 27/42 2023-03-29T10:59:46.188 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gcc-8.5.0-17.el8.x86_64 28/42 2023-03-29T10:59:46.188 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gcc-plugin-annobin-8.5.0-18.el8.x86_64 29/42 2023-03-29T10:59:46.188 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : gcc-plugin-annobin-8.5.0-17.el8.x86_64 30/42 2023-03-29T10:59:46.189 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libgcc-8.5.0-18.el8.x86_64 31/42 2023-03-29T10:59:46.189 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libgcc-8.5.0-17.el8.x86_64 32/42 2023-03-29T10:59:46.189 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libgomp-8.5.0-18.el8.x86_64 33/42 2023-03-29T10:59:46.189 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : libgomp-8.5.0-17.el8.x86_64 34/42 2023-03-29T10:59:46.190 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lvm2-8:2.03.14-9.el8.x86_64 35/42 2023-03-29T10:59:46.190 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lvm2-8:2.03.14-6.el8.x86_64 36/42 2023-03-29T10:59:46.190 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lvm2-libs-8:2.03.14-9.el8.x86_64 37/42 2023-03-29T10:59:46.190 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : lvm2-libs-8:2.03.14-6.el8.x86_64 38/42 2023-03-29T10:59:46.190 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : sqlite-3.26.0-17.el8.x86_64 39/42 2023-03-29T10:59:46.190 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : sqlite-3.26.0-15.el8.x86_64 40/42 2023-03-29T10:59:46.190 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : sqlite-libs-3.26.0-17.el8.x86_64 41/42 2023-03-29T10:59:47.127 INFO:teuthology.orchestra.run.smithi111.stdout: Verifying : sqlite-libs-3.26.0-15.el8.x86_64 42/42 2023-03-29T10:59:47.127 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:59:47.127 INFO:teuthology.orchestra.run.smithi111.stdout:Upgraded: 2023-03-29T10:59:47.127 INFO:teuthology.orchestra.run.smithi111.stdout: cpp-8.5.0-18.el8.x86_64 2023-03-29T10:59:47.127 INFO:teuthology.orchestra.run.smithi111.stdout: device-mapper-8:1.02.181-9.el8.x86_64 2023-03-29T10:59:47.127 INFO:teuthology.orchestra.run.smithi111.stdout: device-mapper-event-8:1.02.181-9.el8.x86_64 2023-03-29T10:59:47.127 INFO:teuthology.orchestra.run.smithi111.stdout: device-mapper-event-libs-8:1.02.181-9.el8.x86_64 2023-03-29T10:59:47.128 INFO:teuthology.orchestra.run.smithi111.stdout: device-mapper-libs-8:1.02.181-9.el8.x86_64 2023-03-29T10:59:47.128 INFO:teuthology.orchestra.run.smithi111.stdout: gcc-8.5.0-18.el8.x86_64 2023-03-29T10:59:47.128 INFO:teuthology.orchestra.run.smithi111.stdout: gcc-gdb-plugin-8.5.0-18.el8.x86_64 2023-03-29T10:59:47.128 INFO:teuthology.orchestra.run.smithi111.stdout: gcc-plugin-annobin-8.5.0-18.el8.x86_64 2023-03-29T10:59:47.128 INFO:teuthology.orchestra.run.smithi111.stdout: libgcc-8.5.0-18.el8.x86_64 2023-03-29T10:59:47.128 INFO:teuthology.orchestra.run.smithi111.stdout: libgomp-8.5.0-18.el8.x86_64 2023-03-29T10:59:47.128 INFO:teuthology.orchestra.run.smithi111.stdout: lvm2-8:2.03.14-9.el8.x86_64 2023-03-29T10:59:47.128 INFO:teuthology.orchestra.run.smithi111.stdout: lvm2-libs-8:2.03.14-9.el8.x86_64 2023-03-29T10:59:47.128 INFO:teuthology.orchestra.run.smithi111.stdout: sqlite-3.26.0-17.el8.x86_64 2023-03-29T10:59:47.129 INFO:teuthology.orchestra.run.smithi111.stdout: sqlite-libs-3.26.0-17.el8.x86_64 2023-03-29T10:59:47.129 INFO:teuthology.orchestra.run.smithi111.stdout:Installed: 2023-03-29T10:59:47.129 INFO:teuthology.orchestra.run.smithi111.stdout: daxctl-libs-71.1-4.el8.x86_64 2023-03-29T10:59:47.129 INFO:teuthology.orchestra.run.smithi111.stdout: dump-1:0.4-0.36.b46.el8.x86_64 2023-03-29T10:59:47.129 INFO:teuthology.orchestra.run.smithi111.stdout: fio-3.19-4.el8.x86_64 2023-03-29T10:59:47.129 INFO:teuthology.orchestra.run.smithi111.stdout: indent-2.2.12-4.el8.x86_64 2023-03-29T10:59:47.129 INFO:teuthology.orchestra.run.smithi111.stdout: libpmemblk-1.6.1-1.el8.x86_64 2023-03-29T10:59:47.129 INFO:teuthology.orchestra.run.smithi111.stdout: ndctl-libs-71.1-4.el8.x86_64 2023-03-29T10:59:47.129 INFO:teuthology.orchestra.run.smithi111.stdout: python2-2.7.18-12.module_el8+299+aa6e9afa.x86_64 2023-03-29T10:59:47.129 INFO:teuthology.orchestra.run.smithi111.stdout: python2-libs-2.7.18-12.module_el8+299+aa6e9afa.x86_64 2023-03-29T10:59:47.130 INFO:teuthology.orchestra.run.smithi111.stdout: python2-pip-9.0.3-19.module_el8+299+aa6e9afa.noarch 2023-03-29T10:59:47.130 INFO:teuthology.orchestra.run.smithi111.stdout: python2-pip-wheel-9.0.3-19.module_el8+299+aa6e9afa.noarch 2023-03-29T10:59:47.130 INFO:teuthology.orchestra.run.smithi111.stdout: python2-setuptools-39.0.1-13.module_el8+299+aa6e9afa.noarch 2023-03-29T10:59:47.130 INFO:teuthology.orchestra.run.smithi111.stdout: python2-setuptools-wheel-39.0.1-13.module_el8+299+aa6e9afa.noarch 2023-03-29T10:59:47.130 INFO:teuthology.orchestra.run.smithi111.stdout: python2-wheel-wheel-1:0.31.1-3.module_el8+299+aa6e9afa.noarch 2023-03-29T10:59:47.130 INFO:teuthology.orchestra.run.smithi111.stdout: rmt-2:1.5.3-13.el8.x86_64 2023-03-29T10:59:47.130 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:59:47.130 INFO:teuthology.orchestra.run.smithi111.stdout:Complete! 2023-03-29T10:59:47.428 DEBUG:teuthology.orchestra.run.smithi111:> sudo useradd -m fsgqa 2023-03-29T10:59:47.492 INFO:teuthology.orchestra.run.smithi111.stderr:useradd: user 'fsgqa' already exists 2023-03-29T10:59:47.495 DEBUG:teuthology.orchestra.run:got remote process result: 9 2023-03-29T10:59:47.495 DEBUG:teuthology.orchestra.run.smithi111:> sudo groupadd fsgqa 2023-03-29T10:59:47.555 INFO:teuthology.orchestra.run.smithi111.stderr:groupadd: group 'fsgqa' already exists 2023-03-29T10:59:47.557 DEBUG:teuthology.orchestra.run:got remote process result: 9 2023-03-29T10:59:47.557 DEBUG:teuthology.orchestra.run.smithi111:> sudo useradd fsgqa2 2023-03-29T10:59:48.901 DEBUG:teuthology.orchestra.run.smithi111:> sudo useradd 123456-fsgqa 2023-03-29T10:59:52.112 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T10:59:52.116 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T10:59:52.743 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2023-03-29T10:59:52.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.929+0000 7f623c7ff700 1 Processor -- start 2023-03-29T10:59:52.929 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.929+0000 7f623c7ff700 1 -- start start 2023-03-29T10:59:52.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.930+0000 7f623c7ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f62340af620 0x7f62340ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:52.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.930+0000 7f623c7ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f623406adf0 con 0x7f623415fea0 2023-03-29T10:59:52.932 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.930+0000 7f623c7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f623406b320 con 0x7f62340cec10 2023-03-29T10:59:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.930+0000 7f623c7ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62340cd5d0 con 0x7f62340af620 2023-03-29T10:59:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.931+0000 7f623a59b700 1 --1- >> v1:172.21.15.111:6789/0 conn(0x7f623415fea0 0x7f62340ccec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.111:6789/0 says I am v1:172.21.15.111:57784/0 (socket says 172.21.15.111:57784) 2023-03-29T10:59:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.931+0000 7f623a59b700 1 -- 172.21.15.111:0/3499443636 learned_addr learned my addr 172.21.15.111:0/3499443636 (peer_addr_for_me v1:172.21.15.111:0/0) 2023-03-29T10:59:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.931+0000 7f6239d9a700 1 --2- 172.21.15.111:0/3499443636 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f62340af620 0x7f62340ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.931+0000 7f6239599700 1 -- 172.21.15.111:0/3499443636 <== mon.0 v1:172.21.15.111:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964849499 0 0) 0x7f623406adf0 con 0x7f623415fea0 2023-03-29T10:59:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.931+0000 7f6239599700 1 -- 172.21.15.111:0/3499443636 --> v1:172.21.15.111:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62200035b0 con 0x7f623415fea0 2023-03-29T10:59:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.931+0000 7f6239599700 1 -- 172.21.15.111:0/3499443636 <== mon.1 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626949511 0 0) 0x7f623406b320 con 0x7f62340cec10 2023-03-29T10:59:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.931+0000 7f6239599700 1 -- 172.21.15.111:0/3499443636 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f623406adf0 con 0x7f62340cec10 2023-03-29T10:59:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.931+0000 7f6239599700 1 -- 172.21.15.111:0/3499443636 <== mon.0 v1:172.21.15.111:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1041792981 0 0) 0x7f62200035b0 con 0x7f623415fea0 2023-03-29T10:59:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.931+0000 7f6239599700 1 -- 172.21.15.111:0/3499443636 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f62340af620 msgr2=0x7f62340ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.931+0000 7f6239599700 1 --2- 172.21.15.111:0/3499443636 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f62340af620 0x7f62340ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.931+0000 7f6239599700 1 -- 172.21.15.111:0/3499443636 >> v1:172.21.15.119:6789/0 conn(0x7f62340cec10 legacy=0x7f62340b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.931+0000 7f6239599700 1 -- 172.21.15.111:0/3499443636 --> v1:172.21.15.111:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623406b040 con 0x7f623415fea0 2023-03-29T10:59:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.931+0000 7f6239599700 1 -- 172.21.15.111:0/3499443636 <== mon.0 v1:172.21.15.111:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1936884954 0 0) 0x7f6230001670 con 0x7f623415fea0 2023-03-29T10:59:52.933 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.932+0000 7f6239599700 1 -- 172.21.15.111:0/3499443636 >> v1:172.21.15.111:6789/0 conn(0x7f623415fea0 legacy=0x7f62340ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:52.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.932+0000 7f6239599700 1 --2- 172.21.15.111:0/3499443636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6220003c30 0x7f6220004020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:52.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.932+0000 7f6239599700 1 --2- 172.21.15.111:0/3499443636 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f62340af620 0x7f6220004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:52.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.932+0000 7f6239599700 1 --2- 172.21.15.111:0/3499443636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6220004d90 0x7f6220005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:52.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.932+0000 7f6239599700 1 -- 172.21.15.111:0/3499443636 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62340cd5d0 con 0x7f6220004d90 2023-03-29T10:59:52.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.932+0000 7f6239599700 1 -- 172.21.15.111:0/3499443636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622000bb20 con 0x7f6220003c30 2023-03-29T10:59:52.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.932+0000 7f6239599700 1 -- 172.21.15.111:0/3499443636 --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622000bce0 con 0x7f62340af620 2023-03-29T10:59:52.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.932+0000 7f623ad9c700 1 --2- 172.21.15.111:0/3499443636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6220004d90 0x7f6220005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:52.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.932+0000 7f623a59b700 1 --2- 172.21.15.111:0/3499443636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6220003c30 0x7f6220004020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:52.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.932+0000 7f6239d9a700 1 --2- 172.21.15.111:0/3499443636 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f62340af620 0x7f6220004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:52.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.932+0000 7f623ad9c700 1 -- 172.21.15.111:0/3499443636 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f62340af620 msgr2=0x7f6220004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:52.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.932+0000 7f623ad9c700 1 --2- 172.21.15.111:0/3499443636 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f62340af620 0x7f6220004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:52.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.932+0000 7f623ad9c700 1 -- 172.21.15.111:0/3499443636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6220003c30 msgr2=0x7f6220004020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-03-29T10:59:52.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.932+0000 7f623ad9c700 1 --2- 172.21.15.111:0/3499443636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6220003c30 0x7f6220004020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:52.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.932+0000 7f623ad9c700 1 -- 172.21.15.111:0/3499443636 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62200035b0 con 0x7f6220004d90 2023-03-29T10:59:52.934 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.932+0000 7f6239d9a700 1 --2- 172.21.15.111:0/3499443636 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f62340af620 0x7f6220004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:59:52.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.933+0000 7f623ad9c700 1 --2- 172.21.15.111:0/3499443636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6220004d90 0x7f6220005200 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f622c00c2f0 tx=0x7f6220005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d4e2427d54cd37 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:52.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.933+0000 7f6239599700 1 -- 172.21.15.111:0/3499443636 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c001f30 con 0x7f6220004d90 2023-03-29T10:59:52.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.933+0000 7f6239599700 1 -- 172.21.15.111:0/3499443636 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c00ce90 con 0x7f6220004d90 2023-03-29T10:59:52.935 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.933+0000 7f6239599700 1 -- 172.21.15.111:0/3499443636 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c003c40 con 0x7f6220004d90 2023-03-29T10:59:52.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.933+0000 7f623c7ff700 1 -- 172.21.15.111:0/3499443636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6220004d90 msgr2=0x7f6220005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:52.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.933+0000 7f623c7ff700 1 --2- 172.21.15.111:0/3499443636 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6220004d90 0x7f6220005200 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f622c00c2f0 tx=0x7f6220005880 comp rx=0 tx=0).stop 2023-03-29T10:59:52.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.933+0000 7f623ad9c700 1 -- 172.21.15.111:0/3499443636 reap_dead start 2023-03-29T10:59:52.937 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.933+0000 7f623c7ff700 1 -- 172.21.15.111:0/3499443636 shutdown_connections 2023-03-29T10:59:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.933+0000 7f623c7ff700 1 -- 172.21.15.111:0/3499443636 >> 172.21.15.111:0/3499443636 conn(0x7f6234099050 msgr2=0x7f62340cfcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.933+0000 7f623c7ff700 1 -- 172.21.15.111:0/3499443636 shutdown_connections 2023-03-29T10:59:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.933+0000 7f623c7ff700 1 -- 172.21.15.111:0/3499443636 wait complete. 2023-03-29T10:59:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.935+0000 7f623c7ff700 1 Processor -- start 2023-03-29T10:59:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.935+0000 7f623c7ff700 1 -- start start 2023-03-29T10:59:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.936+0000 7f623c7ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6220004d90 0x7f62341f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.936+0000 7f623c7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6234156900 0x7f6234156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.936+0000 7f623c7ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6234145520 0x7f6234145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.936+0000 7f623c7ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6234145100 con 0x7f6234145520 2023-03-29T10:59:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.936+0000 7f623c7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6234157660 con 0x7f6234156900 2023-03-29T10:59:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.936+0000 7f623c7ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6234144a40 con 0x7f6220004d90 2023-03-29T10:59:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.936+0000 7f623ad9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6234145520 0x7f6234145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.936+0000 7f623ad9c700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6234145520 0x7f6234145990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38154/0 (socket says 172.21.15.111:38154) 2023-03-29T10:59:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.936+0000 7f623ad9c700 1 -- 172.21.15.111:0/3804224406 learned_addr learned my addr 172.21.15.111:0/3804224406 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.937+0000 7f6239d9a700 1 --2- 172.21.15.111:0/3804224406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6234156900 0x7f6234156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.937+0000 7f623a59b700 1 --2- 172.21.15.111:0/3804224406 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6220004d90 0x7f62341f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:52.938 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.937+0000 7f623ad9c700 1 -- 172.21.15.111:0/3804224406 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6220004d90 msgr2=0x7f62341f2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:52.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.937+0000 7f623ad9c700 1 --2- 172.21.15.111:0/3804224406 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6220004d90 0x7f62341f2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:52.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.937+0000 7f623ad9c700 1 -- 172.21.15.111:0/3804224406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6234156900 msgr2=0x7f6234156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:52.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.937+0000 7f623ad9c700 1 --2- 172.21.15.111:0/3804224406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6234156900 0x7f6234156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:52.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.937+0000 7f623ad9c700 1 -- 172.21.15.111:0/3804224406 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62340cc9a0 con 0x7f6234145520 2023-03-29T10:59:52.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.937+0000 7f6239d9a700 1 --2- 172.21.15.111:0/3804224406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6234156900 0x7f6234156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-03-29T10:59:52.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.937+0000 7f623ad9c700 1 --2- 172.21.15.111:0/3804224406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6234145520 0x7f6234145990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f622c004000 tx=0x7f622c001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:52.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.937+0000 7f622affd700 1 -- 172.21.15.111:0/3804224406 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c00cc90 con 0x7f6234145520 2023-03-29T10:59:52.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.937+0000 7f622affd700 1 -- 172.21.15.111:0/3804224406 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c0050d0 con 0x7f6234145520 2023-03-29T10:59:52.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.937+0000 7f623c7ff700 1 -- 172.21.15.111:0/3804224406 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62340cc6c0 con 0x7f6234145520 2023-03-29T10:59:52.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.937+0000 7f622affd700 1 -- 172.21.15.111:0/3804224406 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c005480 con 0x7f6234145520 2023-03-29T10:59:52.939 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.938+0000 7f623c7ff700 1 -- 172.21.15.111:0/3804224406 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62340afe90 con 0x7f6234145520 2023-03-29T10:59:52.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.939+0000 7f623c7ff700 1 -- 172.21.15.111:0/3804224406 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6234074430 con 0x7f6234145520 2023-03-29T10:59:52.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.940+0000 7f622affd700 1 -- 172.21.15.111:0/3804224406 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7f622c019070 con 0x7f6234145520 2023-03-29T10:59:52.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.940+0000 7f622affd700 1 --2- 172.21.15.111:0/3804224406 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6218069de0 0x7f621806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:52.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.941+0000 7f622affd700 1 -- 172.21.15.111:0/3804224406 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f622c088ca0 con 0x7f6234145520 2023-03-29T10:59:52.940 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.941+0000 7f623a59b700 1 --2- 172.21.15.111:0/3804224406 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6218069de0 0x7f621806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:52.941 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.941+0000 7f623a59b700 1 --2- 172.21.15.111:0/3804224406 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6218069de0 0x7f621806c290 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6230001eb0 tx=0x7f6230009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:52.944 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:52.945+0000 7f622affd700 1 -- 172.21.15.111:0/3804224406 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7f622c054400 con 0x7f6234145520 2023-03-29T10:59:53.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.216+0000 7f623c7ff700 1 -- 172.21.15.111:0/3804224406 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f62340c6770 con 0x7f6234145520 2023-03-29T10:59:53.216 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.217+0000 7f622affd700 1 -- 172.21.15.111:0/3804224406 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+1280 (secure 0 0 0) 0x7f622c02c510 con 0x7f6234145520 2023-03-29T10:59:53.217 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T10:59:53.217 INFO:teuthology.orchestra.run.smithi111.stdout:{"epoch":1,"fsid":"c8526f71-9b58-4b2e-978f-708199ec8aef","modified":"2023-03-29T10:56:23.727486Z","created":"2023-03-29T10:56:23.727486Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.111:3300","nonce":0},{"type":"v1","addr":"172.21.15.111:6789","nonce":0}]},"addr":"172.21.15.111:6789/0","public_addr":"172.21.15.111:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3300","nonce":0},{"type":"v1","addr":"172.21.15.119:6789","nonce":0}]},"addr":"172.21.15.119:6789/0","public_addr":"172.21.15.119:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:3301","nonce":0},{"type":"v1","addr":"172.21.15.119:6790","nonce":0}]},"addr":"172.21.15.119:6790/0","public_addr":"172.21.15.119:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-03-29T10:59:53.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.219+0000 7f623c7ff700 1 -- 172.21.15.111:0/3804224406 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6218069de0 msgr2=0x7f621806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:53.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.219+0000 7f623c7ff700 1 --2- 172.21.15.111:0/3804224406 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6218069de0 0x7f621806c290 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6230001eb0 tx=0x7f6230009040 comp rx=0 tx=0).stop 2023-03-29T10:59:53.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.219+0000 7f623c7ff700 1 -- 172.21.15.111:0/3804224406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6234145520 msgr2=0x7f6234145990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:53.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.219+0000 7f623c7ff700 1 --2- 172.21.15.111:0/3804224406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6234145520 0x7f6234145990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f622c004000 tx=0x7f622c001650 comp rx=0 tx=0).stop 2023-03-29T10:59:53.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.219+0000 7f623c7ff700 1 -- 172.21.15.111:0/3804224406 shutdown_connections 2023-03-29T10:59:53.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.219+0000 7f623c7ff700 1 --2- 172.21.15.111:0/3804224406 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7f6218069de0 0x7f621806c290 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:53.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.219+0000 7f623c7ff700 1 --2- 172.21.15.111:0/3804224406 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7f6234145520 0x7f6234145990 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:53.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.219+0000 7f623c7ff700 1 --2- 172.21.15.111:0/3804224406 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7f6220004d90 0x7f62341f2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:53.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.219+0000 7f623c7ff700 1 --2- 172.21.15.111:0/3804224406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6234156900 0x7f6234156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:53.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.219+0000 7f623c7ff700 1 -- 172.21.15.111:0/3804224406 >> 172.21.15.111:0/3804224406 conn(0x7f6234099050 msgr2=0x7f62340ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:53.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.219+0000 7f623c7ff700 1 -- 172.21.15.111:0/3804224406 shutdown_connections 2023-03-29T10:59:53.219 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.220+0000 7f623c7ff700 1 -- 172.21.15.111:0/3804224406 wait complete. 2023-03-29T10:59:53.220 INFO:teuthology.orchestra.run.smithi111.stderr:dumped monmap epoch 1 2023-03-29T10:59:53.237 DEBUG:teuthology.orchestra.run.smithi111:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.admin 2023-03-29T10:59:53.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.425+0000 7fbbd69ff700 1 Processor -- start 2023-03-29T10:59:53.424 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.425+0000 7fbbd69ff700 1 -- start start 2023-03-29T10:59:53.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.426+0000 7fbbd69ff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbbd00cd790 0x7fbbd00ca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:53.425 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.426+0000 7fbbd69ff700 1 -- --> v1:172.21.15.111:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd006a5f0 con 0x7fbbd00cdb60 2023-03-29T10:59:53.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.426+0000 7fbbd69ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbd006a840 con 0x7fbbd00b1d50 2023-03-29T10:59:53.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.426+0000 7fbbd69ff700 1 -- --> v2:172.21.15.111:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbd006ab20 con 0x7fbbd00cd790 2023-03-29T10:59:53.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.426+0000 7fbbcffff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbbd00cd790 0x7fbbd00ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:53.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.426+0000 7fbbcffff700 1 --2- >> v2:172.21.15.111:3300/0 conn(0x7fbbd00cd790 0x7fbbd00ca210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38158/0 (socket says 172.21.15.111:38158) 2023-03-29T10:59:53.426 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.426+0000 7fbbcffff700 1 -- 172.21.15.111:0/3816154442 learned_addr learned my addr 172.21.15.111:0/3816154442 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:53.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.426+0000 7fbbcffff700 1 -- 172.21.15.111:0/3816154442 >> v1:172.21.15.119:6789/0 conn(0x7fbbd00b1d50 legacy=0x7fbbd006b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:53.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.426+0000 7fbbcffff700 1 -- 172.21.15.111:0/3816154442 >> v1:172.21.15.111:6789/0 conn(0x7fbbd00cdb60 legacy=0x7fbbd00b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:53.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.427+0000 7fbbcffff700 1 -- 172.21.15.111:0/3816154442 --> v2:172.21.15.111:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd006ae00 con 0x7fbbd00cd790 2023-03-29T10:59:53.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.427+0000 7fbbcffff700 1 --2- 172.21.15.111:0/3816154442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbd00cd790 0x7fbbd00ca210 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009920 tx=0x7fbbc0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe4130a8daea225 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:53.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.427+0000 7fbbcf7fe700 1 -- 172.21.15.111:0/3816154442 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc000ba20 con 0x7fbbd00cd790 2023-03-29T10:59:53.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.427+0000 7fbbcf7fe700 1 -- 172.21.15.111:0/3816154442 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc000bb80 con 0x7fbbd00cd790 2023-03-29T10:59:53.427 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.427+0000 7fbbcf7fe700 1 -- 172.21.15.111:0/3816154442 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0009e70 con 0x7fbbd00cd790 2023-03-29T10:59:53.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.427+0000 7fbbd69ff700 1 -- 172.21.15.111:0/3816154442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbd00cd790 msgr2=0x7fbbd00ca210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:53.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.427+0000 7fbbd69ff700 1 --2- 172.21.15.111:0/3816154442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbd00cd790 0x7fbbd00ca210 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009920 tx=0x7fbbc0009c30 comp rx=0 tx=0).stop 2023-03-29T10:59:53.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.428+0000 7fbbd69ff700 1 -- 172.21.15.111:0/3816154442 shutdown_connections 2023-03-29T10:59:53.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.428+0000 7fbbd69ff700 1 --2- 172.21.15.111:0/3816154442 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbd00cd790 0x7fbbd00ca210 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:53.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.428+0000 7fbbd69ff700 1 -- 172.21.15.111:0/3816154442 >> 172.21.15.111:0/3816154442 conn(0x7fbbd006e980 msgr2=0x7fbbd00cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:53.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.428+0000 7fbbd69ff700 1 -- 172.21.15.111:0/3816154442 shutdown_connections 2023-03-29T10:59:53.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.428+0000 7fbbd69ff700 1 -- 172.21.15.111:0/3816154442 wait complete. 2023-03-29T10:59:53.428 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.429+0000 7fbbd69ff700 1 Processor -- start 2023-03-29T10:59:53.429 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.429+0000 7fbbd69ff700 1 -- start start 2023-03-29T10:59:53.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.430+0000 7fbbd69ff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbd00b1d50 0x7fbbd01509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:53.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.430+0000 7fbbd69ff700 1 --2- >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbbd00cd790 0x7fbbd00c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:53.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.430+0000 7fbbd69ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbd00cdb60 0x7fbbd00b29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:53.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.430+0000 7fbbd69ff700 1 -- --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd015f220 con 0x7fbbd00b1d50 2023-03-29T10:59:53.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.430+0000 7fbbd69ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd015fcd0 con 0x7fbbd00cdb60 2023-03-29T10:59:53.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.430+0000 7fbbd69ff700 1 -- --> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd015f430 con 0x7fbbd00cd790 2023-03-29T10:59:53.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.430+0000 7fbbcffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbd00b1d50 0x7fbbd01509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:53.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.430+0000 7fbbcffff700 1 --2- >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbd00b1d50 0x7fbbd01509b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.111:3300/0 says I am v2:172.21.15.111:38168/0 (socket says 172.21.15.111:38168) 2023-03-29T10:59:53.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.430+0000 7fbbcffff700 1 -- 172.21.15.111:0/1296504795 learned_addr learned my addr 172.21.15.111:0/1296504795 (peer_addr_for_me v2:172.21.15.111:0/0) 2023-03-29T10:59:53.430 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.431+0000 7fbbd4f9c700 1 --2- 172.21.15.111:0/1296504795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbd00cdb60 0x7fbbd00b29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:53.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.431+0000 7fbbc7fff700 1 --2- 172.21.15.111:0/1296504795 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbbd00cd790 0x7fbbd00c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:53.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.431+0000 7fbbcffff700 1 -- 172.21.15.111:0/1296504795 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbbd00cd790 msgr2=0x7fbbd00c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:53.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.431+0000 7fbbcffff700 1 --2- 172.21.15.111:0/1296504795 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbbd00cd790 0x7fbbd00c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:53.431 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.431+0000 7fbbcffff700 1 -- 172.21.15.111:0/1296504795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbd00cdb60 msgr2=0x7fbbd00b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:53.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.431+0000 7fbbcffff700 1 --2- 172.21.15.111:0/1296504795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbd00cdb60 0x7fbbd00b29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:53.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.431+0000 7fbbcffff700 1 -- 172.21.15.111:0/1296504795 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc0009550 con 0x7fbbd00b1d50 2023-03-29T10:59:53.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.431+0000 7fbbcffff700 1 --2- 172.21.15.111:0/1296504795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbd00b1d50 0x7fbbd01509b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0014040 tx=0x7fbbc0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:53.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.431+0000 7fbbcd7fa700 1 -- 172.21.15.111:0/1296504795 <== mon.0 v2:172.21.15.111:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc00036a0 con 0x7fbbd00b1d50 2023-03-29T10:59:53.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.431+0000 7fbbcd7fa700 1 -- 172.21.15.111:0/1296504795 <== mon.0 v2:172.21.15.111:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc001e070 con 0x7fbbd00b1d50 2023-03-29T10:59:53.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.431+0000 7fbbd69ff700 1 -- 172.21.15.111:0/1296504795 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd00ce000 con 0x7fbbd00b1d50 2023-03-29T10:59:53.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.431+0000 7fbbcd7fa700 1 -- 172.21.15.111:0/1296504795 <== mon.0 v2:172.21.15.111:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0003a20 con 0x7fbbd00b1d50 2023-03-29T10:59:53.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.431+0000 7fbbd69ff700 1 -- 172.21.15.111:0/1296504795 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd006b0e0 con 0x7fbbd00b1d50 2023-03-29T10:59:53.432 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.432+0000 7fbbd69ff700 1 -- 172.21.15.111:0/1296504795 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd015ca20 con 0x7fbbd00b1d50 2023-03-29T10:59:53.433 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.433+0000 7fbbcd7fa700 1 -- 172.21.15.111:0/1296504795 <== mon.0 v2:172.21.15.111:3300/0 4 ==== mgrmap(e 5) v1 ==== 87609+0+0 (secure 0 0 0) 0x7fbbc0003b80 con 0x7fbbd00b1d50 2023-03-29T10:59:53.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.434+0000 7fbbcd7fa700 1 --2- 172.21.15.111:0/1296504795 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbbb0069de0 0x7fbbb006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-03-29T10:59:53.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.434+0000 7fbbcd7fa700 1 -- 172.21.15.111:0/1296504795 <== mon.0 v2:172.21.15.111:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbbc0020070 con 0x7fbbd00b1d50 2023-03-29T10:59:53.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.435+0000 7fbbc7fff700 1 --2- 172.21.15.111:0/1296504795 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbbb0069de0 0x7fbbb006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-03-29T10:59:53.434 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.435+0000 7fbbc7fff700 1 --2- 172.21.15.111:0/1296504795 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbbb0069de0 0x7fbbb006c290 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbbd015efe0 tx=0x7fbbb8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-03-29T10:59:53.438 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.439+0000 7fbbcd7fa700 1 -- 172.21.15.111:0/1296504795 <== mon.0 v2:172.21.15.111:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176450 (secure 0 0 0) 0x7fbbc0055050 con 0x7fbbd00b1d50 2023-03-29T10:59:53.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.707+0000 7fbbd69ff700 1 -- 172.21.15.111:0/1296504795 --> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.admin"} v 0) v1 -- 0x7fbbd006c510 con 0x7fbbd00b1d50 2023-03-29T10:59:53.707 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.707+0000 7fbbcd7fa700 1 -- 172.21.15.111:0/1296504795 <== mon.0 v2:172.21.15.111:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.admin"}]=0 v13) v1 ==== 82+0+151 (secure 0 0 0) 0x7fbbc0058670 con 0x7fbbd00b1d50 2023-03-29T10:59:53.707 INFO:teuthology.orchestra.run.smithi111.stdout:[client.admin] 2023-03-29T10:59:53.708 INFO:teuthology.orchestra.run.smithi111.stdout: key = AQBYGSRkuxcuAxAAXotRkav5CccVRCV8v8dLNA== 2023-03-29T10:59:53.708 INFO:teuthology.orchestra.run.smithi111.stdout: caps mds = "allow *" 2023-03-29T10:59:53.708 INFO:teuthology.orchestra.run.smithi111.stdout: caps mgr = "allow *" 2023-03-29T10:59:53.708 INFO:teuthology.orchestra.run.smithi111.stdout: caps mon = "allow *" 2023-03-29T10:59:53.708 INFO:teuthology.orchestra.run.smithi111.stdout: caps osd = "allow *" 2023-03-29T10:59:53.709 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.710+0000 7fbbd69ff700 1 -- 172.21.15.111:0/1296504795 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbbb0069de0 msgr2=0x7fbbb006c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:53.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.710+0000 7fbbd69ff700 1 --2- 172.21.15.111:0/1296504795 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbbb0069de0 0x7fbbb006c290 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbbd015efe0 tx=0x7fbbb8009040 comp rx=0 tx=0).stop 2023-03-29T10:59:53.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.710+0000 7fbbd69ff700 1 -- 172.21.15.111:0/1296504795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbd00b1d50 msgr2=0x7fbbd01509b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-03-29T10:59:53.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.710+0000 7fbbd69ff700 1 --2- 172.21.15.111:0/1296504795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbd00b1d50 0x7fbbd01509b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0014040 tx=0x7fbbc0004750 comp rx=0 tx=0).stop 2023-03-29T10:59:53.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.710+0000 7fbbd69ff700 1 -- 172.21.15.111:0/1296504795 shutdown_connections 2023-03-29T10:59:53.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.710+0000 7fbbd69ff700 1 --2- 172.21.15.111:0/1296504795 >> [v2:172.21.15.111:6832/110141,v1:172.21.15.111:6833/110141] conn(0x7fbbb0069de0 0x7fbbb006c290 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:53.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.710+0000 7fbbd69ff700 1 --2- 172.21.15.111:0/1296504795 >> [v2:172.21.15.111:3300/0,v1:172.21.15.111:6789/0] conn(0x7fbbd00b1d50 0x7fbbd01509b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:53.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.710+0000 7fbbd69ff700 1 --2- 172.21.15.111:0/1296504795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbd00cdb60 0x7fbbd00b29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:53.710 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.710+0000 7fbbd69ff700 1 --2- 172.21.15.111:0/1296504795 >> [v2:172.21.15.119:3301/0,v1:172.21.15.119:6790/0] conn(0x7fbbd00cd790 0x7fbbd00c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-03-29T10:59:53.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.710+0000 7fbbd69ff700 1 -- 172.21.15.111:0/1296504795 >> 172.21.15.111:0/1296504795 conn(0x7fbbd006e980 msgr2=0x7fbbd0160b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-03-29T10:59:53.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.710+0000 7fbbd69ff700 1 -- 172.21.15.111:0/1296504795 shutdown_connections 2023-03-29T10:59:53.711 INFO:teuthology.orchestra.run.smithi111.stderr:2023-03-29T10:59:53.710+0000 7fbbd69ff700 1 -- 172.21.15.111:0/1296504795 wait complete. 2023-03-29T10:59:53.726 DEBUG:teuthology.orchestra.run.smithi111:> mktemp 2023-03-29T10:59:53.767 INFO:teuthology.orchestra.run.smithi111.stdout:/tmp/tmp.VsLC135jKL 2023-03-29T10:59:53.767 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:59:53.768 DEBUG:teuthology.orchestra.run.smithi111:> dd of=/tmp/tmp.VsLC135jKL 2023-03-29T10:59:53.848 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:59:53.848 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/tmp/tmp.CzuMqE4LuJxfstests-dev/local.config 2023-03-29T10:59:53.950 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph-fuse export CEPH_FUSE_BIN_PATH=ceph-fuse export TEST_DEV=ceph-fuse # without this tests won't get started export TEST_DIR=/tmp/tmp.6dtVtxiLHFtest export SCRATCH_DEV= export SCRATCH_MNT=/tmp/tmp.KcLS4LFr4Iscratch export CEPHFS_MOUNT_OPTIONS="-m 172.21.15.111:6789 -k /tmp/tmp.VsLC135jKL --client_mountpoint /test" 2023-03-29T10:59:53.951 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T10:59:53.951 DEBUG:teuthology.orchestra.run.smithi111:> sudo dd of=/tmp/tmp.CzuMqE4LuJxfstests-dev/ceph.exclude 2023-03-29T10:59:54.052 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo make) 2023-03-29T10:59:54.173 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize -cfi 2023-03-29T10:59:54.652 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: putting auxiliary files in '.'. 2023-03-29T10:59:54.652 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: copying file './config.guess' 2023-03-29T10:59:54.661 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: copying file './config.sub' 2023-03-29T10:59:54.670 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: copying file './install-sh' 2023-03-29T10:59:54.679 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: copying file './ltmain.sh' 2023-03-29T10:59:54.734 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2023-03-29T10:59:54.734 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: copying file 'm4/libtool.m4' 2023-03-29T10:59:54.745 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2023-03-29T10:59:54.756 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2023-03-29T10:59:54.767 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: copying file 'm4/ltversion.m4' 2023-03-29T10:59:54.778 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2023-03-29T10:59:54.792 INFO:teuthology.orchestra.run.smithi111.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2023-03-29T10:59:54.795 INFO:teuthology.orchestra.run.smithi111.stdout:cp include/install-sh . 2023-03-29T10:59:54.797 INFO:teuthology.orchestra.run.smithi111.stdout:aclocal -I m4 2023-03-29T10:59:56.045 INFO:teuthology.orchestra.run.smithi111.stdout:autoheader 2023-03-29T10:59:56.405 INFO:teuthology.orchestra.run.smithi111.stdout:autoconf 2023-03-29T10:59:56.606 INFO:teuthology.orchestra.run.smithi111.stdout:./configure \ 2023-03-29T10:59:56.606 INFO:teuthology.orchestra.run.smithi111.stdout: --libexecdir=/usr/lib \ 2023-03-29T10:59:56.606 INFO:teuthology.orchestra.run.smithi111.stdout: --exec_prefix=/var/lib 2023-03-29T10:59:56.934 INFO:teuthology.orchestra.run.smithi111.stdout:checking build system type... x86_64-pc-linux-gnu 2023-03-29T10:59:56.935 INFO:teuthology.orchestra.run.smithi111.stdout:checking host system type... x86_64-pc-linux-gnu 2023-03-29T10:59:56.937 INFO:teuthology.orchestra.run.smithi111.stdout:checking how to print strings... printf 2023-03-29T10:59:56.938 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gcc... gcc 2023-03-29T10:59:57.086 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether the C compiler works... yes 2023-03-29T10:59:57.086 INFO:teuthology.orchestra.run.smithi111.stdout:checking for C compiler default output file name... a.out 2023-03-29T10:59:57.133 INFO:teuthology.orchestra.run.smithi111.stdout:checking for suffix of executables... 2023-03-29T10:59:57.192 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether we are cross compiling... no 2023-03-29T10:59:57.227 INFO:teuthology.orchestra.run.smithi111.stdout:checking for suffix of object files... o 2023-03-29T10:59:57.258 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether we are using the GNU C compiler... yes 2023-03-29T10:59:57.289 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether gcc accepts -g... yes 2023-03-29T10:59:57.334 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gcc option to accept ISO C89... none needed 2023-03-29T10:59:57.340 INFO:teuthology.orchestra.run.smithi111.stdout:checking for a sed that does not truncate output... /bin/sed 2023-03-29T10:59:57.345 INFO:teuthology.orchestra.run.smithi111.stdout:checking for grep that handles long lines and -e... /bin/grep 2023-03-29T10:59:57.347 INFO:teuthology.orchestra.run.smithi111.stdout:checking for egrep... /bin/grep -E 2023-03-29T10:59:57.350 INFO:teuthology.orchestra.run.smithi111.stdout:checking for fgrep... /bin/grep -F 2023-03-29T10:59:57.357 INFO:teuthology.orchestra.run.smithi111.stdout:checking for ld used by gcc... /bin/ld 2023-03-29T10:59:57.361 INFO:teuthology.orchestra.run.smithi111.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2023-03-29T10:59:57.368 INFO:teuthology.orchestra.run.smithi111.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2023-03-29T10:59:57.404 INFO:teuthology.orchestra.run.smithi111.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2023-03-29T10:59:57.404 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether ln -s works... yes 2023-03-29T10:59:57.421 INFO:teuthology.orchestra.run.smithi111.stdout:checking the maximum length of command line arguments... 1572864 2023-03-29T10:59:57.426 INFO:teuthology.orchestra.run.smithi111.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2023-03-29T10:59:57.427 INFO:teuthology.orchestra.run.smithi111.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2023-03-29T10:59:57.427 INFO:teuthology.orchestra.run.smithi111.stdout:checking for /bin/ld option to reload object files... -r 2023-03-29T10:59:57.427 INFO:teuthology.orchestra.run.smithi111.stdout:checking for objdump... objdump 2023-03-29T10:59:57.428 INFO:teuthology.orchestra.run.smithi111.stdout:checking how to recognize dependent libraries... pass_all 2023-03-29T10:59:57.428 INFO:teuthology.orchestra.run.smithi111.stdout:checking for dlltool... no 2023-03-29T10:59:57.429 INFO:teuthology.orchestra.run.smithi111.stdout:checking how to associate runtime and link libraries... printf %s\n 2023-03-29T10:59:57.429 INFO:teuthology.orchestra.run.smithi111.stdout:checking for ar... ar 2023-03-29T10:59:57.474 INFO:teuthology.orchestra.run.smithi111.stdout:checking for archiver @FILE support... @ 2023-03-29T10:59:57.474 INFO:teuthology.orchestra.run.smithi111.stdout:checking for strip... strip 2023-03-29T10:59:57.474 INFO:teuthology.orchestra.run.smithi111.stdout:checking for ranlib... ranlib 2023-03-29T10:59:57.475 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gawk... gawk 2023-03-29T10:59:57.577 INFO:teuthology.orchestra.run.smithi111.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2023-03-29T10:59:57.579 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sysroot... no 2023-03-29T10:59:57.586 INFO:teuthology.orchestra.run.smithi111.stdout:checking for a working dd... /bin/dd 2023-03-29T10:59:57.592 INFO:teuthology.orchestra.run.smithi111.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2023-03-29T10:59:57.619 INFO:teuthology.orchestra.run.smithi111.stdout:checking for mt... no 2023-03-29T10:59:57.624 INFO:teuthology.orchestra.run.smithi111.stdout:checking if : is a manifest tool... no 2023-03-29T10:59:57.679 INFO:teuthology.orchestra.run.smithi111.stdout:checking how to run the C preprocessor... gcc -E 2023-03-29T10:59:57.917 INFO:teuthology.orchestra.run.smithi111.stdout:checking for ANSI C header files... yes 2023-03-29T10:59:57.958 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/types.h... yes 2023-03-29T10:59:58.007 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/stat.h... yes 2023-03-29T10:59:58.055 INFO:teuthology.orchestra.run.smithi111.stdout:checking for stdlib.h... yes 2023-03-29T10:59:58.104 INFO:teuthology.orchestra.run.smithi111.stdout:checking for string.h... yes 2023-03-29T10:59:58.160 INFO:teuthology.orchestra.run.smithi111.stdout:checking for memory.h... yes 2023-03-29T10:59:58.209 INFO:teuthology.orchestra.run.smithi111.stdout:checking for strings.h... yes 2023-03-29T10:59:58.260 INFO:teuthology.orchestra.run.smithi111.stdout:checking for inttypes.h... yes 2023-03-29T10:59:58.310 INFO:teuthology.orchestra.run.smithi111.stdout:checking for stdint.h... yes 2023-03-29T10:59:58.370 INFO:teuthology.orchestra.run.smithi111.stdout:checking for unistd.h... yes 2023-03-29T10:59:58.418 INFO:teuthology.orchestra.run.smithi111.stdout:checking for dlfcn.h... yes 2023-03-29T10:59:58.426 INFO:teuthology.orchestra.run.smithi111.stdout:checking for objdir... .libs 2023-03-29T10:59:58.552 INFO:teuthology.orchestra.run.smithi111.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2023-03-29T10:59:58.552 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2023-03-29T10:59:58.586 INFO:teuthology.orchestra.run.smithi111.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2023-03-29T10:59:58.621 INFO:teuthology.orchestra.run.smithi111.stdout:checking if gcc static flag -static works... no 2023-03-29T10:59:58.669 INFO:teuthology.orchestra.run.smithi111.stdout:checking if gcc supports -c -o file.o... yes 2023-03-29T10:59:58.669 INFO:teuthology.orchestra.run.smithi111.stdout:checking if gcc supports -c -o file.o... (cached) yes 2023-03-29T10:59:58.683 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2023-03-29T10:59:58.728 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether -lc should be explicitly linked in... no 2023-03-29T10:59:58.818 INFO:teuthology.orchestra.run.smithi111.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2023-03-29T10:59:58.818 INFO:teuthology.orchestra.run.smithi111.stdout:checking how to hardcode library paths into programs... immediate 2023-03-29T10:59:58.828 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether stripping libraries is possible... yes 2023-03-29T10:59:58.828 INFO:teuthology.orchestra.run.smithi111.stdout:checking if libtool supports shared libraries... yes 2023-03-29T10:59:58.829 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether to build shared libraries... yes 2023-03-29T10:59:58.829 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether to build static libraries... yes 2023-03-29T10:59:58.839 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gcc... (cached) gcc 2023-03-29T10:59:58.874 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether we are using the GNU C compiler... (cached) yes 2023-03-29T10:59:58.874 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether gcc accepts -g... (cached) yes 2023-03-29T10:59:58.874 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2023-03-29T10:59:58.874 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gmake... /usr/bin/gmake 2023-03-29T10:59:58.875 INFO:teuthology.orchestra.run.smithi111.stderr:./configure: line 12070: test: $(SHELL): binary operator expected 2023-03-29T10:59:58.875 INFO:teuthology.orchestra.run.smithi111.stdout:checking for tar... /bin/tar 2023-03-29T10:59:58.875 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gzip... /bin/gzip 2023-03-29T10:59:58.890 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether gcc -MM is supported... yes 2023-03-29T10:59:58.891 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sort... /bin/sort 2023-03-29T10:59:58.891 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether ln -s works... yes 2023-03-29T10:59:58.891 INFO:teuthology.orchestra.run.smithi111.stdout:checking for rpm... /bin/rpm 2023-03-29T10:59:58.899 INFO:teuthology.orchestra.run.smithi111.stdout:checking for rpmbuild... no 2023-03-29T10:59:58.948 INFO:teuthology.orchestra.run.smithi111.stdout:checking assert.h usability... yes 2023-03-29T10:59:58.962 INFO:teuthology.orchestra.run.smithi111.stdout:checking assert.h presence... yes 2023-03-29T10:59:58.962 INFO:teuthology.orchestra.run.smithi111.stdout:checking for assert.h... yes 2023-03-29T10:59:59.014 INFO:teuthology.orchestra.run.smithi111.stdout:checking bstring.h usability... no 2023-03-29T10:59:59.036 INFO:teuthology.orchestra.run.smithi111.stdout:checking bstring.h presence... no 2023-03-29T10:59:59.036 INFO:teuthology.orchestra.run.smithi111.stdout:checking for bstring.h... no 2023-03-29T10:59:59.083 INFO:teuthology.orchestra.run.smithi111.stdout:checking libgen.h usability... yes 2023-03-29T10:59:59.097 INFO:teuthology.orchestra.run.smithi111.stdout:checking libgen.h presence... yes 2023-03-29T10:59:59.097 INFO:teuthology.orchestra.run.smithi111.stdout:checking for libgen.h... yes 2023-03-29T10:59:59.156 INFO:teuthology.orchestra.run.smithi111.stdout:checking dirent.h usability... yes 2023-03-29T10:59:59.172 INFO:teuthology.orchestra.run.smithi111.stdout:checking dirent.h presence... yes 2023-03-29T10:59:59.172 INFO:teuthology.orchestra.run.smithi111.stdout:checking for dirent.h... yes 2023-03-29T10:59:59.266 INFO:teuthology.orchestra.run.smithi111.stdout:checking errno.h usability... yes 2023-03-29T10:59:59.282 INFO:teuthology.orchestra.run.smithi111.stdout:checking errno.h presence... yes 2023-03-29T10:59:59.282 INFO:teuthology.orchestra.run.smithi111.stdout:checking for errno.h... yes 2023-03-29T10:59:59.335 INFO:teuthology.orchestra.run.smithi111.stdout:checking malloc.h usability... yes 2023-03-29T10:59:59.352 INFO:teuthology.orchestra.run.smithi111.stdout:checking malloc.h presence... yes 2023-03-29T10:59:59.352 INFO:teuthology.orchestra.run.smithi111.stdout:checking for malloc.h... yes 2023-03-29T10:59:59.405 INFO:teuthology.orchestra.run.smithi111.stdout:checking uuid.h usability... no 2023-03-29T10:59:59.428 INFO:teuthology.orchestra.run.smithi111.stdout:checking uuid.h presence... no 2023-03-29T10:59:59.428 INFO:teuthology.orchestra.run.smithi111.stdout:checking for uuid.h... no 2023-03-29T10:59:59.483 INFO:teuthology.orchestra.run.smithi111.stdout:checking uuid/uuid.h usability... yes 2023-03-29T10:59:59.500 INFO:teuthology.orchestra.run.smithi111.stdout:checking uuid/uuid.h presence... yes 2023-03-29T10:59:59.501 INFO:teuthology.orchestra.run.smithi111.stdout:checking for uuid/uuid.h... yes 2023-03-29T10:59:59.552 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/uuid.h usability... no 2023-03-29T10:59:59.574 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/uuid.h presence... no 2023-03-29T10:59:59.575 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/uuid.h... no 2023-03-29T10:59:59.629 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/file.h usability... yes 2023-03-29T10:59:59.644 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/file.h presence... yes 2023-03-29T10:59:59.644 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/file.h... yes 2023-03-29T10:59:59.703 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/fcntl.h usability... yes 2023-03-29T10:59:59.718 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/fcntl.h presence... yes 2023-03-29T10:59:59.718 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/fcntl.h... yes 2023-03-29T10:59:59.770 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/syssgi.h usability... no 2023-03-29T10:59:59.792 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/syssgi.h presence... no 2023-03-29T10:59:59.792 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/syssgi.h... no 2023-03-29T10:59:59.868 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/param.h usability... yes 2023-03-29T10:59:59.888 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/param.h presence... yes 2023-03-29T10:59:59.888 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/param.h... yes 2023-03-29T10:59:59.901 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/stat.h... (cached) yes 2023-03-29T10:59:59.954 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/statvfs.h usability... yes 2023-03-29T10:59:59.969 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/statvfs.h presence... yes 2023-03-29T10:59:59.969 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/statvfs.h... yes 2023-03-29T11:00:00.022 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/time.h usability... yes 2023-03-29T11:00:00.038 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/time.h presence... yes 2023-03-29T11:00:00.038 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/time.h... yes 2023-03-29T11:00:00.109 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/ioctl.h usability... yes 2023-03-29T11:00:00.125 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/ioctl.h presence... yes 2023-03-29T11:00:00.125 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/ioctl.h... yes 2023-03-29T11:00:00.182 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/wait.h usability... yes 2023-03-29T11:00:00.199 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/wait.h presence... yes 2023-03-29T11:00:00.200 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/wait.h... yes 2023-03-29T11:00:00.212 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/types.h... (cached) yes 2023-03-29T11:00:00.224 INFO:teuthology.orchestra.run.smithi111.stdout:checking for strings.h... (cached) yes 2023-03-29T11:00:00.278 INFO:teuthology.orchestra.run.smithi111.stdout:checking err.h usability... yes 2023-03-29T11:00:00.293 INFO:teuthology.orchestra.run.smithi111.stdout:checking err.h presence... yes 2023-03-29T11:00:00.293 INFO:teuthology.orchestra.run.smithi111.stdout:checking for err.h... yes 2023-03-29T11:00:00.350 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/falloc.h usability... yes 2023-03-29T11:00:00.363 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/falloc.h presence... yes 2023-03-29T11:00:00.364 INFO:teuthology.orchestra.run.smithi111.stdout:checking for linux/falloc.h... yes 2023-03-29T11:00:00.415 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/fs/xfs_fsops.h usability... no 2023-03-29T11:00:00.437 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/fs/xfs_fsops.h presence... no 2023-03-29T11:00:00.437 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/fs/xfs_fsops.h... no 2023-03-29T11:00:00.482 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/fs/xfs_itable.h usability... no 2023-03-29T11:00:00.503 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/fs/xfs_itable.h presence... no 2023-03-29T11:00:00.504 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/fs/xfs_itable.h... no 2023-03-29T11:00:00.548 INFO:teuthology.orchestra.run.smithi111.stdout:checking xfs/platform_defs.h usability... no 2023-03-29T11:00:00.570 INFO:teuthology.orchestra.run.smithi111.stdout:checking xfs/platform_defs.h presence... no 2023-03-29T11:00:00.570 INFO:teuthology.orchestra.run.smithi111.stdout:checking for xfs/platform_defs.h... no 2023-03-29T11:00:00.615 INFO:teuthology.orchestra.run.smithi111.stdout:checking btrfs/ioctl.h usability... no 2023-03-29T11:00:00.636 INFO:teuthology.orchestra.run.smithi111.stdout:checking btrfs/ioctl.h presence... no 2023-03-29T11:00:00.636 INFO:teuthology.orchestra.run.smithi111.stdout:checking for btrfs/ioctl.h... no 2023-03-29T11:00:00.681 INFO:teuthology.orchestra.run.smithi111.stdout:checking cifs/ioctl.h usability... no 2023-03-29T11:00:00.703 INFO:teuthology.orchestra.run.smithi111.stdout:checking cifs/ioctl.h presence... no 2023-03-29T11:00:00.703 INFO:teuthology.orchestra.run.smithi111.stdout:checking for cifs/ioctl.h... no 2023-03-29T11:00:00.760 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/mman.h usability... yes 2023-03-29T11:00:00.775 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/mman.h presence... yes 2023-03-29T11:00:00.775 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/mman.h... yes 2023-03-29T11:00:00.809 INFO:teuthology.orchestra.run.smithi111.stdout:checking for xfs/xfs_log_format.h... no 2023-03-29T11:00:00.873 INFO:teuthology.orchestra.run.smithi111.stdout:checking for uuid_compare... no 2023-03-29T11:00:00.926 INFO:teuthology.orchestra.run.smithi111.stdout:checking for uuid_compare in -luuid... yes 2023-03-29T11:00:01.024 INFO:teuthology.orchestra.run.smithi111.stdout:checking for xfs/xfs.h... yes 2023-03-29T11:00:01.053 INFO:teuthology.orchestra.run.smithi111.stdout:checking for xfs/libxfs.h... no 2023-03-29T11:00:01.088 INFO:teuthology.orchestra.run.smithi111.stdout:checking whether xlog_assign_lsn is declared... no 2023-03-29T11:00:01.138 INFO:teuthology.orchestra.run.smithi111.stdout:checking for xfs/xqm.h... yes 2023-03-29T11:00:01.215 INFO:teuthology.orchestra.run.smithi111.stdout:checking xfsctl from xfs/xfs.h... ok 2023-03-29T11:00:01.258 INFO:teuthology.orchestra.run.smithi111.stdout:checking xfs/handle.h usability... yes 2023-03-29T11:00:01.271 INFO:teuthology.orchestra.run.smithi111.stdout:checking xfs/handle.h presence... yes 2023-03-29T11:00:01.272 INFO:teuthology.orchestra.run.smithi111.stdout:checking for xfs/handle.h... yes 2023-03-29T11:00:01.336 INFO:teuthology.orchestra.run.smithi111.stdout:checking for attr_list_by_handle in -lhandle... yes 2023-03-29T11:00:01.379 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/acl.h usability... yes 2023-03-29T11:00:01.396 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/acl.h presence... yes 2023-03-29T11:00:01.396 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/acl.h... yes 2023-03-29T11:00:01.448 INFO:teuthology.orchestra.run.smithi111.stdout:checking acl/libacl.h usability... yes 2023-03-29T11:00:01.465 INFO:teuthology.orchestra.run.smithi111.stdout:checking acl/libacl.h presence... yes 2023-03-29T11:00:01.465 INFO:teuthology.orchestra.run.smithi111.stdout:checking for acl/libacl.h... yes 2023-03-29T11:00:01.522 INFO:teuthology.orchestra.run.smithi111.stdout:checking for acl_init in -lacl... yes 2023-03-29T11:00:01.564 INFO:teuthology.orchestra.run.smithi111.stdout:checking gdbm-ndbm.h usability... no 2023-03-29T11:00:01.585 INFO:teuthology.orchestra.run.smithi111.stdout:checking gdbm-ndbm.h presence... no 2023-03-29T11:00:01.586 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gdbm-ndbm.h... no 2023-03-29T11:00:01.625 INFO:teuthology.orchestra.run.smithi111.stdout:checking gdbm/ndbm.h usability... no 2023-03-29T11:00:01.647 INFO:teuthology.orchestra.run.smithi111.stdout:checking gdbm/ndbm.h presence... no 2023-03-29T11:00:01.647 INFO:teuthology.orchestra.run.smithi111.stdout:checking for gdbm/ndbm.h... no 2023-03-29T11:00:01.687 INFO:teuthology.orchestra.run.smithi111.stdout:checking ndbm.h usability... no 2023-03-29T11:00:01.709 INFO:teuthology.orchestra.run.smithi111.stdout:checking ndbm.h presence... no 2023-03-29T11:00:01.709 INFO:teuthology.orchestra.run.smithi111.stdout:checking for ndbm.h... no 2023-03-29T11:00:01.753 INFO:teuthology.orchestra.run.smithi111.stdout:checking libaio.h usability... yes 2023-03-29T11:00:01.773 INFO:teuthology.orchestra.run.smithi111.stdout:checking libaio.h presence... yes 2023-03-29T11:00:01.773 INFO:teuthology.orchestra.run.smithi111.stdout:checking for libaio.h... yes 2023-03-29T11:00:01.814 INFO:teuthology.orchestra.run.smithi111.stdout:checking liburing.h usability... no 2023-03-29T11:00:01.836 INFO:teuthology.orchestra.run.smithi111.stdout:checking liburing.h presence... no 2023-03-29T11:00:01.836 INFO:teuthology.orchestra.run.smithi111.stdout:checking for liburing.h... no 2023-03-29T11:00:01.875 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/capability.h usability... no 2023-03-29T11:00:01.897 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/capability.h presence... no 2023-03-29T11:00:01.897 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/capability.h... no 2023-03-29T11:00:01.939 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/fiemap.h usability... yes 2023-03-29T11:00:01.953 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/fiemap.h presence... yes 2023-03-29T11:00:01.953 INFO:teuthology.orchestra.run.smithi111.stdout:checking for linux/fiemap.h... yes 2023-03-29T11:00:02.011 INFO:teuthology.orchestra.run.smithi111.stdout:checking for fallocate... yes 2023-03-29T11:00:02.068 INFO:teuthology.orchestra.run.smithi111.stdout:checking for open_by_handle_at... yes 2023-03-29T11:00:02.113 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/prctl.h usability... yes 2023-03-29T11:00:02.129 INFO:teuthology.orchestra.run.smithi111.stdout:checking sys/prctl.h presence... yes 2023-03-29T11:00:02.129 INFO:teuthology.orchestra.run.smithi111.stdout:checking for sys/prctl.h... yes 2023-03-29T11:00:02.175 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/fs.h usability... yes 2023-03-29T11:00:02.190 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/fs.h presence... yes 2023-03-29T11:00:02.190 INFO:teuthology.orchestra.run.smithi111.stdout:checking for linux/fs.h... yes 2023-03-29T11:00:02.230 INFO:teuthology.orchestra.run.smithi111.stdout:checking btrfsutil.h usability... no 2023-03-29T11:00:02.252 INFO:teuthology.orchestra.run.smithi111.stdout:checking btrfsutil.h presence... no 2023-03-29T11:00:02.252 INFO:teuthology.orchestra.run.smithi111.stdout:checking for btrfsutil.h... no 2023-03-29T11:00:02.311 INFO:teuthology.orchestra.run.smithi111.stdout:checking for copy_file_range... yes 2023-03-29T11:00:02.372 INFO:teuthology.orchestra.run.smithi111.stdout:checking for SEEK_DATA... yes 2023-03-29T11:00:02.449 INFO:teuthology.orchestra.run.smithi111.stdout:checking for BMV_OF_SHARED... yes 2023-03-29T11:00:02.517 INFO:teuthology.orchestra.run.smithi111.stdout:checking for nftw... yes 2023-03-29T11:00:02.577 INFO:teuthology.orchestra.run.smithi111.stdout:checking for RLIMIT_NOFILE... yes 2023-03-29T11:00:02.612 INFO:teuthology.orchestra.run.smithi111.stdout:checking for FIEXCHANGE_RANGE... no 2023-03-29T11:00:02.670 INFO:teuthology.orchestra.run.smithi111.stdout:checking for renameat2... yes 2023-03-29T11:00:02.727 INFO:teuthology.orchestra.run.smithi111.stdout:checking for reallocarray... yes 2023-03-29T11:00:02.757 INFO:teuthology.orchestra.run.smithi111.stdout:checking for struct mount_attr... no 2023-03-29T11:00:02.827 INFO:teuthology.orchestra.run.smithi111.stdout:checking for struct btrfs_qgroup_limit... yes 2023-03-29T11:00:02.895 INFO:teuthology.orchestra.run.smithi111.stdout:checking for struct btrfs_qgroup_inherit... yes 2023-03-29T11:00:02.963 INFO:teuthology.orchestra.run.smithi111.stdout:checking for struct btrfs_ioctl_vol_args... yes 2023-03-29T11:00:03.031 INFO:teuthology.orchestra.run.smithi111.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2023-03-29T11:00:03.099 INFO:teuthology.orchestra.run.smithi111.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2023-03-29T11:00:03.166 INFO:teuthology.orchestra.run.smithi111.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2023-03-29T11:00:03.234 INFO:teuthology.orchestra.run.smithi111.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2023-03-29T11:00:03.285 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/btrfs.h usability... yes 2023-03-29T11:00:03.300 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/btrfs.h presence... yes 2023-03-29T11:00:03.300 INFO:teuthology.orchestra.run.smithi111.stdout:checking for linux/btrfs.h... yes 2023-03-29T11:00:03.356 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/btrfs_tree.h usability... yes 2023-03-29T11:00:03.372 INFO:teuthology.orchestra.run.smithi111.stdout:checking linux/btrfs_tree.h presence... yes 2023-03-29T11:00:03.372 INFO:teuthology.orchestra.run.smithi111.stdout:checking for linux/btrfs_tree.h... yes 2023-03-29T11:00:03.439 INFO:teuthology.orchestra.run.smithi111.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... no 2023-03-29T11:00:03.466 INFO:teuthology.orchestra.run.smithi111.stdout:configure: creating ./config.status 2023-03-29T11:00:04.276 INFO:teuthology.orchestra.run.smithi111.stdout:config.status: creating include/builddefs 2023-03-29T11:00:04.303 INFO:teuthology.orchestra.run.smithi111.stdout:config.status: creating include/config.h 2023-03-29T11:00:04.323 INFO:teuthology.orchestra.run.smithi111.stdout:config.status: executing libtool commands 2023-03-29T11:00:04.395 INFO:teuthology.orchestra.run.smithi111.stdout:Building include 2023-03-29T11:00:04.402 INFO:teuthology.orchestra.run.smithi111.stdout:Building lib 2023-03-29T11:00:04.408 INFO:teuthology.orchestra.run.smithi111.stdout: [LTDEP] 2023-03-29T11:00:04.603 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] dataascii.lo 2023-03-29T11:00:04.792 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] databin.lo 2023-03-29T11:00:05.012 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] datapid.lo 2023-03-29T11:00:05.189 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] file_lock.lo 2023-03-29T11:00:05.412 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] forker.lo 2023-03-29T11:00:05.618 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] pattern.lo 2023-03-29T11:00:05.808 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] open_flags.lo 2023-03-29T11:00:06.024 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] random_range.lo 2023-03-29T11:00:06.318 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] string_to_tokens.lo 2023-03-29T11:00:06.493 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] str_to_bytes.lo 2023-03-29T11:00:06.695 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] tlibio.lo 2023-03-29T11:00:07.000 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] write_log.lo 2023-03-29T11:00:07.313 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] random.lo 2023-03-29T11:00:07.506 INFO:teuthology.orchestra.run.smithi111.stdout: [LD] 2023-03-29T11:00:07.619 INFO:teuthology.orchestra.run.smithi111.stdout:Building ltp 2023-03-29T11:00:07.625 INFO:teuthology.orchestra.run.smithi111.stdout: [DEP] 2023-03-29T11:00:07.810 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] doio 2023-03-29T11:00:08.548 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fsstress 2023-03-29T11:00:09.897 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fsx 2023-03-29T11:00:10.818 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] growfiles 2023-03-29T11:00:11.397 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] iogen 2023-03-29T11:00:11.832 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-stress 2023-03-29T11:00:12.327 INFO:teuthology.orchestra.run.smithi111.stdout:Building src 2023-03-29T11:00:12.336 INFO:teuthology.orchestra.run.smithi111.stdout: [DEP] 2023-03-29T11:00:14.694 INFO:teuthology.orchestra.run.smithi111.stdout: [DEP] 2023-03-29T11:00:14.743 INFO:teuthology.orchestra.run.smithi111.stdout: [DEP] 2023-03-29T11:00:14.758 INFO:teuthology.orchestra.run.smithi111.stdout: [DEP] 2023-03-29T11:00:15.091 INFO:teuthology.orchestra.run.smithi111.stdout: [DEP] 2023-03-29T11:00:15.129 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] dirstress 2023-03-29T11:00:15.450 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fill 2023-03-29T11:00:15.687 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fill2 2023-03-29T11:00:15.998 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] getpagesize 2023-03-29T11:00:16.216 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] holes 2023-03-29T11:00:16.499 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] lstat64 2023-03-29T11:00:16.764 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] nametest 2023-03-29T11:00:17.086 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] permname 2023-03-29T11:00:17.368 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] randholes 2023-03-29T11:00:17.699 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] runas 2023-03-29T11:00:17.961 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] truncfile 2023-03-29T11:00:18.229 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] usemem 2023-03-29T11:00:18.466 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] mmapcat 2023-03-29T11:00:18.698 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] append_reader 2023-03-29T11:00:18.932 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] append_writer 2023-03-29T11:00:19.169 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] dirperf 2023-03-29T11:00:19.475 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] metaperf 2023-03-29T11:00:19.810 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] devzero 2023-03-29T11:00:20.072 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] feature 2023-03-29T11:00:20.372 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] alloc 2023-03-29T11:00:20.668 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fault 2023-03-29T11:00:20.919 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fstest 2023-03-29T11:00:21.238 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_access_root 2023-03-29T11:00:21.475 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] godown 2023-03-29T11:00:21.729 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] resvtest 2023-03-29T11:00:22.002 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] writemod 2023-03-29T11:00:22.231 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] writev_on_pagefault 2023-03-29T11:00:22.476 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] makeextents 2023-03-29T11:00:22.572 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:00:22.575 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:00:22.746 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] itrash 2023-03-29T11:00:22.983 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] rename 2023-03-29T11:00:23.206 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] multi_open_unlink 2023-03-29T11:00:23.455 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] unwritten_sync 2023-03-29T11:00:23.721 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] genhashnames 2023-03-29T11:00:23.989 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_holes 2023-03-29T11:00:24.221 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mmap_writev 2023-03-29T11:00:24.457 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_truncate_cmtime 2023-03-29T11:00:24.696 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] dirhash_collide 2023-03-29T11:00:24.959 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_rename_overwrite 2023-03-29T11:00:25.183 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] holetest 2023-03-29T11:00:25.483 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_truncate_self 2023-03-29T11:00:25.705 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] af_unix 2023-03-29T11:00:25.942 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mmap_stale_pmd 2023-03-29T11:00:26.179 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mmap_cow_race 2023-03-29T11:00:26.428 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mmap_fallocate 2023-03-29T11:00:26.660 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fsync-err 2023-03-29T11:00:26.927 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mmap_write_ro 2023-03-29T11:00:27.161 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_ext4_dax_journal_corruption 2023-03-29T11:00:27.402 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_ext4_dax_inline_corruption 2023-03-29T11:00:27.641 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_ofd_locks 2023-03-29T11:00:27.925 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mmap_collision 2023-03-29T11:00:28.206 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] mmap-write-concurrent 2023-03-29T11:00:28.458 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_get_file_time 2023-03-29T11:00:28.688 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_create_short_dirs 2023-03-29T11:00:28.936 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_create_long_dirs 2023-03-29T11:00:29.182 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_enospc 2023-03-29T11:00:29.452 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mmap_writev_overlap 2023-03-29T11:00:29.703 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] checkpoint_journal 2023-03-29T11:00:29.942 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] mmap-rw-fault 2023-03-29T11:00:30.206 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] allocstale 2023-03-29T11:00:30.460 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mmap_cow_memory_failure 2023-03-29T11:00:30.717 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fake-dump-rootino 2023-03-29T11:00:30.949 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] dio-buf-fault 2023-03-29T11:00:31.185 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] xfsctl 2023-03-29T11:00:31.458 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] bstat 2023-03-29T11:00:31.745 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mtab 2023-03-29T11:00:32.016 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] getdevicesize 2023-03-29T11:00:32.240 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] preallo_rw_pattern_reader 2023-03-29T11:00:32.487 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] preallo_rw_pattern_writer 2023-03-29T11:00:32.733 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] ftrunc 2023-03-29T11:00:32.964 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] trunc 2023-03-29T11:00:33.209 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fs_perms 2023-03-29T11:00:33.458 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] testx 2023-03-29T11:00:33.672 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] looptest 2023-03-29T11:00:33.947 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] locktest 2023-03-29T11:00:34.333 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] unwritten_mmap 2023-03-29T11:00:34.583 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] bulkstat_unlink_test 2023-03-29T11:00:34.867 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] deduperace 2023-03-29T11:00:35.145 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] bulkstat_unlink_test_modified 2023-03-29T11:00:35.428 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_dir_offset 2023-03-29T11:00:35.660 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_futimens 2023-03-29T11:00:35.885 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_immutable 2023-03-29T11:00:36.561 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] stale_handle 2023-03-29T11:00:36.818 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] pwrite_mmap_blocked 2023-03-29T11:00:37.054 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_dir_offset2 2023-03-29T11:00:37.328 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] seek_sanity_test 2023-03-29T11:00:37.816 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] seek_copy_test 2023-03-29T11:00:38.076 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_readdir_1 2023-03-29T11:00:38.305 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_readdir_2 2023-03-29T11:00:38.541 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fsync-tester 2023-03-29T11:00:38.868 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] nsexec 2023-03-29T11:00:39.126 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] cloner 2023-03-29T11:00:39.387 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] renameat2 2023-03-29T11:00:39.639 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_getcwd 2023-03-29T11:00:39.882 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] e4compact 2023-03-29T11:00:40.167 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] test-nextquota 2023-03-29T11:00:40.425 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] punch-alternating 2023-03-29T11:00:40.683 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] attr-list-by-handle-cursor-test 2023-03-29T11:00:40.946 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] listxattr 2023-03-29T11:00:41.176 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] dio-interleaved 2023-03-29T11:00:41.419 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_dir_type 2023-03-29T11:00:41.661 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] dio-invalidate-cache 2023-03-29T11:00:41.946 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] stat_test 2023-03-29T11:00:42.323 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_encrypted_d_revalidate 2023-03-29T11:00:42.576 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] attr_replace_test 2023-03-29T11:00:42.823 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] swapon 2023-03-29T11:00:43.071 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] mkswap 2023-03-29T11:00:43.305 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_attr_corruption 2023-03-29T11:00:43.540 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_open_tmpfiles 2023-03-29T11:00:43.805 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fscrypt-crypt-util 2023-03-29T11:00:44.506 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] bulkstat_null_ocount 2023-03-29T11:00:44.753 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] splice-test 2023-03-29T11:00:45.017 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] chprojid_fail 2023-03-29T11:00:45.266 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] detached_mounts_propagation 2023-03-29T11:00:45.536 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] ext4_resize 2023-03-29T11:00:45.768 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_readdir_3 2023-03-29T11:00:46.045 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] splice2pipe 2023-03-29T11:00:46.292 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] uuid_ioctl 2023-03-29T11:00:46.533 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fiemap-tester 2023-03-29T11:00:46.885 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_stripealign 2023-03-29T11:00:47.122 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] open_by_handle 2023-03-29T11:00:47.455 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] t_mmap_dio 2023-03-29T11:00:47.700 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] xfsfind 2023-03-29T11:00:47.981 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] fssum 2023-03-29T11:00:48.492 INFO:teuthology.orchestra.run.smithi111.stdout:Building log-writes 2023-03-29T11:00:48.501 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] replay-log 2023-03-29T11:00:48.898 INFO:teuthology.orchestra.run.smithi111.stdout:Building perf 2023-03-29T11:00:48.904 INFO:teuthology.orchestra.run.smithi111.stdout:Building aio-dio-regress 2023-03-29T11:00:48.914 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-append-write-fallocate-race 2023-03-29T11:00:49.176 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-fcntl-race 2023-03-29T11:00:49.424 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-subblock-eof-read 2023-03-29T11:00:49.655 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-append-write-read-race 2023-03-29T11:00:49.919 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-invalidate-readahead 2023-03-29T11:00:50.170 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-write-verify 2023-03-29T11:00:50.447 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2023-03-29T11:00:50.670 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-cycle-write 2023-03-29T11:00:50.940 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-extend-stat 2023-03-29T11:00:51.194 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2023-03-29T11:00:51.414 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aiocp 2023-03-29T11:00:51.722 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-hole-filling-race 2023-03-29T11:00:51.959 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-invalidate-failure 2023-03-29T11:00:52.206 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aiodio_sparse2 2023-03-29T11:00:52.505 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-last-ref-held-by-io 2023-03-29T11:00:52.674 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:00:52.678 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:00:52.771 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-eof-race 2023-03-29T11:00:53.040 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] aio-dio-cow-race 2023-03-29T11:00:53.282 INFO:teuthology.orchestra.run.smithi111.stdout:Building vfs 2023-03-29T11:00:53.291 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] vfstest 2023-03-29T11:00:55.250 INFO:teuthology.orchestra.run.smithi111.stdout: [CC] mount-idmapped 2023-03-29T11:00:55.714 INFO:teuthology.orchestra.run.smithi111.stdout:Building m4 2023-03-29T11:00:55.720 INFO:teuthology.orchestra.run.smithi111.stdout:Building common 2023-03-29T11:00:55.727 INFO:teuthology.orchestra.run.smithi111.stdout:Building tests 2023-03-29T11:00:55.736 INFO:teuthology.orchestra.run.smithi111.stdout:Building perf 2023-03-29T11:00:55.745 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/perf/group.list 2023-03-29T11:00:55.772 INFO:teuthology.orchestra.run.smithi111.stdout:Building ext4 2023-03-29T11:00:55.781 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/ext4/group.list 2023-03-29T11:00:55.865 INFO:teuthology.orchestra.run.smithi111.stdout:Building ocfs2 2023-03-29T11:00:55.874 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/ocfs2/group.list 2023-03-29T11:00:55.905 INFO:teuthology.orchestra.run.smithi111.stdout:Building overlay 2023-03-29T11:00:55.914 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/overlay/group.list 2023-03-29T11:00:56.000 INFO:teuthology.orchestra.run.smithi111.stdout:Building cifs 2023-03-29T11:00:56.009 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/cifs/group.list 2023-03-29T11:00:56.037 INFO:teuthology.orchestra.run.smithi111.stdout:Building generic 2023-03-29T11:00:56.048 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/generic/group.list 2023-03-29T11:00:56.267 INFO:teuthology.orchestra.run.smithi111.stdout:Building shared 2023-03-29T11:00:56.276 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/shared/group.list 2023-03-29T11:00:56.314 INFO:teuthology.orchestra.run.smithi111.stdout:Building xfs 2023-03-29T11:00:56.326 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/xfs/group.list 2023-03-29T11:00:56.552 INFO:teuthology.orchestra.run.smithi111.stdout:Building f2fs 2023-03-29T11:00:56.561 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/f2fs/group.list 2023-03-29T11:00:56.601 INFO:teuthology.orchestra.run.smithi111.stdout:Building udf 2023-03-29T11:00:56.609 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/udf/group.list 2023-03-29T11:00:56.636 INFO:teuthology.orchestra.run.smithi111.stdout:Building ceph 2023-03-29T11:00:56.645 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/ceph/group.list 2023-03-29T11:00:56.677 INFO:teuthology.orchestra.run.smithi111.stdout:Building nfs 2023-03-29T11:00:56.686 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/nfs/group.list 2023-03-29T11:00:56.718 INFO:teuthology.orchestra.run.smithi111.stdout:Building btrfs 2023-03-29T11:00:56.728 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/btrfs/group.list 2023-03-29T11:00:56.856 INFO:teuthology.orchestra.run.smithi111.stdout:Building tmpfs 2023-03-29T11:00:56.865 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/tmpfs/group.list 2023-03-29T11:00:56.895 INFO:teuthology.orchestra.run.smithi111.stdout:Building selftest 2023-03-29T11:00:56.904 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/selftest/group.list 2023-03-29T11:00:56.935 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo make install) 2023-03-29T11:00:57.018 INFO:teuthology.orchestra.run.smithi111.stdout:Building include 2023-03-29T11:00:57.025 INFO:teuthology.orchestra.run.smithi111.stdout:Building lib 2023-03-29T11:00:57.032 INFO:teuthology.orchestra.run.smithi111.stdout:Building ltp 2023-03-29T11:00:57.039 INFO:teuthology.orchestra.run.smithi111.stdout:Building src 2023-03-29T11:00:57.096 INFO:teuthology.orchestra.run.smithi111.stdout:Building log-writes 2023-03-29T11:00:57.103 INFO:teuthology.orchestra.run.smithi111.stdout:Building perf 2023-03-29T11:00:57.109 INFO:teuthology.orchestra.run.smithi111.stdout:Building aio-dio-regress 2023-03-29T11:00:57.117 INFO:teuthology.orchestra.run.smithi111.stdout:Building vfs 2023-03-29T11:00:57.124 INFO:teuthology.orchestra.run.smithi111.stdout:Building m4 2023-03-29T11:00:57.130 INFO:teuthology.orchestra.run.smithi111.stdout:Building common 2023-03-29T11:00:57.137 INFO:teuthology.orchestra.run.smithi111.stdout:Building tests 2023-03-29T11:00:57.146 INFO:teuthology.orchestra.run.smithi111.stdout:Building perf 2023-03-29T11:00:57.155 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/perf/group.list 2023-03-29T11:00:57.182 INFO:teuthology.orchestra.run.smithi111.stdout:Building ext4 2023-03-29T11:00:57.191 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/ext4/group.list 2023-03-29T11:00:57.274 INFO:teuthology.orchestra.run.smithi111.stdout:Building ocfs2 2023-03-29T11:00:57.283 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/ocfs2/group.list 2023-03-29T11:00:57.313 INFO:teuthology.orchestra.run.smithi111.stdout:Building overlay 2023-03-29T11:00:57.322 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/overlay/group.list 2023-03-29T11:00:57.407 INFO:teuthology.orchestra.run.smithi111.stdout:Building cifs 2023-03-29T11:00:57.416 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/cifs/group.list 2023-03-29T11:00:57.444 INFO:teuthology.orchestra.run.smithi111.stdout:Building generic 2023-03-29T11:00:57.456 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/generic/group.list 2023-03-29T11:00:57.671 INFO:teuthology.orchestra.run.smithi111.stdout:Building shared 2023-03-29T11:00:57.680 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/shared/group.list 2023-03-29T11:00:57.719 INFO:teuthology.orchestra.run.smithi111.stdout:Building xfs 2023-03-29T11:00:57.730 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/xfs/group.list 2023-03-29T11:00:57.951 INFO:teuthology.orchestra.run.smithi111.stdout:Building f2fs 2023-03-29T11:00:57.960 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/f2fs/group.list 2023-03-29T11:00:57.999 INFO:teuthology.orchestra.run.smithi111.stdout:Building udf 2023-03-29T11:00:58.051 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/udf/group.list 2023-03-29T11:00:58.051 INFO:teuthology.orchestra.run.smithi111.stdout:Building ceph 2023-03-29T11:00:58.051 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/ceph/group.list 2023-03-29T11:00:58.076 INFO:teuthology.orchestra.run.smithi111.stdout:Building nfs 2023-03-29T11:00:58.085 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/nfs/group.list 2023-03-29T11:00:58.117 INFO:teuthology.orchestra.run.smithi111.stdout:Building btrfs 2023-03-29T11:00:58.127 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/btrfs/group.list 2023-03-29T11:00:58.255 INFO:teuthology.orchestra.run.smithi111.stdout:Building tmpfs 2023-03-29T11:00:58.264 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/tmpfs/group.list 2023-03-29T11:00:58.294 INFO:teuthology.orchestra.run.smithi111.stdout:Building selftest 2023-03-29T11:00:58.303 INFO:teuthology.orchestra.run.smithi111.stdout: [GROUP] /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/selftest/group.list 2023-03-29T11:00:58.331 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2023-03-29T11:00:58.333 INFO:teuthology.orchestra.run.smithi111.stdout:gmake[1]: Nothing to be done for 'install'. 2023-03-29T11:00:58.334 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2023-03-29T11:00:58.336 INFO:teuthology.orchestra.run.smithi111.stdout:gmake[1]: Nothing to be done for 'install'. 2023-03-29T11:00:58.337 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2023-03-29T11:00:58.340 INFO:teuthology.orchestra.run.smithi111.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2023-03-29T11:00:58.361 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2023-03-29T11:00:58.519 INFO:teuthology.orchestra.run.smithi111.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2023-03-29T11:00:58.540 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2023-03-29T11:00:58.568 INFO:teuthology.orchestra.run.smithi111.stdout:Building log-writes 2023-03-29T11:00:58.575 INFO:teuthology.orchestra.run.smithi111.stdout:Building perf 2023-03-29T11:00:58.582 INFO:teuthology.orchestra.run.smithi111.stdout:Building aio-dio-regress 2023-03-29T11:00:58.590 INFO:teuthology.orchestra.run.smithi111.stdout:Building vfs 2023-03-29T11:00:58.595 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake -C log-writes install 2023-03-29T11:00:58.597 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2023-03-29T11:00:58.618 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2023-03-29T11:00:58.639 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake -C perf install 2023-03-29T11:00:58.641 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2023-03-29T11:00:58.662 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 ResultData.py fio-insert-and-compare.py FioResultDecoder.py FioCompare.py generate-schema.py fio-results.sql /var/lib/xfstests/src/perf 2023-03-29T11:00:58.715 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake -C aio-dio-regress install 2023-03-29T11:00:58.718 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2023-03-29T11:00:58.738 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 aio-dio-append-write-fallocate-race aio-dio-fcntl-race aio-dio-subblock-eof-read aio-dio-append-write-read-race aio-dio-invalidate-readahead aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-dio-cycle-write aio-dio-extend-stat aio-io-setup-with-nonwritable-context-pointer aiocp aio-dio-hole-filling-race aio-dio-invalidate-failure aiodio_sparse2 aio-last-ref-held-by-io aio-dio-eof-race aio-dio-cow-race /var/lib/xfstests/src/aio-dio-regress 2023-03-29T11:00:58.863 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake -C vfs install 2023-03-29T11:00:58.865 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2023-03-29T11:00:58.886 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2023-03-29T11:00:58.915 INFO:teuthology.orchestra.run.smithi111.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2023-03-29T11:00:58.937 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl fiemap-tester t_stripealign open_by_handle t_mmap_dio xfsfind fssum /var/lib/xfstests/src 2023-03-29T11:01:01.614 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py /var/lib/xfstests/src 2023-03-29T11:01:01.812 INFO:teuthology.orchestra.run.smithi111.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2023-03-29T11:01:01.864 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2023-03-29T11:01:01.866 INFO:teuthology.orchestra.run.smithi111.stdout:gmake[1]: Nothing to be done for 'install'. 2023-03-29T11:01:01.867 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2023-03-29T11:01:01.869 INFO:teuthology.orchestra.run.smithi111.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2023-03-29T11:01:01.890 INFO:teuthology.orchestra.run.smithi111.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2023-03-29T11:01:02.204 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2023-03-29T11:01:02.206 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/btrfs/ install 2023-03-29T11:01:02.210 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2023-03-29T11:01:02.231 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 290 291 299 /var/lib/xfstests/tests/btrfs 2023-03-29T11:01:04.109 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2023-03-29T11:01:04.130 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 240.out 095.out 140.out 265.out 208.out 008.out 040.out 104.out 004.out 021.out 233.out 194.out 152.out 055.out 133.out 109.out 188.out 033.out 252.out 270.out 237.out 088.out 257.out 202.out 137.out 210.out 102.out 125.out 037.out 035.out 243.out 192.out 282.out 267.out 178.out 284.out 052.out 062.out 256.out 207.out 228.out 065.out 247.out 273.out 168.out 251.out 036.out 087.out 089.out 258.out 215.out 115.out 015.out 250.out 191.out 046.out 051.out 011.out 177.out 219.out 119.out 244.out 128.out 232.out 028.out 248.out 162.out 147.out 242.out 143.out 276.out 189.out 220.out 148.out 120.out 020.out 002.out 171.out 261.out 135.out 142.out 092.out 072.out 048.out 130.out 038.out 129.out 029.out 229.out 211.out 111.out 155.out 134.out 170.out 260.out 157.out 034.out 161.out 246.out 175.out 216.out 116.out 043.out 110.out 010.out 001.out 114.out 138.out 074.out 277.out 187.out 184.out 153.out 146.out 160.out 291.out 274.out 264.out 071.out 061.out 239.out 047.out 200.out 100.out 139.out 286.out 269.out 039.out 222.out 196.out 098.out 290.out 080.out 070.out 060.out 206.out 106.out 006.out 167.out 217.out 117.out 017.out 159.out 158.out 085.out 057.out 212.out 204.out 059.out 254.out 279.out 281.out 225.out 235.out 007.out 253.out 025.out 124.out 024.out 172.out 063.out 058.out 169.out 042.out 193.out 097.out 271.out 283.out 083.out 199.out 099.out 068.out 045.out 186.out 209.out 053.out 255.out 164.out 154.out 094.out 241.out 084.out 165.out 141.out 214.out 014.out 176.out 278.out 041.out 016.out 156.out 067.out 005.out 223.out 086.out 123.out 179.out 236.out 227.out 032.out 182.out 205.out 136.out 144.out 096.out 198.out 108.out 190.out 280.out 050.out 105.out 185.out 091.out 266.out 285.out 183.out 056.out 201.out 019.out 101.out 079.out 151.out 231.out 081.out 075.out 069.out 163.out 044.out 131.out 213.out 113.out 013.out 031.out 272.out 078.out 173.out 259.out 132.out 093.out 197.out 263.out 127.out 230.out 027.out 299.out 023.out 073.out 030.out 203.out 103.out 003.out 245.out 145.out 249.out 122.out 234.out 022.out 174.out 066.out 009.out 090.out 224.out 181.out 076.out 149.out 064.out 077.out 218.out 118.out 018.out 238.out 107.out 262.out 054.out 150.out 049.out 082.out 226.out 126.out 026.out 166.out 180.out 268.out 275.out 221.out 121.out 112.out 012.out 195.out /var/lib/xfstests/tests/btrfs 2023-03-29T11:01:06.003 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/ceph/ install 2023-03-29T11:01:06.006 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2023-03-29T11:01:06.026 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2023-03-29T11:01:06.072 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2023-03-29T11:01:06.093 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 005.out 003.out 001.out 004.out 002.out /var/lib/xfstests/tests/ceph 2023-03-29T11:01:06.139 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/cifs/ install 2023-03-29T11:01:06.141 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2023-03-29T11:01:06.162 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2023-03-29T11:01:06.183 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2023-03-29T11:01:06.204 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2023-03-29T11:01:06.225 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/ext4/ install 2023-03-29T11:01:06.227 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2023-03-29T11:01:06.248 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2023-03-29T11:01:06.698 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2023-03-29T11:01:06.719 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 001.out.nodelalloc 001.out.default 001.out.nozero 001.cfg 049.out 032.out 039.out 005.out 308.out 026.out 008.out 050.out 040.out 305.out 006.out 021.out 016.out 017.out 012.out 009.out 054.out 033.out 303.out 056.out 004.out 046.out 025.out 057.out 302.out 020.out 002.out 044.out 024.out 013.out 031.out 027.out 306.out 048.out 038.out 052.out 042.out 047.out 029.out 304.out 030.out 003.out 015.out 043.out 053.out 035.out 055.out 059.out 018.out 301.out 010.out 058.out 022.out 007.out 045.out 014.out 036.out 271.out 051.out 011.out 041.out 037.out 307.out 019.out 034.out 028.out 023.out /var/lib/xfstests/tests/ext4 2023-03-29T11:01:07.190 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/f2fs/ install 2023-03-29T11:01:07.192 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2023-03-29T11:01:07.213 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2023-03-29T11:01:07.240 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2023-03-29T11:01:07.260 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2023-03-29T11:01:07.288 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/generic/ install 2023-03-29T11:01:07.293 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2023-03-29T11:01:07.314 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 /var/lib/xfstests/tests/generic 2023-03-29T11:01:12.004 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2023-03-29T11:01:12.025 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 050.out.default 035.out.nfs 050.out.xfsquota 035.out.default 050.out.nojournal 035.cfg 050.cfg 377.out 537.out 443.out 683.out 497.out 033.out 385.out 702.out 696.out 088.out 597.out 097.out 202.out 137.out 210.out 102.out 034.out 110.out 389.out 646.out 592.out 148.out 282.out 267.out 472.out 468.out 628.out 528.out 607.out 428.out 659.out 328.out 207.out 228.out 171.out 183.out 157.out 247.out 273.out 356.out 520.out 474.out 420.out 043.out 509.out 006.out 397.out 460.out 632.out 311.out 532.out 591.out 064.out 719.out 111.out 723.out 519.out 011.out 319.out 246.out 332.out 119.out 423.out 323.out 718.out 638.out 616.out 516.out 416.out 216.out 116.out 358.out 338.out 287.out 238.out 533.out 295.out 138.out 277.out 153.out 160.out 291.out 274.out 061.out 239.out 620.out 047.out 530.out 258.out 379.out 691.out 039.out 173.out 310.out 498.out 665.out 098.out 426.out 690.out 220.out 706.out 606.out 506.out 406.out 306.out 106.out 447.out 217.out 191.out 117.out 688.out 357.out 485.out 182.out 442.out 188.out 026.out 556.out 099.out 177.out 281.out 524.out 174.out 542.out 324.out 655.out 224.out 313.out 124.out 142.out 672.out 278.out 058.out 384.out 296.out 042.out 626.out 593.out 271.out 456.out 399.out 375.out 437.out 263.out 364.out 209.out 255.out 558.out 341.out 094.out 337.out 712.out 241.out 089.out 540.out 458.out 614.out 514.out 025.out 414.out 200.out 314.out 440.out 014.out 342.out 041.out 016.out 340.out 649.out 572.out 576.out 586.out 492.out 549.out 633.out 280.out 240.out 543.out 140.out 065.out 348.out 608.out 568.out 508.out 408.out 308.out 373.out 670.out 604.out 040.out 104.out 349.out 476.out 087.out 233.out 639.out 236.out 186.out 539.out 276.out 711.out 221.out 439.out 376.out 587.out 339.out 294.out 352.out 139.out 481.out 082.out 317.out 552.out 480.out 017.out 120.out 067.out 193.out 452.out 252.out 077.out 167.out 206.out 624.out 663.out 709.out 199.out 651.out 184.out 409.out 466.out 309.out 495.out 725.out 109.out 625.out 283.out 488.out 673.out 390.out 325.out 465.out 225.out 322.out 636.out 125.out 715.out 279.out 615.out 222.out 650.out 152.out 658.out 315.out 563.out 143.out 451.out 551.out 015.out 392.out 682.out 595.out 578.out 642.out 550.out 052.out 062.out 716.out 351.out 298.out 571.out 583.out 450.out 463.out 674.out 251.out 176.out 069.out 644.out 446.out 489.out 487.out 544.out 395.out 515.out 415.out 344.out 215.out 115.out 250.out 244.out 391.out 681.out 594.out 605.out 156.out 536.out 577.out 030.out 436.out 128.out 637.out 336.out 234.out 144.out 505.out 667.out 136.out 363.out 496.out 198.out 108.out 008.out 680.out 705.out 178.out 405.out 305.out 205.out 105.out 005.out 367.out 523.out 517.out 619.out 727.out 685.out 562.out 382.out 570.out 388.out 527.out 158.out 019.out 003.out 695.out 079.out 237.out 151.out 473.out 163.out 127.out 661.out 664.out 187.out 027.out 478.out 612.out 584.out 462.out 623.out 350.out 132.out 037.out 693.out 483.out 499.out 112.out 044.out 660.out 063.out 561.out 343.out 648.out 362.out 387.out 194.out 335.out 721.out 289.out 699.out 284.out 333.out 477.out 621.out 290.out 560.out 419.out 581.out 076.out 707.out 647.out 457.out 493.out 007.out 448.out 262.out 380.out 093.out 479.out 662.out 547.out 192.out 326.out 668.out 413.out 126.out 555.out 580.out 021.out 694.out 347.out 675.out 361.out 564.out 066.out 455.out 432.out 585.out 162.out 686.out 147.out 288.out 197.out 396.out 051.out 145.out 429.out 355.out 381.out 360.out 374.out 261.out 045.out 292.out 304.out 482.out 372.out 046.out 726.out 378.out 534.out 231.out 434.out 503.out 526.out 329.out 229.out 129.out 441.out 371.out 243.out 671.out 134.out 190.out 260.out 226.out 095.out 486.out 223.out 123.out 179.out 669.out 023.out 459.out 032.out 266.out 146.out 398.out 366.out 161.out 427.out 590.out 470.out 484.out 491.out 185.out 656.out 677.out 285.out 631.out 654.out 359.out 501.out 401.out 155.out 431.out 635.out 494.out 687.out 722.out 331.out 613.out 513.out 554.out 131.out 213.out 113.out 013.out 031.out 435.out 630.out 645.out 596.out 272.out 268.out 627.out 573.out 430.out 569.out 454.out 259.out 692.out 449.out 327.out 330.out 643.out 227.out 010.out 230.out 071.out 083.out 130.out 219.out 370.out 603.out 135.out 403.out 303.out 203.out 103.out 345.out 004.out 386.out 521.out 055.out 275.out 461.out 334.out 622.out 724.out 522.out 422.out 394.out 598.out 249.out 122.out 689.out 166.out 022.out 424.out 346.out 684.out 490.out 090.out 353.out 181.out 676.out 149.out 618.out 518.out 018.out 418.out 318.out 024.out 218.out 118.out 575.out 232.out 172.out 168.out 293.out 060.out 469.out 054.out 059.out 049.out 657.out 180.out 270.out 421.out 299.out 321.out 057.out 512.out 412.out 312.out 121.out 538.out 256.out 012.out 195.out 028.out 438.out 471.out 070.out 433.out 720.out 565.out 248.out 445.out 316.out 525.out 242.out 679.out 320.out 629.out 602.out 502.out 402.out 302.out 020.out 529.out 002.out 717.out 617.out 417.out 365.out 092.out 369.out 475.out 072.out 068.out 154.out 038.out 084.out 056.out 611.out 511.out 411.out 029.out 211.out 257.out 170.out 053.out 133.out 286.out 713.out 245.out 175.out 545.out 714.out 559.out 710.out 036.out 610.out 510.out 410.out 548.out 701.out 601.out 301.out 201.out 101.out 001.out 214.out 048.out 589.out 074.out 164.out 697.out 641.out 091.out 081.out 634.out 075.out 653.out 700.out 600.out 500.out 400.out 300.out 368.out 100.out 464.out 354.out 541.out 086.out 269.out 546.out 652.out 698.out 553.out 196.out 444.out 080.out 708.out 609.out 507.out 407.out 307.out 208.out 107.out 009.out 567.out 150.out 557.out 453.out 666.out 078.out 159.out 566.out 582.out 383.out 574.out 085.out 704.out 588.out 212.out 504.out 404.out 189.out 204.out 535.out 425.out 467.out 254.out 531.out 073.out 235.out 703.out 253.out 165.out 264.out 678.out 141.out 265.out 169.out 114.out 640.out 579.out 096.out 297.out 393.out 599.out /var/lib/xfstests/tests/generic 2023-03-29T11:01:16.784 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/nfs/ install 2023-03-29T11:01:16.787 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2023-03-29T11:01:16.807 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2023-03-29T11:01:16.827 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2023-03-29T11:01:16.848 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2023-03-29T11:01:16.869 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/ocfs2/ install 2023-03-29T11:01:16.873 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2023-03-29T11:01:16.892 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2023-03-29T11:01:16.913 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2023-03-29T11:01:16.933 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2023-03-29T11:01:16.954 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/overlay/ install 2023-03-29T11:01:16.957 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2023-03-29T11:01:16.977 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2023-03-29T11:01:17.608 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2023-03-29T11:01:17.628 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 051.out 101.out 034.out 048.out 037.out 020.out 102.out 002.out 060.out 113.out 013.out 031.out 015.out 117.out 077.out 072.out 078.out 063.out 058.out 062.out 038.out 036.out 042.out 052.out 067.out 029.out 010.out 111.out 027.out 112.out 073.out 068.out 030.out 103.out 003.out 017.out 055.out 053.out 116.out 043.out 059.out 014.out 018.out 110.out 057.out 076.out 022.out 106.out 006.out 115.out 045.out 001.out 064.out 114.out 074.out 024.out 071.out 061.out 035.out 011.out 041.out 047.out 100.out 019.out 105.out 104.out 028.out 069.out 023.out 049.out 032.out 039.out 005.out 008.out 065.out 026.out 056.out 070.out 108.out 050.out 040.out 107.out 009.out 075.out 004.out 021.out 046.out 016.out 025.out 066.out 012.out 109.out 007.out 033.out 054.out 044.out /var/lib/xfstests/tests/overlay 2023-03-29T11:01:18.259 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/perf/ install 2023-03-29T11:01:18.262 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2023-03-29T11:01:18.282 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2023-03-29T11:01:18.303 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2023-03-29T11:01:18.324 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2023-03-29T11:01:18.344 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/selftest/ install 2023-03-29T11:01:18.347 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2023-03-29T11:01:18.367 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2023-03-29T11:01:18.420 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2023-03-29T11:01:18.440 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 006.out 005.out 003.out 001.out 004.out 002.out /var/lib/xfstests/tests/selftest 2023-03-29T11:01:18.493 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/shared/ install 2023-03-29T11:01:18.495 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2023-03-29T11:01:18.516 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2023-03-29T11:01:18.549 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2023-03-29T11:01:18.570 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 032.out 298.out 002.out /var/lib/xfstests/tests/shared 2023-03-29T11:01:18.604 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/tmpfs/ install 2023-03-29T11:01:18.607 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2023-03-29T11:01:18.628 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2023-03-29T11:01:18.648 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2023-03-29T11:01:18.669 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2023-03-29T11:01:18.690 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/udf/ install 2023-03-29T11:01:18.692 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2023-03-29T11:01:18.713 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2023-03-29T11:01:18.733 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2023-03-29T11:01:18.754 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2023-03-29T11:01:18.775 INFO:teuthology.orchestra.run.smithi111.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.CzuMqE4LuJxfstests-dev/tests/xfs/ install 2023-03-29T11:01:18.781 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2023-03-29T11:01:18.801 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2023-03-29T11:01:22.845 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:01:22.850 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:01:23.224 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2023-03-29T11:01:23.245 INFO:teuthology.orchestra.run.smithi111.stdout:../../install-sh -o root -g root -m 644 033.out.crc 096.out.external 096.out.internal 033.out.default 071.out.64 216.out.classic 216.out.64mblog 071.out.32 216.cfg 033.cfg 349.out 067.out 537.out 497.out 337.out 385.out 489.out 460.out 088.out 097.out 296.out 137.out 102.out 343.out 574.out 389.out 593.out 243.out 592.out 148.out 284.out 282.out 267.out 072.out 472.out 468.out 528.out 769.out 428.out 328.out 207.out 228.out 128.out 171.out 183.out 157.out 247.out 273.out 520.out 555.out 709.out 109.out 595.out 506.out 313.out 789.out 195.out 732.out 397.out 532.out 191.out 281.out 064.out 719.out 723.out 519.out 011.out 131.out 319.out 467.out 332.out 119.out 775.out 323.out 260.out 095.out 034.out 757.out 161.out 246.out 518.out 416.out 116.out 043.out 358.out 338.out 287.out 238.out 533.out 295.out 138.out 189.out 263.out 765.out 187.out 153.out 160.out 291.out 061.out 239.out 047.out 772.out 530.out 258.out 379.out 322.out 039.out 173.out 526.out 498.out 742.out 098.out 426.out 290.out 220.out 060.out 406.out 306.out 206.out 106.out 006.out 447.out 003.out 217.out 120.out 117.out 357.out 783.out 485.out 589.out 442.out 188.out 030.out 342.out 177.out 174.out 771.out 324.out 224.out 050.out 767.out 124.out 142.out 278.out 063.out 058.out 741.out 384.out 262.out 042.out 377.out 193.out 799.out 271.out 456.out 399.out 770.out 437.out 309.out 441.out 255.out 740.out 356.out 766.out 341.out 094.out 712.out 241.out 089.out 540.out 256.out 514.out 516.out 414.out 314.out 440.out 074.out 056.out 041.out 340.out 156.out 572.out 393.out 586.out 521.out 240.out 549.out 092.out 280.out 543.out 140.out 348.out 249.out 568.out 185.out 508.out 408.out 308.out 208.out 380.out 040.out 104.out 582.out 735.out 383.out 085.out 066.out 588.out 212.out 504.out 404.out 776.out 204.out 535.out 795.out 733.out 487.out 254.out 335.out 387.out 107.out 556.out 253.out 165.out 264.out 784.out 373.out 265.out 169.out 579.out 018.out 793.out 476.out 087.out 739.out 068.out 233.out 752.out 736.out 236.out 773.out 186.out 539.out 279.out 133.out 711.out 439.out 376.out 587.out 339.out 294.out 352.out 139.out 481.out 082.out 390.out 552.out 781.out 017.out 768.out 444.out 452.out 020.out 252.out 585.out 751.out 563.out 167.out 164.out 794.out 199.out 184.out 509.out 409.out 466.out 495.out 785.out 424.out 222.out 283.out 750.out 297.out 325.out 465.out 225.out 744.out 125.out 715.out 581.out 152.out 774.out 122.out 551.out 015.out 792.out 075.out 748.out 578.out 550.out 052.out 062.out 716.out 577.out 571.out 583.out 451.out 425.out 450.out 463.out 542.out 176.out 069.out 762.out 544.out 728.out 395.out 515.out 415.out 315.out 344.out 215.out 115.out 351.out 250.out 391.out 363.out 244.out 475.out 536.out 436.out 480.out 336.out 086.out 144.out 505.out 405.out 205.out 136.out 496.out 198.out 008.out 790.out 178.out 305.out 458.out 105.out 005.out 523.out 761.out 025.out 517.out 423.out 150.out 562.out 382.out 570.out 388.out 527.out 158.out 019.out 079.out 237.out 151.out 760.out 591.out 473.out 163.out 127.out 796.out 232.out 038.out 584.out 462.out 350.out 132.out 037.out 251.out 483.out 499.out 112.out 044.out 561.out 758.out 362.out 194.out 438.out 076.out 289.out 545.out 333.out 477.out 548.out 560.out 419.out 474.out 235.out 461.out 445.out 786.out 007.out 448.out 093.out 479.out 547.out 192.out 221.out 326.out 226.out 078.out 413.out 126.out 026.out 580.out 021.out 488.out 347.out 275.out 361.out 564.out 788.out 455.out 747.out 558.out 162.out 083.out 147.out 288.out 197.out 396.out 446.out 051.out 145.out 143.out 355.out 360.out 101.out 261.out 734.out 755.out 292.out 304.out 482.out 372.out 046.out 272.out 729.out 378.out 484.out 534.out 434.out 503.out 403.out 329.out 334.out 155.out 234.out 134.out 190.out 368.out 722.out 486.out 223.out 123.out 179.out 023.out 032.out 798.out 277.out 146.out 398.out 366.out 427.out 590.out 470.out 491.out 749.out 016.out 227.out 731.out 782.out 374.out 154.out 285.out 359.out 531.out 301.out 431.out 266.out 331.out 791.out 743.out 457.out 231.out 081.out 513.out 554.out 371.out 213.out 730.out 113.out 013.out 031.out 367.out 726.out 182.out 727.out 268.out 754.out 573.out 430.out 454.out 259.out 210.out 449.out 327.out 330.out 777.out 493.out 010.out 230.out 027.out 764.out 130.out 219.out 073.out 370.out 135.out 303.out 203.out 103.out 345.out 004.out 386.out 055.out 035.out 245.out 724.out 522.out 787.out 745.out 422.out 394.out 159.out 432.out 524.out 381.out 166.out 022.out 346.out 045.out 435.out 090.out 353.out 375.out 181.out 566.out 718.out 149.out 443.out 418.out 318.out 024.out 218.out 118.out 575.out 780.out 172.out 168.out 293.out 392.out 469.out 054.out 059.out 049.out 738.out 721.out 180.out 270.out 421.out 299.out 321.out 057.out 512.out 494.out 412.out 312.out 121.out 538.out 797.out 012.out 028.out 070.out 433.out 720.out 565.out 248.out 316.out 525.out 242.out 576.out 420.out 478.out 320.out 077.out 502.out 402.out 302.out 202.out 529.out 002.out 717.out 429.out 417.out 317.out 365.out 492.out 369.out 759.out 778.out 229.out 737.out 084.out 129.out 594.out 511.out 411.out 029.out 311.out 211.out 111.out 257.out 170.out 053.out 065.out 713.out 756.out 175.out 714.out 559.out 710.out 036.out 510.out 274.out 410.out 310.out 501.out 401.out 110.out 201.out 001.out 141.out 214.out 048.out 276.out 114.out 753.out 014.out 459.out 091.out 746.out 800.out 500.out 400.out 300.out 200.out 100.out 464.out 354.out 471.out 286.out 541.out 779.out 269.out 546.out 099.out 553.out 763.out 298.out 196.out 490.out 080.out 708.out 507.out 407.out 307.out 209.out 108.out 009.out 567.out 557.out 453.out 364.out 725.out /var/lib/xfstests/tests/xfs 2023-03-29T11:01:27.737 INFO:teuthology.orchestra.run.smithi111.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2023-03-29T11:01:27.758 INFO:teuthology.orchestra.run.smithi111.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2023-03-29T11:01:27.778 INFO:teuthology.orchestra.run.smithi111.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2023-03-29T11:01:27.799 INFO:teuthology.orchestra.run.smithi111.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2023-03-29T11:01:27.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T11:01:27.823 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec ls tests/generic) 2023-03-29T11:01:27.866 INFO:teuthology.orchestra.run.smithi111.stdout:001 2023-03-29T11:01:27.866 INFO:teuthology.orchestra.run.smithi111.stdout:001.out 2023-03-29T11:01:27.866 INFO:teuthology.orchestra.run.smithi111.stdout:002 2023-03-29T11:01:27.866 INFO:teuthology.orchestra.run.smithi111.stdout:002.out 2023-03-29T11:01:27.866 INFO:teuthology.orchestra.run.smithi111.stdout:003 2023-03-29T11:01:27.866 INFO:teuthology.orchestra.run.smithi111.stdout:003.out 2023-03-29T11:01:27.866 INFO:teuthology.orchestra.run.smithi111.stdout:004 2023-03-29T11:01:27.866 INFO:teuthology.orchestra.run.smithi111.stdout:004.out 2023-03-29T11:01:27.866 INFO:teuthology.orchestra.run.smithi111.stdout:005 2023-03-29T11:01:27.866 INFO:teuthology.orchestra.run.smithi111.stdout:005.out 2023-03-29T11:01:27.866 INFO:teuthology.orchestra.run.smithi111.stdout:006 2023-03-29T11:01:27.866 INFO:teuthology.orchestra.run.smithi111.stdout:006.out 2023-03-29T11:01:27.866 INFO:teuthology.orchestra.run.smithi111.stdout:007 2023-03-29T11:01:27.866 INFO:teuthology.orchestra.run.smithi111.stdout:007.out 2023-03-29T11:01:27.866 INFO:teuthology.orchestra.run.smithi111.stdout:008 2023-03-29T11:01:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:008.out 2023-03-29T11:01:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:009 2023-03-29T11:01:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:009.out 2023-03-29T11:01:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:010 2023-03-29T11:01:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:010.out 2023-03-29T11:01:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:011 2023-03-29T11:01:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:011.out 2023-03-29T11:01:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:012 2023-03-29T11:01:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:012.out 2023-03-29T11:01:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:013 2023-03-29T11:01:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:013.out 2023-03-29T11:01:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:014 2023-03-29T11:01:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:014.out 2023-03-29T11:01:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:015 2023-03-29T11:01:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:015.out 2023-03-29T11:01:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:016 2023-03-29T11:01:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:016.out 2023-03-29T11:01:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:017 2023-03-29T11:01:27.867 INFO:teuthology.orchestra.run.smithi111.stdout:017.out 2023-03-29T11:01:27.868 INFO:teuthology.orchestra.run.smithi111.stdout:018 2023-03-29T11:01:27.868 INFO:teuthology.orchestra.run.smithi111.stdout:018.out 2023-03-29T11:01:27.868 INFO:teuthology.orchestra.run.smithi111.stdout:019 2023-03-29T11:01:27.868 INFO:teuthology.orchestra.run.smithi111.stdout:019.out 2023-03-29T11:01:27.868 INFO:teuthology.orchestra.run.smithi111.stdout:020 2023-03-29T11:01:27.868 INFO:teuthology.orchestra.run.smithi111.stdout:020.out 2023-03-29T11:01:27.868 INFO:teuthology.orchestra.run.smithi111.stdout:021 2023-03-29T11:01:27.868 INFO:teuthology.orchestra.run.smithi111.stdout:021.out 2023-03-29T11:01:27.868 INFO:teuthology.orchestra.run.smithi111.stdout:022 2023-03-29T11:01:27.868 INFO:teuthology.orchestra.run.smithi111.stdout:022.out 2023-03-29T11:01:27.868 INFO:teuthology.orchestra.run.smithi111.stdout:023 2023-03-29T11:01:27.868 INFO:teuthology.orchestra.run.smithi111.stdout:023.out 2023-03-29T11:01:27.868 INFO:teuthology.orchestra.run.smithi111.stdout:024 2023-03-29T11:01:27.868 INFO:teuthology.orchestra.run.smithi111.stdout:024.out 2023-03-29T11:01:27.868 INFO:teuthology.orchestra.run.smithi111.stdout:025 2023-03-29T11:01:27.868 INFO:teuthology.orchestra.run.smithi111.stdout:025.out 2023-03-29T11:01:27.868 INFO:teuthology.orchestra.run.smithi111.stdout:026 2023-03-29T11:01:27.868 INFO:teuthology.orchestra.run.smithi111.stdout:026.out 2023-03-29T11:01:27.869 INFO:teuthology.orchestra.run.smithi111.stdout:027 2023-03-29T11:01:27.869 INFO:teuthology.orchestra.run.smithi111.stdout:027.out 2023-03-29T11:01:27.869 INFO:teuthology.orchestra.run.smithi111.stdout:028 2023-03-29T11:01:27.869 INFO:teuthology.orchestra.run.smithi111.stdout:028.out 2023-03-29T11:01:27.869 INFO:teuthology.orchestra.run.smithi111.stdout:029 2023-03-29T11:01:27.869 INFO:teuthology.orchestra.run.smithi111.stdout:029.out 2023-03-29T11:01:27.869 INFO:teuthology.orchestra.run.smithi111.stdout:030 2023-03-29T11:01:27.869 INFO:teuthology.orchestra.run.smithi111.stdout:030.out 2023-03-29T11:01:27.869 INFO:teuthology.orchestra.run.smithi111.stdout:031 2023-03-29T11:01:27.869 INFO:teuthology.orchestra.run.smithi111.stdout:031.out 2023-03-29T11:01:27.869 INFO:teuthology.orchestra.run.smithi111.stdout:032 2023-03-29T11:01:27.869 INFO:teuthology.orchestra.run.smithi111.stdout:032.out 2023-03-29T11:01:27.869 INFO:teuthology.orchestra.run.smithi111.stdout:033 2023-03-29T11:01:27.869 INFO:teuthology.orchestra.run.smithi111.stdout:033.out 2023-03-29T11:01:27.869 INFO:teuthology.orchestra.run.smithi111.stdout:034 2023-03-29T11:01:27.869 INFO:teuthology.orchestra.run.smithi111.stdout:034.out 2023-03-29T11:01:27.869 INFO:teuthology.orchestra.run.smithi111.stdout:035 2023-03-29T11:01:27.869 INFO:teuthology.orchestra.run.smithi111.stdout:035.cfg 2023-03-29T11:01:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:035.out.default 2023-03-29T11:01:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:035.out.nfs 2023-03-29T11:01:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:036 2023-03-29T11:01:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:036.out 2023-03-29T11:01:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:037 2023-03-29T11:01:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:037.out 2023-03-29T11:01:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:038 2023-03-29T11:01:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:038.out 2023-03-29T11:01:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:039 2023-03-29T11:01:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:039.out 2023-03-29T11:01:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:040 2023-03-29T11:01:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:040.out 2023-03-29T11:01:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:041 2023-03-29T11:01:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:041.out 2023-03-29T11:01:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:042 2023-03-29T11:01:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:042.out 2023-03-29T11:01:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:043 2023-03-29T11:01:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:043.out 2023-03-29T11:01:27.870 INFO:teuthology.orchestra.run.smithi111.stdout:044 2023-03-29T11:01:27.871 INFO:teuthology.orchestra.run.smithi111.stdout:044.out 2023-03-29T11:01:27.871 INFO:teuthology.orchestra.run.smithi111.stdout:045 2023-03-29T11:01:27.871 INFO:teuthology.orchestra.run.smithi111.stdout:045.out 2023-03-29T11:01:27.871 INFO:teuthology.orchestra.run.smithi111.stdout:046 2023-03-29T11:01:27.871 INFO:teuthology.orchestra.run.smithi111.stdout:046.out 2023-03-29T11:01:27.871 INFO:teuthology.orchestra.run.smithi111.stdout:047 2023-03-29T11:01:27.871 INFO:teuthology.orchestra.run.smithi111.stdout:047.out 2023-03-29T11:01:27.871 INFO:teuthology.orchestra.run.smithi111.stdout:048 2023-03-29T11:01:27.871 INFO:teuthology.orchestra.run.smithi111.stdout:048.out 2023-03-29T11:01:27.871 INFO:teuthology.orchestra.run.smithi111.stdout:049 2023-03-29T11:01:27.871 INFO:teuthology.orchestra.run.smithi111.stdout:049.out 2023-03-29T11:01:27.871 INFO:teuthology.orchestra.run.smithi111.stdout:050 2023-03-29T11:01:27.871 INFO:teuthology.orchestra.run.smithi111.stdout:050.cfg 2023-03-29T11:01:27.871 INFO:teuthology.orchestra.run.smithi111.stdout:050.out.default 2023-03-29T11:01:27.871 INFO:teuthology.orchestra.run.smithi111.stdout:050.out.nojournal 2023-03-29T11:01:27.871 INFO:teuthology.orchestra.run.smithi111.stdout:050.out.xfsquota 2023-03-29T11:01:27.871 INFO:teuthology.orchestra.run.smithi111.stdout:051 2023-03-29T11:01:27.871 INFO:teuthology.orchestra.run.smithi111.stdout:051.out 2023-03-29T11:01:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:052 2023-03-29T11:01:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:052.out 2023-03-29T11:01:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:053 2023-03-29T11:01:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:053.out 2023-03-29T11:01:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:054 2023-03-29T11:01:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:054.out 2023-03-29T11:01:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:055 2023-03-29T11:01:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:055.out 2023-03-29T11:01:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:056 2023-03-29T11:01:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:056.out 2023-03-29T11:01:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:057 2023-03-29T11:01:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:057.out 2023-03-29T11:01:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:058 2023-03-29T11:01:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:058.out 2023-03-29T11:01:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:059 2023-03-29T11:01:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:059.out 2023-03-29T11:01:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:060 2023-03-29T11:01:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:060.out 2023-03-29T11:01:27.872 INFO:teuthology.orchestra.run.smithi111.stdout:061 2023-03-29T11:01:27.873 INFO:teuthology.orchestra.run.smithi111.stdout:061.out 2023-03-29T11:01:27.873 INFO:teuthology.orchestra.run.smithi111.stdout:062 2023-03-29T11:01:27.873 INFO:teuthology.orchestra.run.smithi111.stdout:062.out 2023-03-29T11:01:27.873 INFO:teuthology.orchestra.run.smithi111.stdout:063 2023-03-29T11:01:27.873 INFO:teuthology.orchestra.run.smithi111.stdout:063.out 2023-03-29T11:01:27.873 INFO:teuthology.orchestra.run.smithi111.stdout:064 2023-03-29T11:01:27.873 INFO:teuthology.orchestra.run.smithi111.stdout:064.out 2023-03-29T11:01:27.873 INFO:teuthology.orchestra.run.smithi111.stdout:065 2023-03-29T11:01:27.873 INFO:teuthology.orchestra.run.smithi111.stdout:065.out 2023-03-29T11:01:27.873 INFO:teuthology.orchestra.run.smithi111.stdout:066 2023-03-29T11:01:27.873 INFO:teuthology.orchestra.run.smithi111.stdout:066.out 2023-03-29T11:01:27.873 INFO:teuthology.orchestra.run.smithi111.stdout:067 2023-03-29T11:01:27.873 INFO:teuthology.orchestra.run.smithi111.stdout:067.out 2023-03-29T11:01:27.873 INFO:teuthology.orchestra.run.smithi111.stdout:068 2023-03-29T11:01:27.873 INFO:teuthology.orchestra.run.smithi111.stdout:068.out 2023-03-29T11:01:27.873 INFO:teuthology.orchestra.run.smithi111.stdout:069 2023-03-29T11:01:27.873 INFO:teuthology.orchestra.run.smithi111.stdout:069.out 2023-03-29T11:01:27.873 INFO:teuthology.orchestra.run.smithi111.stdout:070 2023-03-29T11:01:27.874 INFO:teuthology.orchestra.run.smithi111.stdout:070.out 2023-03-29T11:01:27.874 INFO:teuthology.orchestra.run.smithi111.stdout:071 2023-03-29T11:01:27.874 INFO:teuthology.orchestra.run.smithi111.stdout:071.out 2023-03-29T11:01:27.874 INFO:teuthology.orchestra.run.smithi111.stdout:072 2023-03-29T11:01:27.874 INFO:teuthology.orchestra.run.smithi111.stdout:072.out 2023-03-29T11:01:27.874 INFO:teuthology.orchestra.run.smithi111.stdout:073 2023-03-29T11:01:27.874 INFO:teuthology.orchestra.run.smithi111.stdout:073.out 2023-03-29T11:01:27.874 INFO:teuthology.orchestra.run.smithi111.stdout:074 2023-03-29T11:01:27.874 INFO:teuthology.orchestra.run.smithi111.stdout:074.out 2023-03-29T11:01:27.874 INFO:teuthology.orchestra.run.smithi111.stdout:075 2023-03-29T11:01:27.874 INFO:teuthology.orchestra.run.smithi111.stdout:075.out 2023-03-29T11:01:27.874 INFO:teuthology.orchestra.run.smithi111.stdout:076 2023-03-29T11:01:27.874 INFO:teuthology.orchestra.run.smithi111.stdout:076.out 2023-03-29T11:01:27.874 INFO:teuthology.orchestra.run.smithi111.stdout:077 2023-03-29T11:01:27.874 INFO:teuthology.orchestra.run.smithi111.stdout:077.out 2023-03-29T11:01:27.874 INFO:teuthology.orchestra.run.smithi111.stdout:078 2023-03-29T11:01:27.874 INFO:teuthology.orchestra.run.smithi111.stdout:078.out 2023-03-29T11:01:27.874 INFO:teuthology.orchestra.run.smithi111.stdout:079 2023-03-29T11:01:27.875 INFO:teuthology.orchestra.run.smithi111.stdout:079.out 2023-03-29T11:01:27.875 INFO:teuthology.orchestra.run.smithi111.stdout:080 2023-03-29T11:01:27.875 INFO:teuthology.orchestra.run.smithi111.stdout:080.out 2023-03-29T11:01:27.875 INFO:teuthology.orchestra.run.smithi111.stdout:081 2023-03-29T11:01:27.875 INFO:teuthology.orchestra.run.smithi111.stdout:081.out 2023-03-29T11:01:27.875 INFO:teuthology.orchestra.run.smithi111.stdout:082 2023-03-29T11:01:27.875 INFO:teuthology.orchestra.run.smithi111.stdout:082.out 2023-03-29T11:01:27.875 INFO:teuthology.orchestra.run.smithi111.stdout:083 2023-03-29T11:01:27.875 INFO:teuthology.orchestra.run.smithi111.stdout:083.out 2023-03-29T11:01:27.875 INFO:teuthology.orchestra.run.smithi111.stdout:084 2023-03-29T11:01:27.875 INFO:teuthology.orchestra.run.smithi111.stdout:084.out 2023-03-29T11:01:27.875 INFO:teuthology.orchestra.run.smithi111.stdout:085 2023-03-29T11:01:27.875 INFO:teuthology.orchestra.run.smithi111.stdout:085.out 2023-03-29T11:01:27.875 INFO:teuthology.orchestra.run.smithi111.stdout:086 2023-03-29T11:01:27.875 INFO:teuthology.orchestra.run.smithi111.stdout:086.out 2023-03-29T11:01:27.875 INFO:teuthology.orchestra.run.smithi111.stdout:087 2023-03-29T11:01:27.875 INFO:teuthology.orchestra.run.smithi111.stdout:087.out 2023-03-29T11:01:27.875 INFO:teuthology.orchestra.run.smithi111.stdout:088 2023-03-29T11:01:27.875 INFO:teuthology.orchestra.run.smithi111.stdout:088.out 2023-03-29T11:01:27.876 INFO:teuthology.orchestra.run.smithi111.stdout:089 2023-03-29T11:01:27.876 INFO:teuthology.orchestra.run.smithi111.stdout:089.out 2023-03-29T11:01:27.876 INFO:teuthology.orchestra.run.smithi111.stdout:090 2023-03-29T11:01:27.876 INFO:teuthology.orchestra.run.smithi111.stdout:090.out 2023-03-29T11:01:27.876 INFO:teuthology.orchestra.run.smithi111.stdout:091 2023-03-29T11:01:27.876 INFO:teuthology.orchestra.run.smithi111.stdout:091.out 2023-03-29T11:01:27.876 INFO:teuthology.orchestra.run.smithi111.stdout:092 2023-03-29T11:01:27.876 INFO:teuthology.orchestra.run.smithi111.stdout:092.out 2023-03-29T11:01:27.876 INFO:teuthology.orchestra.run.smithi111.stdout:093 2023-03-29T11:01:27.876 INFO:teuthology.orchestra.run.smithi111.stdout:093.out 2023-03-29T11:01:27.876 INFO:teuthology.orchestra.run.smithi111.stdout:094 2023-03-29T11:01:27.876 INFO:teuthology.orchestra.run.smithi111.stdout:094.out 2023-03-29T11:01:27.876 INFO:teuthology.orchestra.run.smithi111.stdout:095 2023-03-29T11:01:27.876 INFO:teuthology.orchestra.run.smithi111.stdout:095.out 2023-03-29T11:01:27.876 INFO:teuthology.orchestra.run.smithi111.stdout:096 2023-03-29T11:01:27.876 INFO:teuthology.orchestra.run.smithi111.stdout:096.out 2023-03-29T11:01:27.876 INFO:teuthology.orchestra.run.smithi111.stdout:097 2023-03-29T11:01:27.876 INFO:teuthology.orchestra.run.smithi111.stdout:097.out 2023-03-29T11:01:27.877 INFO:teuthology.orchestra.run.smithi111.stdout:098 2023-03-29T11:01:27.877 INFO:teuthology.orchestra.run.smithi111.stdout:098.out 2023-03-29T11:01:27.877 INFO:teuthology.orchestra.run.smithi111.stdout:099 2023-03-29T11:01:27.877 INFO:teuthology.orchestra.run.smithi111.stdout:099.out 2023-03-29T11:01:27.877 INFO:teuthology.orchestra.run.smithi111.stdout:100 2023-03-29T11:01:27.877 INFO:teuthology.orchestra.run.smithi111.stdout:100.out 2023-03-29T11:01:27.877 INFO:teuthology.orchestra.run.smithi111.stdout:101 2023-03-29T11:01:27.877 INFO:teuthology.orchestra.run.smithi111.stdout:101.out 2023-03-29T11:01:27.877 INFO:teuthology.orchestra.run.smithi111.stdout:102 2023-03-29T11:01:27.877 INFO:teuthology.orchestra.run.smithi111.stdout:102.out 2023-03-29T11:01:27.877 INFO:teuthology.orchestra.run.smithi111.stdout:103 2023-03-29T11:01:27.877 INFO:teuthology.orchestra.run.smithi111.stdout:103.out 2023-03-29T11:01:27.877 INFO:teuthology.orchestra.run.smithi111.stdout:104 2023-03-29T11:01:27.877 INFO:teuthology.orchestra.run.smithi111.stdout:104.out 2023-03-29T11:01:27.877 INFO:teuthology.orchestra.run.smithi111.stdout:105 2023-03-29T11:01:27.877 INFO:teuthology.orchestra.run.smithi111.stdout:105.out 2023-03-29T11:01:27.877 INFO:teuthology.orchestra.run.smithi111.stdout:106 2023-03-29T11:01:27.877 INFO:teuthology.orchestra.run.smithi111.stdout:106.out 2023-03-29T11:01:27.878 INFO:teuthology.orchestra.run.smithi111.stdout:107 2023-03-29T11:01:27.878 INFO:teuthology.orchestra.run.smithi111.stdout:107.out 2023-03-29T11:01:27.878 INFO:teuthology.orchestra.run.smithi111.stdout:108 2023-03-29T11:01:27.878 INFO:teuthology.orchestra.run.smithi111.stdout:108.out 2023-03-29T11:01:27.878 INFO:teuthology.orchestra.run.smithi111.stdout:109 2023-03-29T11:01:27.878 INFO:teuthology.orchestra.run.smithi111.stdout:109.out 2023-03-29T11:01:27.878 INFO:teuthology.orchestra.run.smithi111.stdout:110 2023-03-29T11:01:27.878 INFO:teuthology.orchestra.run.smithi111.stdout:110.out 2023-03-29T11:01:27.878 INFO:teuthology.orchestra.run.smithi111.stdout:111 2023-03-29T11:01:27.878 INFO:teuthology.orchestra.run.smithi111.stdout:111.out 2023-03-29T11:01:27.878 INFO:teuthology.orchestra.run.smithi111.stdout:112 2023-03-29T11:01:27.878 INFO:teuthology.orchestra.run.smithi111.stdout:112.out 2023-03-29T11:01:27.878 INFO:teuthology.orchestra.run.smithi111.stdout:113 2023-03-29T11:01:27.878 INFO:teuthology.orchestra.run.smithi111.stdout:113.out 2023-03-29T11:01:27.878 INFO:teuthology.orchestra.run.smithi111.stdout:114 2023-03-29T11:01:27.878 INFO:teuthology.orchestra.run.smithi111.stdout:114.out 2023-03-29T11:01:27.878 INFO:teuthology.orchestra.run.smithi111.stdout:115 2023-03-29T11:01:27.878 INFO:teuthology.orchestra.run.smithi111.stdout:115.out 2023-03-29T11:01:27.879 INFO:teuthology.orchestra.run.smithi111.stdout:116 2023-03-29T11:01:27.879 INFO:teuthology.orchestra.run.smithi111.stdout:116.out 2023-03-29T11:01:27.879 INFO:teuthology.orchestra.run.smithi111.stdout:117 2023-03-29T11:01:27.879 INFO:teuthology.orchestra.run.smithi111.stdout:117.out 2023-03-29T11:01:27.879 INFO:teuthology.orchestra.run.smithi111.stdout:118 2023-03-29T11:01:27.879 INFO:teuthology.orchestra.run.smithi111.stdout:118.out 2023-03-29T11:01:27.879 INFO:teuthology.orchestra.run.smithi111.stdout:119 2023-03-29T11:01:27.879 INFO:teuthology.orchestra.run.smithi111.stdout:119.out 2023-03-29T11:01:27.879 INFO:teuthology.orchestra.run.smithi111.stdout:120 2023-03-29T11:01:27.879 INFO:teuthology.orchestra.run.smithi111.stdout:120.out 2023-03-29T11:01:27.880 INFO:teuthology.orchestra.run.smithi111.stdout:121 2023-03-29T11:01:27.880 INFO:teuthology.orchestra.run.smithi111.stdout:121.out 2023-03-29T11:01:27.880 INFO:teuthology.orchestra.run.smithi111.stdout:122 2023-03-29T11:01:27.880 INFO:teuthology.orchestra.run.smithi111.stdout:122.out 2023-03-29T11:01:27.880 INFO:teuthology.orchestra.run.smithi111.stdout:123 2023-03-29T11:01:27.880 INFO:teuthology.orchestra.run.smithi111.stdout:123.out 2023-03-29T11:01:27.880 INFO:teuthology.orchestra.run.smithi111.stdout:124 2023-03-29T11:01:27.880 INFO:teuthology.orchestra.run.smithi111.stdout:124.out 2023-03-29T11:01:27.880 INFO:teuthology.orchestra.run.smithi111.stdout:125 2023-03-29T11:01:27.880 INFO:teuthology.orchestra.run.smithi111.stdout:125.out 2023-03-29T11:01:27.880 INFO:teuthology.orchestra.run.smithi111.stdout:126 2023-03-29T11:01:27.880 INFO:teuthology.orchestra.run.smithi111.stdout:126.out 2023-03-29T11:01:27.880 INFO:teuthology.orchestra.run.smithi111.stdout:127 2023-03-29T11:01:27.880 INFO:teuthology.orchestra.run.smithi111.stdout:127.out 2023-03-29T11:01:27.880 INFO:teuthology.orchestra.run.smithi111.stdout:128 2023-03-29T11:01:27.881 INFO:teuthology.orchestra.run.smithi111.stdout:128.out 2023-03-29T11:01:27.881 INFO:teuthology.orchestra.run.smithi111.stdout:129 2023-03-29T11:01:27.881 INFO:teuthology.orchestra.run.smithi111.stdout:129.out 2023-03-29T11:01:27.881 INFO:teuthology.orchestra.run.smithi111.stdout:130 2023-03-29T11:01:27.881 INFO:teuthology.orchestra.run.smithi111.stdout:130.out 2023-03-29T11:01:27.881 INFO:teuthology.orchestra.run.smithi111.stdout:131 2023-03-29T11:01:27.881 INFO:teuthology.orchestra.run.smithi111.stdout:131.out 2023-03-29T11:01:27.881 INFO:teuthology.orchestra.run.smithi111.stdout:132 2023-03-29T11:01:27.881 INFO:teuthology.orchestra.run.smithi111.stdout:132.out 2023-03-29T11:01:27.881 INFO:teuthology.orchestra.run.smithi111.stdout:133 2023-03-29T11:01:27.881 INFO:teuthology.orchestra.run.smithi111.stdout:133.out 2023-03-29T11:01:27.881 INFO:teuthology.orchestra.run.smithi111.stdout:134 2023-03-29T11:01:27.881 INFO:teuthology.orchestra.run.smithi111.stdout:134.out 2023-03-29T11:01:27.881 INFO:teuthology.orchestra.run.smithi111.stdout:135 2023-03-29T11:01:27.881 INFO:teuthology.orchestra.run.smithi111.stdout:135.out 2023-03-29T11:01:27.881 INFO:teuthology.orchestra.run.smithi111.stdout:136 2023-03-29T11:01:27.881 INFO:teuthology.orchestra.run.smithi111.stdout:136.out 2023-03-29T11:01:27.882 INFO:teuthology.orchestra.run.smithi111.stdout:137 2023-03-29T11:01:27.882 INFO:teuthology.orchestra.run.smithi111.stdout:137.out 2023-03-29T11:01:27.882 INFO:teuthology.orchestra.run.smithi111.stdout:138 2023-03-29T11:01:27.882 INFO:teuthology.orchestra.run.smithi111.stdout:138.out 2023-03-29T11:01:27.882 INFO:teuthology.orchestra.run.smithi111.stdout:139 2023-03-29T11:01:27.882 INFO:teuthology.orchestra.run.smithi111.stdout:139.out 2023-03-29T11:01:27.882 INFO:teuthology.orchestra.run.smithi111.stdout:140 2023-03-29T11:01:27.882 INFO:teuthology.orchestra.run.smithi111.stdout:140.out 2023-03-29T11:01:27.882 INFO:teuthology.orchestra.run.smithi111.stdout:141 2023-03-29T11:01:27.882 INFO:teuthology.orchestra.run.smithi111.stdout:141.out 2023-03-29T11:01:27.882 INFO:teuthology.orchestra.run.smithi111.stdout:142 2023-03-29T11:01:27.882 INFO:teuthology.orchestra.run.smithi111.stdout:142.out 2023-03-29T11:01:27.882 INFO:teuthology.orchestra.run.smithi111.stdout:143 2023-03-29T11:01:27.882 INFO:teuthology.orchestra.run.smithi111.stdout:143.out 2023-03-29T11:01:27.882 INFO:teuthology.orchestra.run.smithi111.stdout:144 2023-03-29T11:01:27.882 INFO:teuthology.orchestra.run.smithi111.stdout:144.out 2023-03-29T11:01:27.882 INFO:teuthology.orchestra.run.smithi111.stdout:145 2023-03-29T11:01:27.882 INFO:teuthology.orchestra.run.smithi111.stdout:145.out 2023-03-29T11:01:27.883 INFO:teuthology.orchestra.run.smithi111.stdout:146 2023-03-29T11:01:27.883 INFO:teuthology.orchestra.run.smithi111.stdout:146.out 2023-03-29T11:01:27.883 INFO:teuthology.orchestra.run.smithi111.stdout:147 2023-03-29T11:01:27.883 INFO:teuthology.orchestra.run.smithi111.stdout:147.out 2023-03-29T11:01:27.883 INFO:teuthology.orchestra.run.smithi111.stdout:148 2023-03-29T11:01:27.883 INFO:teuthology.orchestra.run.smithi111.stdout:148.out 2023-03-29T11:01:27.883 INFO:teuthology.orchestra.run.smithi111.stdout:149 2023-03-29T11:01:27.883 INFO:teuthology.orchestra.run.smithi111.stdout:149.out 2023-03-29T11:01:27.883 INFO:teuthology.orchestra.run.smithi111.stdout:150 2023-03-29T11:01:27.883 INFO:teuthology.orchestra.run.smithi111.stdout:150.out 2023-03-29T11:01:27.883 INFO:teuthology.orchestra.run.smithi111.stdout:151 2023-03-29T11:01:27.883 INFO:teuthology.orchestra.run.smithi111.stdout:151.out 2023-03-29T11:01:27.883 INFO:teuthology.orchestra.run.smithi111.stdout:152 2023-03-29T11:01:27.883 INFO:teuthology.orchestra.run.smithi111.stdout:152.out 2023-03-29T11:01:27.883 INFO:teuthology.orchestra.run.smithi111.stdout:153 2023-03-29T11:01:27.883 INFO:teuthology.orchestra.run.smithi111.stdout:153.out 2023-03-29T11:01:27.883 INFO:teuthology.orchestra.run.smithi111.stdout:154 2023-03-29T11:01:27.883 INFO:teuthology.orchestra.run.smithi111.stdout:154.out 2023-03-29T11:01:27.884 INFO:teuthology.orchestra.run.smithi111.stdout:155 2023-03-29T11:01:27.884 INFO:teuthology.orchestra.run.smithi111.stdout:155.out 2023-03-29T11:01:27.884 INFO:teuthology.orchestra.run.smithi111.stdout:156 2023-03-29T11:01:27.884 INFO:teuthology.orchestra.run.smithi111.stdout:156.out 2023-03-29T11:01:27.884 INFO:teuthology.orchestra.run.smithi111.stdout:157 2023-03-29T11:01:27.884 INFO:teuthology.orchestra.run.smithi111.stdout:157.out 2023-03-29T11:01:27.884 INFO:teuthology.orchestra.run.smithi111.stdout:158 2023-03-29T11:01:27.884 INFO:teuthology.orchestra.run.smithi111.stdout:158.out 2023-03-29T11:01:27.884 INFO:teuthology.orchestra.run.smithi111.stdout:159 2023-03-29T11:01:27.884 INFO:teuthology.orchestra.run.smithi111.stdout:159.out 2023-03-29T11:01:27.884 INFO:teuthology.orchestra.run.smithi111.stdout:160 2023-03-29T11:01:27.884 INFO:teuthology.orchestra.run.smithi111.stdout:160.out 2023-03-29T11:01:27.884 INFO:teuthology.orchestra.run.smithi111.stdout:161 2023-03-29T11:01:27.884 INFO:teuthology.orchestra.run.smithi111.stdout:161.out 2023-03-29T11:01:27.884 INFO:teuthology.orchestra.run.smithi111.stdout:162 2023-03-29T11:01:27.884 INFO:teuthology.orchestra.run.smithi111.stdout:162.out 2023-03-29T11:01:27.884 INFO:teuthology.orchestra.run.smithi111.stdout:163 2023-03-29T11:01:27.885 INFO:teuthology.orchestra.run.smithi111.stdout:163.out 2023-03-29T11:01:27.885 INFO:teuthology.orchestra.run.smithi111.stdout:164 2023-03-29T11:01:27.885 INFO:teuthology.orchestra.run.smithi111.stdout:164.out 2023-03-29T11:01:27.885 INFO:teuthology.orchestra.run.smithi111.stdout:165 2023-03-29T11:01:27.885 INFO:teuthology.orchestra.run.smithi111.stdout:165.out 2023-03-29T11:01:27.885 INFO:teuthology.orchestra.run.smithi111.stdout:166 2023-03-29T11:01:27.885 INFO:teuthology.orchestra.run.smithi111.stdout:166.out 2023-03-29T11:01:27.885 INFO:teuthology.orchestra.run.smithi111.stdout:167 2023-03-29T11:01:27.885 INFO:teuthology.orchestra.run.smithi111.stdout:167.out 2023-03-29T11:01:27.885 INFO:teuthology.orchestra.run.smithi111.stdout:168 2023-03-29T11:01:27.885 INFO:teuthology.orchestra.run.smithi111.stdout:168.out 2023-03-29T11:01:27.885 INFO:teuthology.orchestra.run.smithi111.stdout:169 2023-03-29T11:01:27.885 INFO:teuthology.orchestra.run.smithi111.stdout:169.out 2023-03-29T11:01:27.885 INFO:teuthology.orchestra.run.smithi111.stdout:170 2023-03-29T11:01:27.885 INFO:teuthology.orchestra.run.smithi111.stdout:170.out 2023-03-29T11:01:27.885 INFO:teuthology.orchestra.run.smithi111.stdout:171 2023-03-29T11:01:27.885 INFO:teuthology.orchestra.run.smithi111.stdout:171.out 2023-03-29T11:01:27.885 INFO:teuthology.orchestra.run.smithi111.stdout:172 2023-03-29T11:01:27.885 INFO:teuthology.orchestra.run.smithi111.stdout:172.out 2023-03-29T11:01:27.886 INFO:teuthology.orchestra.run.smithi111.stdout:173 2023-03-29T11:01:27.886 INFO:teuthology.orchestra.run.smithi111.stdout:173.out 2023-03-29T11:01:27.886 INFO:teuthology.orchestra.run.smithi111.stdout:174 2023-03-29T11:01:27.886 INFO:teuthology.orchestra.run.smithi111.stdout:174.out 2023-03-29T11:01:27.886 INFO:teuthology.orchestra.run.smithi111.stdout:175 2023-03-29T11:01:27.886 INFO:teuthology.orchestra.run.smithi111.stdout:175.out 2023-03-29T11:01:27.886 INFO:teuthology.orchestra.run.smithi111.stdout:176 2023-03-29T11:01:27.886 INFO:teuthology.orchestra.run.smithi111.stdout:176.out 2023-03-29T11:01:27.886 INFO:teuthology.orchestra.run.smithi111.stdout:177 2023-03-29T11:01:27.886 INFO:teuthology.orchestra.run.smithi111.stdout:177.out 2023-03-29T11:01:27.886 INFO:teuthology.orchestra.run.smithi111.stdout:178 2023-03-29T11:01:27.886 INFO:teuthology.orchestra.run.smithi111.stdout:178.out 2023-03-29T11:01:27.886 INFO:teuthology.orchestra.run.smithi111.stdout:179 2023-03-29T11:01:27.886 INFO:teuthology.orchestra.run.smithi111.stdout:179.out 2023-03-29T11:01:27.886 INFO:teuthology.orchestra.run.smithi111.stdout:180 2023-03-29T11:01:27.886 INFO:teuthology.orchestra.run.smithi111.stdout:180.out 2023-03-29T11:01:27.886 INFO:teuthology.orchestra.run.smithi111.stdout:181 2023-03-29T11:01:27.887 INFO:teuthology.orchestra.run.smithi111.stdout:181.out 2023-03-29T11:01:27.887 INFO:teuthology.orchestra.run.smithi111.stdout:182 2023-03-29T11:01:27.887 INFO:teuthology.orchestra.run.smithi111.stdout:182.out 2023-03-29T11:01:27.887 INFO:teuthology.orchestra.run.smithi111.stdout:183 2023-03-29T11:01:27.887 INFO:teuthology.orchestra.run.smithi111.stdout:183.out 2023-03-29T11:01:27.887 INFO:teuthology.orchestra.run.smithi111.stdout:184 2023-03-29T11:01:27.887 INFO:teuthology.orchestra.run.smithi111.stdout:184.out 2023-03-29T11:01:27.887 INFO:teuthology.orchestra.run.smithi111.stdout:185 2023-03-29T11:01:27.887 INFO:teuthology.orchestra.run.smithi111.stdout:185.out 2023-03-29T11:01:27.887 INFO:teuthology.orchestra.run.smithi111.stdout:186 2023-03-29T11:01:27.887 INFO:teuthology.orchestra.run.smithi111.stdout:186.out 2023-03-29T11:01:27.887 INFO:teuthology.orchestra.run.smithi111.stdout:187 2023-03-29T11:01:27.887 INFO:teuthology.orchestra.run.smithi111.stdout:187.out 2023-03-29T11:01:27.887 INFO:teuthology.orchestra.run.smithi111.stdout:188 2023-03-29T11:01:27.887 INFO:teuthology.orchestra.run.smithi111.stdout:188.out 2023-03-29T11:01:27.887 INFO:teuthology.orchestra.run.smithi111.stdout:189 2023-03-29T11:01:27.887 INFO:teuthology.orchestra.run.smithi111.stdout:189.out 2023-03-29T11:01:27.887 INFO:teuthology.orchestra.run.smithi111.stdout:190 2023-03-29T11:01:27.888 INFO:teuthology.orchestra.run.smithi111.stdout:190.out 2023-03-29T11:01:27.888 INFO:teuthology.orchestra.run.smithi111.stdout:191 2023-03-29T11:01:27.888 INFO:teuthology.orchestra.run.smithi111.stdout:191.out 2023-03-29T11:01:27.888 INFO:teuthology.orchestra.run.smithi111.stdout:192 2023-03-29T11:01:27.888 INFO:teuthology.orchestra.run.smithi111.stdout:192.out 2023-03-29T11:01:27.888 INFO:teuthology.orchestra.run.smithi111.stdout:193 2023-03-29T11:01:27.888 INFO:teuthology.orchestra.run.smithi111.stdout:193.out 2023-03-29T11:01:27.888 INFO:teuthology.orchestra.run.smithi111.stdout:194 2023-03-29T11:01:27.888 INFO:teuthology.orchestra.run.smithi111.stdout:194.out 2023-03-29T11:01:27.888 INFO:teuthology.orchestra.run.smithi111.stdout:195 2023-03-29T11:01:27.888 INFO:teuthology.orchestra.run.smithi111.stdout:195.out 2023-03-29T11:01:27.888 INFO:teuthology.orchestra.run.smithi111.stdout:196 2023-03-29T11:01:27.888 INFO:teuthology.orchestra.run.smithi111.stdout:196.out 2023-03-29T11:01:27.888 INFO:teuthology.orchestra.run.smithi111.stdout:197 2023-03-29T11:01:27.888 INFO:teuthology.orchestra.run.smithi111.stdout:197.out 2023-03-29T11:01:27.888 INFO:teuthology.orchestra.run.smithi111.stdout:198 2023-03-29T11:01:27.889 INFO:teuthology.orchestra.run.smithi111.stdout:198.out 2023-03-29T11:01:27.889 INFO:teuthology.orchestra.run.smithi111.stdout:199 2023-03-29T11:01:27.889 INFO:teuthology.orchestra.run.smithi111.stdout:199.out 2023-03-29T11:01:27.889 INFO:teuthology.orchestra.run.smithi111.stdout:200 2023-03-29T11:01:27.889 INFO:teuthology.orchestra.run.smithi111.stdout:200.out 2023-03-29T11:01:27.889 INFO:teuthology.orchestra.run.smithi111.stdout:201 2023-03-29T11:01:27.889 INFO:teuthology.orchestra.run.smithi111.stdout:201.out 2023-03-29T11:01:27.889 INFO:teuthology.orchestra.run.smithi111.stdout:202 2023-03-29T11:01:27.889 INFO:teuthology.orchestra.run.smithi111.stdout:202.out 2023-03-29T11:01:27.889 INFO:teuthology.orchestra.run.smithi111.stdout:203 2023-03-29T11:01:27.889 INFO:teuthology.orchestra.run.smithi111.stdout:203.out 2023-03-29T11:01:27.889 INFO:teuthology.orchestra.run.smithi111.stdout:204 2023-03-29T11:01:27.889 INFO:teuthology.orchestra.run.smithi111.stdout:204.out 2023-03-29T11:01:27.889 INFO:teuthology.orchestra.run.smithi111.stdout:205 2023-03-29T11:01:27.889 INFO:teuthology.orchestra.run.smithi111.stdout:205.out 2023-03-29T11:01:27.889 INFO:teuthology.orchestra.run.smithi111.stdout:206 2023-03-29T11:01:27.889 INFO:teuthology.orchestra.run.smithi111.stdout:206.out 2023-03-29T11:01:27.889 INFO:teuthology.orchestra.run.smithi111.stdout:207 2023-03-29T11:01:27.890 INFO:teuthology.orchestra.run.smithi111.stdout:207.out 2023-03-29T11:01:27.890 INFO:teuthology.orchestra.run.smithi111.stdout:208 2023-03-29T11:01:27.890 INFO:teuthology.orchestra.run.smithi111.stdout:208.out 2023-03-29T11:01:27.890 INFO:teuthology.orchestra.run.smithi111.stdout:209 2023-03-29T11:01:27.890 INFO:teuthology.orchestra.run.smithi111.stdout:209.out 2023-03-29T11:01:27.890 INFO:teuthology.orchestra.run.smithi111.stdout:210 2023-03-29T11:01:27.890 INFO:teuthology.orchestra.run.smithi111.stdout:210.out 2023-03-29T11:01:27.890 INFO:teuthology.orchestra.run.smithi111.stdout:211 2023-03-29T11:01:27.890 INFO:teuthology.orchestra.run.smithi111.stdout:211.out 2023-03-29T11:01:27.890 INFO:teuthology.orchestra.run.smithi111.stdout:212 2023-03-29T11:01:27.890 INFO:teuthology.orchestra.run.smithi111.stdout:212.out 2023-03-29T11:01:27.890 INFO:teuthology.orchestra.run.smithi111.stdout:213 2023-03-29T11:01:27.890 INFO:teuthology.orchestra.run.smithi111.stdout:213.out 2023-03-29T11:01:27.890 INFO:teuthology.orchestra.run.smithi111.stdout:214 2023-03-29T11:01:27.890 INFO:teuthology.orchestra.run.smithi111.stdout:214.out 2023-03-29T11:01:27.890 INFO:teuthology.orchestra.run.smithi111.stdout:215 2023-03-29T11:01:27.890 INFO:teuthology.orchestra.run.smithi111.stdout:215.out 2023-03-29T11:01:27.891 INFO:teuthology.orchestra.run.smithi111.stdout:216 2023-03-29T11:01:27.891 INFO:teuthology.orchestra.run.smithi111.stdout:216.out 2023-03-29T11:01:27.891 INFO:teuthology.orchestra.run.smithi111.stdout:217 2023-03-29T11:01:27.891 INFO:teuthology.orchestra.run.smithi111.stdout:217.out 2023-03-29T11:01:27.891 INFO:teuthology.orchestra.run.smithi111.stdout:218 2023-03-29T11:01:27.891 INFO:teuthology.orchestra.run.smithi111.stdout:218.out 2023-03-29T11:01:27.891 INFO:teuthology.orchestra.run.smithi111.stdout:219 2023-03-29T11:01:27.891 INFO:teuthology.orchestra.run.smithi111.stdout:219.out 2023-03-29T11:01:27.891 INFO:teuthology.orchestra.run.smithi111.stdout:220 2023-03-29T11:01:27.891 INFO:teuthology.orchestra.run.smithi111.stdout:220.out 2023-03-29T11:01:27.891 INFO:teuthology.orchestra.run.smithi111.stdout:221 2023-03-29T11:01:27.891 INFO:teuthology.orchestra.run.smithi111.stdout:221.out 2023-03-29T11:01:27.891 INFO:teuthology.orchestra.run.smithi111.stdout:222 2023-03-29T11:01:27.891 INFO:teuthology.orchestra.run.smithi111.stdout:222.out 2023-03-29T11:01:27.891 INFO:teuthology.orchestra.run.smithi111.stdout:223 2023-03-29T11:01:27.891 INFO:teuthology.orchestra.run.smithi111.stdout:223.out 2023-03-29T11:01:27.891 INFO:teuthology.orchestra.run.smithi111.stdout:224 2023-03-29T11:01:27.891 INFO:teuthology.orchestra.run.smithi111.stdout:224.out 2023-03-29T11:01:27.892 INFO:teuthology.orchestra.run.smithi111.stdout:225 2023-03-29T11:01:27.892 INFO:teuthology.orchestra.run.smithi111.stdout:225.out 2023-03-29T11:01:27.892 INFO:teuthology.orchestra.run.smithi111.stdout:226 2023-03-29T11:01:27.892 INFO:teuthology.orchestra.run.smithi111.stdout:226.out 2023-03-29T11:01:27.892 INFO:teuthology.orchestra.run.smithi111.stdout:227 2023-03-29T11:01:27.892 INFO:teuthology.orchestra.run.smithi111.stdout:227.out 2023-03-29T11:01:27.892 INFO:teuthology.orchestra.run.smithi111.stdout:228 2023-03-29T11:01:27.892 INFO:teuthology.orchestra.run.smithi111.stdout:228.out 2023-03-29T11:01:27.892 INFO:teuthology.orchestra.run.smithi111.stdout:229 2023-03-29T11:01:27.892 INFO:teuthology.orchestra.run.smithi111.stdout:229.out 2023-03-29T11:01:27.892 INFO:teuthology.orchestra.run.smithi111.stdout:230 2023-03-29T11:01:27.892 INFO:teuthology.orchestra.run.smithi111.stdout:230.out 2023-03-29T11:01:27.892 INFO:teuthology.orchestra.run.smithi111.stdout:231 2023-03-29T11:01:27.892 INFO:teuthology.orchestra.run.smithi111.stdout:231.out 2023-03-29T11:01:27.892 INFO:teuthology.orchestra.run.smithi111.stdout:232 2023-03-29T11:01:27.892 INFO:teuthology.orchestra.run.smithi111.stdout:232.out 2023-03-29T11:01:27.892 INFO:teuthology.orchestra.run.smithi111.stdout:233 2023-03-29T11:01:27.892 INFO:teuthology.orchestra.run.smithi111.stdout:233.out 2023-03-29T11:01:27.893 INFO:teuthology.orchestra.run.smithi111.stdout:234 2023-03-29T11:01:27.893 INFO:teuthology.orchestra.run.smithi111.stdout:234.out 2023-03-29T11:01:27.893 INFO:teuthology.orchestra.run.smithi111.stdout:235 2023-03-29T11:01:27.893 INFO:teuthology.orchestra.run.smithi111.stdout:235.out 2023-03-29T11:01:27.893 INFO:teuthology.orchestra.run.smithi111.stdout:236 2023-03-29T11:01:27.893 INFO:teuthology.orchestra.run.smithi111.stdout:236.out 2023-03-29T11:01:27.893 INFO:teuthology.orchestra.run.smithi111.stdout:237 2023-03-29T11:01:27.893 INFO:teuthology.orchestra.run.smithi111.stdout:237.out 2023-03-29T11:01:27.893 INFO:teuthology.orchestra.run.smithi111.stdout:238 2023-03-29T11:01:27.893 INFO:teuthology.orchestra.run.smithi111.stdout:238.out 2023-03-29T11:01:27.893 INFO:teuthology.orchestra.run.smithi111.stdout:239 2023-03-29T11:01:27.893 INFO:teuthology.orchestra.run.smithi111.stdout:239.out 2023-03-29T11:01:27.893 INFO:teuthology.orchestra.run.smithi111.stdout:240 2023-03-29T11:01:27.893 INFO:teuthology.orchestra.run.smithi111.stdout:240.out 2023-03-29T11:01:27.893 INFO:teuthology.orchestra.run.smithi111.stdout:241 2023-03-29T11:01:27.893 INFO:teuthology.orchestra.run.smithi111.stdout:241.out 2023-03-29T11:01:27.893 INFO:teuthology.orchestra.run.smithi111.stdout:242 2023-03-29T11:01:27.893 INFO:teuthology.orchestra.run.smithi111.stdout:242.out 2023-03-29T11:01:27.894 INFO:teuthology.orchestra.run.smithi111.stdout:243 2023-03-29T11:01:27.894 INFO:teuthology.orchestra.run.smithi111.stdout:243.out 2023-03-29T11:01:27.894 INFO:teuthology.orchestra.run.smithi111.stdout:244 2023-03-29T11:01:27.894 INFO:teuthology.orchestra.run.smithi111.stdout:244.out 2023-03-29T11:01:27.894 INFO:teuthology.orchestra.run.smithi111.stdout:245 2023-03-29T11:01:27.894 INFO:teuthology.orchestra.run.smithi111.stdout:245.out 2023-03-29T11:01:27.894 INFO:teuthology.orchestra.run.smithi111.stdout:246 2023-03-29T11:01:27.894 INFO:teuthology.orchestra.run.smithi111.stdout:246.out 2023-03-29T11:01:27.894 INFO:teuthology.orchestra.run.smithi111.stdout:247 2023-03-29T11:01:27.894 INFO:teuthology.orchestra.run.smithi111.stdout:247.out 2023-03-29T11:01:27.894 INFO:teuthology.orchestra.run.smithi111.stdout:248 2023-03-29T11:01:27.894 INFO:teuthology.orchestra.run.smithi111.stdout:248.out 2023-03-29T11:01:27.894 INFO:teuthology.orchestra.run.smithi111.stdout:249 2023-03-29T11:01:27.894 INFO:teuthology.orchestra.run.smithi111.stdout:249.out 2023-03-29T11:01:27.894 INFO:teuthology.orchestra.run.smithi111.stdout:250 2023-03-29T11:01:27.894 INFO:teuthology.orchestra.run.smithi111.stdout:250.out 2023-03-29T11:01:27.894 INFO:teuthology.orchestra.run.smithi111.stdout:251 2023-03-29T11:01:27.895 INFO:teuthology.orchestra.run.smithi111.stdout:251.out 2023-03-29T11:01:27.895 INFO:teuthology.orchestra.run.smithi111.stdout:252 2023-03-29T11:01:27.895 INFO:teuthology.orchestra.run.smithi111.stdout:252.out 2023-03-29T11:01:27.895 INFO:teuthology.orchestra.run.smithi111.stdout:253 2023-03-29T11:01:27.895 INFO:teuthology.orchestra.run.smithi111.stdout:253.out 2023-03-29T11:01:27.895 INFO:teuthology.orchestra.run.smithi111.stdout:254 2023-03-29T11:01:27.895 INFO:teuthology.orchestra.run.smithi111.stdout:254.out 2023-03-29T11:01:27.895 INFO:teuthology.orchestra.run.smithi111.stdout:255 2023-03-29T11:01:27.895 INFO:teuthology.orchestra.run.smithi111.stdout:255.out 2023-03-29T11:01:27.895 INFO:teuthology.orchestra.run.smithi111.stdout:256 2023-03-29T11:01:27.895 INFO:teuthology.orchestra.run.smithi111.stdout:256.out 2023-03-29T11:01:27.895 INFO:teuthology.orchestra.run.smithi111.stdout:257 2023-03-29T11:01:27.895 INFO:teuthology.orchestra.run.smithi111.stdout:257.out 2023-03-29T11:01:27.895 INFO:teuthology.orchestra.run.smithi111.stdout:258 2023-03-29T11:01:27.895 INFO:teuthology.orchestra.run.smithi111.stdout:258.out 2023-03-29T11:01:27.895 INFO:teuthology.orchestra.run.smithi111.stdout:259 2023-03-29T11:01:27.895 INFO:teuthology.orchestra.run.smithi111.stdout:259.out 2023-03-29T11:01:27.895 INFO:teuthology.orchestra.run.smithi111.stdout:260 2023-03-29T11:01:27.895 INFO:teuthology.orchestra.run.smithi111.stdout:260.out 2023-03-29T11:01:27.896 INFO:teuthology.orchestra.run.smithi111.stdout:261 2023-03-29T11:01:27.896 INFO:teuthology.orchestra.run.smithi111.stdout:261.out 2023-03-29T11:01:27.896 INFO:teuthology.orchestra.run.smithi111.stdout:262 2023-03-29T11:01:27.896 INFO:teuthology.orchestra.run.smithi111.stdout:262.out 2023-03-29T11:01:27.896 INFO:teuthology.orchestra.run.smithi111.stdout:263 2023-03-29T11:01:27.896 INFO:teuthology.orchestra.run.smithi111.stdout:263.out 2023-03-29T11:01:27.896 INFO:teuthology.orchestra.run.smithi111.stdout:264 2023-03-29T11:01:27.896 INFO:teuthology.orchestra.run.smithi111.stdout:264.out 2023-03-29T11:01:27.896 INFO:teuthology.orchestra.run.smithi111.stdout:265 2023-03-29T11:01:27.896 INFO:teuthology.orchestra.run.smithi111.stdout:265.out 2023-03-29T11:01:27.896 INFO:teuthology.orchestra.run.smithi111.stdout:266 2023-03-29T11:01:27.896 INFO:teuthology.orchestra.run.smithi111.stdout:266.out 2023-03-29T11:01:27.896 INFO:teuthology.orchestra.run.smithi111.stdout:267 2023-03-29T11:01:27.896 INFO:teuthology.orchestra.run.smithi111.stdout:267.out 2023-03-29T11:01:27.896 INFO:teuthology.orchestra.run.smithi111.stdout:268 2023-03-29T11:01:27.896 INFO:teuthology.orchestra.run.smithi111.stdout:268.out 2023-03-29T11:01:27.896 INFO:teuthology.orchestra.run.smithi111.stdout:269 2023-03-29T11:01:27.896 INFO:teuthology.orchestra.run.smithi111.stdout:269.out 2023-03-29T11:01:27.897 INFO:teuthology.orchestra.run.smithi111.stdout:270 2023-03-29T11:01:27.897 INFO:teuthology.orchestra.run.smithi111.stdout:270.out 2023-03-29T11:01:27.897 INFO:teuthology.orchestra.run.smithi111.stdout:271 2023-03-29T11:01:27.897 INFO:teuthology.orchestra.run.smithi111.stdout:271.out 2023-03-29T11:01:27.897 INFO:teuthology.orchestra.run.smithi111.stdout:272 2023-03-29T11:01:27.897 INFO:teuthology.orchestra.run.smithi111.stdout:272.out 2023-03-29T11:01:27.897 INFO:teuthology.orchestra.run.smithi111.stdout:273 2023-03-29T11:01:27.897 INFO:teuthology.orchestra.run.smithi111.stdout:273.out 2023-03-29T11:01:27.897 INFO:teuthology.orchestra.run.smithi111.stdout:274 2023-03-29T11:01:27.897 INFO:teuthology.orchestra.run.smithi111.stdout:274.out 2023-03-29T11:01:27.897 INFO:teuthology.orchestra.run.smithi111.stdout:275 2023-03-29T11:01:27.897 INFO:teuthology.orchestra.run.smithi111.stdout:275.out 2023-03-29T11:01:27.898 INFO:teuthology.orchestra.run.smithi111.stdout:276 2023-03-29T11:01:27.898 INFO:teuthology.orchestra.run.smithi111.stdout:276.out 2023-03-29T11:01:27.898 INFO:teuthology.orchestra.run.smithi111.stdout:277 2023-03-29T11:01:27.898 INFO:teuthology.orchestra.run.smithi111.stdout:277.out 2023-03-29T11:01:27.898 INFO:teuthology.orchestra.run.smithi111.stdout:278 2023-03-29T11:01:27.898 INFO:teuthology.orchestra.run.smithi111.stdout:278.out 2023-03-29T11:01:27.898 INFO:teuthology.orchestra.run.smithi111.stdout:279 2023-03-29T11:01:27.898 INFO:teuthology.orchestra.run.smithi111.stdout:279.out 2023-03-29T11:01:27.898 INFO:teuthology.orchestra.run.smithi111.stdout:280 2023-03-29T11:01:27.898 INFO:teuthology.orchestra.run.smithi111.stdout:280.out 2023-03-29T11:01:27.898 INFO:teuthology.orchestra.run.smithi111.stdout:281 2023-03-29T11:01:27.898 INFO:teuthology.orchestra.run.smithi111.stdout:281.out 2023-03-29T11:01:27.898 INFO:teuthology.orchestra.run.smithi111.stdout:282 2023-03-29T11:01:27.898 INFO:teuthology.orchestra.run.smithi111.stdout:282.out 2023-03-29T11:01:27.898 INFO:teuthology.orchestra.run.smithi111.stdout:283 2023-03-29T11:01:27.898 INFO:teuthology.orchestra.run.smithi111.stdout:283.out 2023-03-29T11:01:27.898 INFO:teuthology.orchestra.run.smithi111.stdout:284 2023-03-29T11:01:27.898 INFO:teuthology.orchestra.run.smithi111.stdout:284.out 2023-03-29T11:01:27.899 INFO:teuthology.orchestra.run.smithi111.stdout:285 2023-03-29T11:01:27.899 INFO:teuthology.orchestra.run.smithi111.stdout:285.out 2023-03-29T11:01:27.899 INFO:teuthology.orchestra.run.smithi111.stdout:286 2023-03-29T11:01:27.899 INFO:teuthology.orchestra.run.smithi111.stdout:286.out 2023-03-29T11:01:27.899 INFO:teuthology.orchestra.run.smithi111.stdout:287 2023-03-29T11:01:27.899 INFO:teuthology.orchestra.run.smithi111.stdout:287.out 2023-03-29T11:01:27.899 INFO:teuthology.orchestra.run.smithi111.stdout:288 2023-03-29T11:01:27.899 INFO:teuthology.orchestra.run.smithi111.stdout:288.out 2023-03-29T11:01:27.899 INFO:teuthology.orchestra.run.smithi111.stdout:289 2023-03-29T11:01:27.899 INFO:teuthology.orchestra.run.smithi111.stdout:289.out 2023-03-29T11:01:27.899 INFO:teuthology.orchestra.run.smithi111.stdout:290 2023-03-29T11:01:27.899 INFO:teuthology.orchestra.run.smithi111.stdout:290.out 2023-03-29T11:01:27.899 INFO:teuthology.orchestra.run.smithi111.stdout:291 2023-03-29T11:01:27.899 INFO:teuthology.orchestra.run.smithi111.stdout:291.out 2023-03-29T11:01:27.899 INFO:teuthology.orchestra.run.smithi111.stdout:292 2023-03-29T11:01:27.899 INFO:teuthology.orchestra.run.smithi111.stdout:292.out 2023-03-29T11:01:27.899 INFO:teuthology.orchestra.run.smithi111.stdout:293 2023-03-29T11:01:27.899 INFO:teuthology.orchestra.run.smithi111.stdout:293.out 2023-03-29T11:01:27.900 INFO:teuthology.orchestra.run.smithi111.stdout:294 2023-03-29T11:01:27.900 INFO:teuthology.orchestra.run.smithi111.stdout:294.out 2023-03-29T11:01:27.900 INFO:teuthology.orchestra.run.smithi111.stdout:295 2023-03-29T11:01:27.900 INFO:teuthology.orchestra.run.smithi111.stdout:295.out 2023-03-29T11:01:27.900 INFO:teuthology.orchestra.run.smithi111.stdout:296 2023-03-29T11:01:27.900 INFO:teuthology.orchestra.run.smithi111.stdout:296.out 2023-03-29T11:01:27.900 INFO:teuthology.orchestra.run.smithi111.stdout:297 2023-03-29T11:01:27.900 INFO:teuthology.orchestra.run.smithi111.stdout:297.out 2023-03-29T11:01:27.900 INFO:teuthology.orchestra.run.smithi111.stdout:298 2023-03-29T11:01:27.900 INFO:teuthology.orchestra.run.smithi111.stdout:298.out 2023-03-29T11:01:27.900 INFO:teuthology.orchestra.run.smithi111.stdout:299 2023-03-29T11:01:27.900 INFO:teuthology.orchestra.run.smithi111.stdout:299.out 2023-03-29T11:01:27.900 INFO:teuthology.orchestra.run.smithi111.stdout:300 2023-03-29T11:01:27.900 INFO:teuthology.orchestra.run.smithi111.stdout:300.out 2023-03-29T11:01:27.900 INFO:teuthology.orchestra.run.smithi111.stdout:301 2023-03-29T11:01:27.900 INFO:teuthology.orchestra.run.smithi111.stdout:301.out 2023-03-29T11:01:27.900 INFO:teuthology.orchestra.run.smithi111.stdout:302 2023-03-29T11:01:27.900 INFO:teuthology.orchestra.run.smithi111.stdout:302.out 2023-03-29T11:01:27.900 INFO:teuthology.orchestra.run.smithi111.stdout:303 2023-03-29T11:01:27.901 INFO:teuthology.orchestra.run.smithi111.stdout:303.out 2023-03-29T11:01:27.901 INFO:teuthology.orchestra.run.smithi111.stdout:304 2023-03-29T11:01:27.901 INFO:teuthology.orchestra.run.smithi111.stdout:304.out 2023-03-29T11:01:27.901 INFO:teuthology.orchestra.run.smithi111.stdout:305 2023-03-29T11:01:27.901 INFO:teuthology.orchestra.run.smithi111.stdout:305.out 2023-03-29T11:01:27.901 INFO:teuthology.orchestra.run.smithi111.stdout:306 2023-03-29T11:01:27.901 INFO:teuthology.orchestra.run.smithi111.stdout:306.out 2023-03-29T11:01:27.901 INFO:teuthology.orchestra.run.smithi111.stdout:307 2023-03-29T11:01:27.901 INFO:teuthology.orchestra.run.smithi111.stdout:307.out 2023-03-29T11:01:27.901 INFO:teuthology.orchestra.run.smithi111.stdout:308 2023-03-29T11:01:27.901 INFO:teuthology.orchestra.run.smithi111.stdout:308.out 2023-03-29T11:01:27.901 INFO:teuthology.orchestra.run.smithi111.stdout:309 2023-03-29T11:01:27.901 INFO:teuthology.orchestra.run.smithi111.stdout:309.out 2023-03-29T11:01:27.901 INFO:teuthology.orchestra.run.smithi111.stdout:310 2023-03-29T11:01:27.901 INFO:teuthology.orchestra.run.smithi111.stdout:310.out 2023-03-29T11:01:27.901 INFO:teuthology.orchestra.run.smithi111.stdout:311 2023-03-29T11:01:27.901 INFO:teuthology.orchestra.run.smithi111.stdout:311.out 2023-03-29T11:01:27.902 INFO:teuthology.orchestra.run.smithi111.stdout:312 2023-03-29T11:01:27.902 INFO:teuthology.orchestra.run.smithi111.stdout:312.out 2023-03-29T11:01:27.902 INFO:teuthology.orchestra.run.smithi111.stdout:313 2023-03-29T11:01:27.902 INFO:teuthology.orchestra.run.smithi111.stdout:313.out 2023-03-29T11:01:27.902 INFO:teuthology.orchestra.run.smithi111.stdout:314 2023-03-29T11:01:27.902 INFO:teuthology.orchestra.run.smithi111.stdout:314.out 2023-03-29T11:01:27.902 INFO:teuthology.orchestra.run.smithi111.stdout:315 2023-03-29T11:01:27.902 INFO:teuthology.orchestra.run.smithi111.stdout:315.out 2023-03-29T11:01:27.902 INFO:teuthology.orchestra.run.smithi111.stdout:316 2023-03-29T11:01:27.902 INFO:teuthology.orchestra.run.smithi111.stdout:316.out 2023-03-29T11:01:27.902 INFO:teuthology.orchestra.run.smithi111.stdout:317 2023-03-29T11:01:27.902 INFO:teuthology.orchestra.run.smithi111.stdout:317.out 2023-03-29T11:01:27.902 INFO:teuthology.orchestra.run.smithi111.stdout:318 2023-03-29T11:01:27.902 INFO:teuthology.orchestra.run.smithi111.stdout:318.out 2023-03-29T11:01:27.902 INFO:teuthology.orchestra.run.smithi111.stdout:319 2023-03-29T11:01:27.902 INFO:teuthology.orchestra.run.smithi111.stdout:319.out 2023-03-29T11:01:27.902 INFO:teuthology.orchestra.run.smithi111.stdout:320 2023-03-29T11:01:27.903 INFO:teuthology.orchestra.run.smithi111.stdout:320.out 2023-03-29T11:01:27.903 INFO:teuthology.orchestra.run.smithi111.stdout:321 2023-03-29T11:01:27.903 INFO:teuthology.orchestra.run.smithi111.stdout:321.out 2023-03-29T11:01:27.903 INFO:teuthology.orchestra.run.smithi111.stdout:322 2023-03-29T11:01:27.903 INFO:teuthology.orchestra.run.smithi111.stdout:322.out 2023-03-29T11:01:27.903 INFO:teuthology.orchestra.run.smithi111.stdout:323 2023-03-29T11:01:27.903 INFO:teuthology.orchestra.run.smithi111.stdout:323.out 2023-03-29T11:01:27.903 INFO:teuthology.orchestra.run.smithi111.stdout:324 2023-03-29T11:01:27.903 INFO:teuthology.orchestra.run.smithi111.stdout:324.out 2023-03-29T11:01:27.903 INFO:teuthology.orchestra.run.smithi111.stdout:325 2023-03-29T11:01:27.903 INFO:teuthology.orchestra.run.smithi111.stdout:325.out 2023-03-29T11:01:27.903 INFO:teuthology.orchestra.run.smithi111.stdout:326 2023-03-29T11:01:27.903 INFO:teuthology.orchestra.run.smithi111.stdout:326.out 2023-03-29T11:01:27.903 INFO:teuthology.orchestra.run.smithi111.stdout:327 2023-03-29T11:01:27.903 INFO:teuthology.orchestra.run.smithi111.stdout:327.out 2023-03-29T11:01:27.903 INFO:teuthology.orchestra.run.smithi111.stdout:328 2023-03-29T11:01:27.903 INFO:teuthology.orchestra.run.smithi111.stdout:328.out 2023-03-29T11:01:27.903 INFO:teuthology.orchestra.run.smithi111.stdout:329 2023-03-29T11:01:27.904 INFO:teuthology.orchestra.run.smithi111.stdout:329.out 2023-03-29T11:01:27.904 INFO:teuthology.orchestra.run.smithi111.stdout:330 2023-03-29T11:01:27.904 INFO:teuthology.orchestra.run.smithi111.stdout:330.out 2023-03-29T11:01:27.904 INFO:teuthology.orchestra.run.smithi111.stdout:331 2023-03-29T11:01:27.904 INFO:teuthology.orchestra.run.smithi111.stdout:331.out 2023-03-29T11:01:27.904 INFO:teuthology.orchestra.run.smithi111.stdout:332 2023-03-29T11:01:27.904 INFO:teuthology.orchestra.run.smithi111.stdout:332.out 2023-03-29T11:01:27.904 INFO:teuthology.orchestra.run.smithi111.stdout:333 2023-03-29T11:01:27.904 INFO:teuthology.orchestra.run.smithi111.stdout:333.out 2023-03-29T11:01:27.904 INFO:teuthology.orchestra.run.smithi111.stdout:334 2023-03-29T11:01:27.904 INFO:teuthology.orchestra.run.smithi111.stdout:334.out 2023-03-29T11:01:27.904 INFO:teuthology.orchestra.run.smithi111.stdout:335 2023-03-29T11:01:27.904 INFO:teuthology.orchestra.run.smithi111.stdout:335.out 2023-03-29T11:01:27.904 INFO:teuthology.orchestra.run.smithi111.stdout:336 2023-03-29T11:01:27.904 INFO:teuthology.orchestra.run.smithi111.stdout:336.out 2023-03-29T11:01:27.904 INFO:teuthology.orchestra.run.smithi111.stdout:337 2023-03-29T11:01:27.904 INFO:teuthology.orchestra.run.smithi111.stdout:337.out 2023-03-29T11:01:27.904 INFO:teuthology.orchestra.run.smithi111.stdout:338 2023-03-29T11:01:27.905 INFO:teuthology.orchestra.run.smithi111.stdout:338.out 2023-03-29T11:01:27.905 INFO:teuthology.orchestra.run.smithi111.stdout:339 2023-03-29T11:01:27.905 INFO:teuthology.orchestra.run.smithi111.stdout:339.out 2023-03-29T11:01:27.905 INFO:teuthology.orchestra.run.smithi111.stdout:340 2023-03-29T11:01:27.905 INFO:teuthology.orchestra.run.smithi111.stdout:340.out 2023-03-29T11:01:27.905 INFO:teuthology.orchestra.run.smithi111.stdout:341 2023-03-29T11:01:27.905 INFO:teuthology.orchestra.run.smithi111.stdout:341.out 2023-03-29T11:01:27.905 INFO:teuthology.orchestra.run.smithi111.stdout:342 2023-03-29T11:01:27.905 INFO:teuthology.orchestra.run.smithi111.stdout:342.out 2023-03-29T11:01:27.905 INFO:teuthology.orchestra.run.smithi111.stdout:343 2023-03-29T11:01:27.905 INFO:teuthology.orchestra.run.smithi111.stdout:343.out 2023-03-29T11:01:27.905 INFO:teuthology.orchestra.run.smithi111.stdout:344 2023-03-29T11:01:27.905 INFO:teuthology.orchestra.run.smithi111.stdout:344.out 2023-03-29T11:01:27.905 INFO:teuthology.orchestra.run.smithi111.stdout:345 2023-03-29T11:01:27.905 INFO:teuthology.orchestra.run.smithi111.stdout:345.out 2023-03-29T11:01:27.905 INFO:teuthology.orchestra.run.smithi111.stdout:346 2023-03-29T11:01:27.905 INFO:teuthology.orchestra.run.smithi111.stdout:346.out 2023-03-29T11:01:27.905 INFO:teuthology.orchestra.run.smithi111.stdout:347 2023-03-29T11:01:27.906 INFO:teuthology.orchestra.run.smithi111.stdout:347.out 2023-03-29T11:01:27.906 INFO:teuthology.orchestra.run.smithi111.stdout:348 2023-03-29T11:01:27.906 INFO:teuthology.orchestra.run.smithi111.stdout:348.out 2023-03-29T11:01:27.906 INFO:teuthology.orchestra.run.smithi111.stdout:349 2023-03-29T11:01:27.906 INFO:teuthology.orchestra.run.smithi111.stdout:349.out 2023-03-29T11:01:27.906 INFO:teuthology.orchestra.run.smithi111.stdout:350 2023-03-29T11:01:27.906 INFO:teuthology.orchestra.run.smithi111.stdout:350.out 2023-03-29T11:01:27.906 INFO:teuthology.orchestra.run.smithi111.stdout:351 2023-03-29T11:01:27.906 INFO:teuthology.orchestra.run.smithi111.stdout:351.out 2023-03-29T11:01:27.906 INFO:teuthology.orchestra.run.smithi111.stdout:352 2023-03-29T11:01:27.906 INFO:teuthology.orchestra.run.smithi111.stdout:352.out 2023-03-29T11:01:27.906 INFO:teuthology.orchestra.run.smithi111.stdout:353 2023-03-29T11:01:27.906 INFO:teuthology.orchestra.run.smithi111.stdout:353.out 2023-03-29T11:01:27.906 INFO:teuthology.orchestra.run.smithi111.stdout:354 2023-03-29T11:01:27.906 INFO:teuthology.orchestra.run.smithi111.stdout:354.out 2023-03-29T11:01:27.906 INFO:teuthology.orchestra.run.smithi111.stdout:355 2023-03-29T11:01:27.906 INFO:teuthology.orchestra.run.smithi111.stdout:355.out 2023-03-29T11:01:27.907 INFO:teuthology.orchestra.run.smithi111.stdout:356 2023-03-29T11:01:27.907 INFO:teuthology.orchestra.run.smithi111.stdout:356.out 2023-03-29T11:01:27.907 INFO:teuthology.orchestra.run.smithi111.stdout:357 2023-03-29T11:01:27.907 INFO:teuthology.orchestra.run.smithi111.stdout:357.out 2023-03-29T11:01:27.907 INFO:teuthology.orchestra.run.smithi111.stdout:358 2023-03-29T11:01:27.907 INFO:teuthology.orchestra.run.smithi111.stdout:358.out 2023-03-29T11:01:27.907 INFO:teuthology.orchestra.run.smithi111.stdout:359 2023-03-29T11:01:27.907 INFO:teuthology.orchestra.run.smithi111.stdout:359.out 2023-03-29T11:01:27.907 INFO:teuthology.orchestra.run.smithi111.stdout:360 2023-03-29T11:01:27.907 INFO:teuthology.orchestra.run.smithi111.stdout:360.out 2023-03-29T11:01:27.907 INFO:teuthology.orchestra.run.smithi111.stdout:361 2023-03-29T11:01:27.907 INFO:teuthology.orchestra.run.smithi111.stdout:361.out 2023-03-29T11:01:27.907 INFO:teuthology.orchestra.run.smithi111.stdout:362 2023-03-29T11:01:27.907 INFO:teuthology.orchestra.run.smithi111.stdout:362.out 2023-03-29T11:01:27.907 INFO:teuthology.orchestra.run.smithi111.stdout:363 2023-03-29T11:01:27.907 INFO:teuthology.orchestra.run.smithi111.stdout:363.out 2023-03-29T11:01:27.907 INFO:teuthology.orchestra.run.smithi111.stdout:364 2023-03-29T11:01:27.907 INFO:teuthology.orchestra.run.smithi111.stdout:364.out 2023-03-29T11:01:27.907 INFO:teuthology.orchestra.run.smithi111.stdout:365 2023-03-29T11:01:27.908 INFO:teuthology.orchestra.run.smithi111.stdout:365.out 2023-03-29T11:01:27.908 INFO:teuthology.orchestra.run.smithi111.stdout:366 2023-03-29T11:01:27.908 INFO:teuthology.orchestra.run.smithi111.stdout:366.out 2023-03-29T11:01:27.908 INFO:teuthology.orchestra.run.smithi111.stdout:367 2023-03-29T11:01:27.908 INFO:teuthology.orchestra.run.smithi111.stdout:367.out 2023-03-29T11:01:27.908 INFO:teuthology.orchestra.run.smithi111.stdout:368 2023-03-29T11:01:27.908 INFO:teuthology.orchestra.run.smithi111.stdout:368.out 2023-03-29T11:01:27.908 INFO:teuthology.orchestra.run.smithi111.stdout:369 2023-03-29T11:01:27.908 INFO:teuthology.orchestra.run.smithi111.stdout:369.out 2023-03-29T11:01:27.908 INFO:teuthology.orchestra.run.smithi111.stdout:370 2023-03-29T11:01:27.908 INFO:teuthology.orchestra.run.smithi111.stdout:370.out 2023-03-29T11:01:27.908 INFO:teuthology.orchestra.run.smithi111.stdout:371 2023-03-29T11:01:27.908 INFO:teuthology.orchestra.run.smithi111.stdout:371.out 2023-03-29T11:01:27.908 INFO:teuthology.orchestra.run.smithi111.stdout:372 2023-03-29T11:01:27.908 INFO:teuthology.orchestra.run.smithi111.stdout:372.out 2023-03-29T11:01:27.908 INFO:teuthology.orchestra.run.smithi111.stdout:373 2023-03-29T11:01:27.908 INFO:teuthology.orchestra.run.smithi111.stdout:373.out 2023-03-29T11:01:27.908 INFO:teuthology.orchestra.run.smithi111.stdout:374 2023-03-29T11:01:27.909 INFO:teuthology.orchestra.run.smithi111.stdout:374.out 2023-03-29T11:01:27.909 INFO:teuthology.orchestra.run.smithi111.stdout:375 2023-03-29T11:01:27.909 INFO:teuthology.orchestra.run.smithi111.stdout:375.out 2023-03-29T11:01:27.909 INFO:teuthology.orchestra.run.smithi111.stdout:376 2023-03-29T11:01:27.909 INFO:teuthology.orchestra.run.smithi111.stdout:376.out 2023-03-29T11:01:27.909 INFO:teuthology.orchestra.run.smithi111.stdout:377 2023-03-29T11:01:27.909 INFO:teuthology.orchestra.run.smithi111.stdout:377.out 2023-03-29T11:01:27.909 INFO:teuthology.orchestra.run.smithi111.stdout:378 2023-03-29T11:01:27.909 INFO:teuthology.orchestra.run.smithi111.stdout:378.out 2023-03-29T11:01:27.909 INFO:teuthology.orchestra.run.smithi111.stdout:379 2023-03-29T11:01:27.909 INFO:teuthology.orchestra.run.smithi111.stdout:379.out 2023-03-29T11:01:27.909 INFO:teuthology.orchestra.run.smithi111.stdout:380 2023-03-29T11:01:27.909 INFO:teuthology.orchestra.run.smithi111.stdout:380.out 2023-03-29T11:01:27.909 INFO:teuthology.orchestra.run.smithi111.stdout:381 2023-03-29T11:01:27.909 INFO:teuthology.orchestra.run.smithi111.stdout:381.out 2023-03-29T11:01:27.909 INFO:teuthology.orchestra.run.smithi111.stdout:382 2023-03-29T11:01:27.909 INFO:teuthology.orchestra.run.smithi111.stdout:382.out 2023-03-29T11:01:27.910 INFO:teuthology.orchestra.run.smithi111.stdout:383 2023-03-29T11:01:27.910 INFO:teuthology.orchestra.run.smithi111.stdout:383.out 2023-03-29T11:01:27.910 INFO:teuthology.orchestra.run.smithi111.stdout:384 2023-03-29T11:01:27.910 INFO:teuthology.orchestra.run.smithi111.stdout:384.out 2023-03-29T11:01:27.910 INFO:teuthology.orchestra.run.smithi111.stdout:385 2023-03-29T11:01:27.910 INFO:teuthology.orchestra.run.smithi111.stdout:385.out 2023-03-29T11:01:27.910 INFO:teuthology.orchestra.run.smithi111.stdout:386 2023-03-29T11:01:27.910 INFO:teuthology.orchestra.run.smithi111.stdout:386.out 2023-03-29T11:01:27.910 INFO:teuthology.orchestra.run.smithi111.stdout:387 2023-03-29T11:01:27.910 INFO:teuthology.orchestra.run.smithi111.stdout:387.out 2023-03-29T11:01:27.910 INFO:teuthology.orchestra.run.smithi111.stdout:388 2023-03-29T11:01:27.910 INFO:teuthology.orchestra.run.smithi111.stdout:388.out 2023-03-29T11:01:27.910 INFO:teuthology.orchestra.run.smithi111.stdout:389 2023-03-29T11:01:27.910 INFO:teuthology.orchestra.run.smithi111.stdout:389.out 2023-03-29T11:01:27.910 INFO:teuthology.orchestra.run.smithi111.stdout:390 2023-03-29T11:01:27.910 INFO:teuthology.orchestra.run.smithi111.stdout:390.out 2023-03-29T11:01:27.910 INFO:teuthology.orchestra.run.smithi111.stdout:391 2023-03-29T11:01:27.910 INFO:teuthology.orchestra.run.smithi111.stdout:391.out 2023-03-29T11:01:27.911 INFO:teuthology.orchestra.run.smithi111.stdout:392 2023-03-29T11:01:27.911 INFO:teuthology.orchestra.run.smithi111.stdout:392.out 2023-03-29T11:01:27.911 INFO:teuthology.orchestra.run.smithi111.stdout:393 2023-03-29T11:01:27.911 INFO:teuthology.orchestra.run.smithi111.stdout:393.out 2023-03-29T11:01:27.911 INFO:teuthology.orchestra.run.smithi111.stdout:394 2023-03-29T11:01:27.911 INFO:teuthology.orchestra.run.smithi111.stdout:394.out 2023-03-29T11:01:27.911 INFO:teuthology.orchestra.run.smithi111.stdout:395 2023-03-29T11:01:27.911 INFO:teuthology.orchestra.run.smithi111.stdout:395.out 2023-03-29T11:01:27.911 INFO:teuthology.orchestra.run.smithi111.stdout:396 2023-03-29T11:01:27.911 INFO:teuthology.orchestra.run.smithi111.stdout:396.out 2023-03-29T11:01:27.911 INFO:teuthology.orchestra.run.smithi111.stdout:397 2023-03-29T11:01:27.911 INFO:teuthology.orchestra.run.smithi111.stdout:397.out 2023-03-29T11:01:27.911 INFO:teuthology.orchestra.run.smithi111.stdout:398 2023-03-29T11:01:27.911 INFO:teuthology.orchestra.run.smithi111.stdout:398.out 2023-03-29T11:01:27.911 INFO:teuthology.orchestra.run.smithi111.stdout:399 2023-03-29T11:01:27.911 INFO:teuthology.orchestra.run.smithi111.stdout:399.out 2023-03-29T11:01:27.911 INFO:teuthology.orchestra.run.smithi111.stdout:400 2023-03-29T11:01:27.911 INFO:teuthology.orchestra.run.smithi111.stdout:400.out 2023-03-29T11:01:27.911 INFO:teuthology.orchestra.run.smithi111.stdout:401 2023-03-29T11:01:27.912 INFO:teuthology.orchestra.run.smithi111.stdout:401.out 2023-03-29T11:01:27.912 INFO:teuthology.orchestra.run.smithi111.stdout:402 2023-03-29T11:01:27.912 INFO:teuthology.orchestra.run.smithi111.stdout:402.out 2023-03-29T11:01:27.912 INFO:teuthology.orchestra.run.smithi111.stdout:403 2023-03-29T11:01:27.912 INFO:teuthology.orchestra.run.smithi111.stdout:403.out 2023-03-29T11:01:27.912 INFO:teuthology.orchestra.run.smithi111.stdout:404 2023-03-29T11:01:27.912 INFO:teuthology.orchestra.run.smithi111.stdout:404.out 2023-03-29T11:01:27.912 INFO:teuthology.orchestra.run.smithi111.stdout:405 2023-03-29T11:01:27.912 INFO:teuthology.orchestra.run.smithi111.stdout:405.out 2023-03-29T11:01:27.912 INFO:teuthology.orchestra.run.smithi111.stdout:406 2023-03-29T11:01:27.912 INFO:teuthology.orchestra.run.smithi111.stdout:406.out 2023-03-29T11:01:27.912 INFO:teuthology.orchestra.run.smithi111.stdout:407 2023-03-29T11:01:27.912 INFO:teuthology.orchestra.run.smithi111.stdout:407.out 2023-03-29T11:01:27.912 INFO:teuthology.orchestra.run.smithi111.stdout:408 2023-03-29T11:01:27.912 INFO:teuthology.orchestra.run.smithi111.stdout:408.out 2023-03-29T11:01:27.912 INFO:teuthology.orchestra.run.smithi111.stdout:409 2023-03-29T11:01:27.912 INFO:teuthology.orchestra.run.smithi111.stdout:409.out 2023-03-29T11:01:27.912 INFO:teuthology.orchestra.run.smithi111.stdout:410 2023-03-29T11:01:27.913 INFO:teuthology.orchestra.run.smithi111.stdout:410.out 2023-03-29T11:01:27.913 INFO:teuthology.orchestra.run.smithi111.stdout:411 2023-03-29T11:01:27.913 INFO:teuthology.orchestra.run.smithi111.stdout:411.out 2023-03-29T11:01:27.913 INFO:teuthology.orchestra.run.smithi111.stdout:412 2023-03-29T11:01:27.913 INFO:teuthology.orchestra.run.smithi111.stdout:412.out 2023-03-29T11:01:27.913 INFO:teuthology.orchestra.run.smithi111.stdout:413 2023-03-29T11:01:27.913 INFO:teuthology.orchestra.run.smithi111.stdout:413.out 2023-03-29T11:01:27.913 INFO:teuthology.orchestra.run.smithi111.stdout:414 2023-03-29T11:01:27.913 INFO:teuthology.orchestra.run.smithi111.stdout:414.out 2023-03-29T11:01:27.913 INFO:teuthology.orchestra.run.smithi111.stdout:415 2023-03-29T11:01:27.913 INFO:teuthology.orchestra.run.smithi111.stdout:415.out 2023-03-29T11:01:27.913 INFO:teuthology.orchestra.run.smithi111.stdout:416 2023-03-29T11:01:27.913 INFO:teuthology.orchestra.run.smithi111.stdout:416.out 2023-03-29T11:01:27.913 INFO:teuthology.orchestra.run.smithi111.stdout:417 2023-03-29T11:01:27.913 INFO:teuthology.orchestra.run.smithi111.stdout:417.out 2023-03-29T11:01:27.913 INFO:teuthology.orchestra.run.smithi111.stdout:418 2023-03-29T11:01:27.913 INFO:teuthology.orchestra.run.smithi111.stdout:418.out 2023-03-29T11:01:27.913 INFO:teuthology.orchestra.run.smithi111.stdout:419 2023-03-29T11:01:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:419.out 2023-03-29T11:01:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:420 2023-03-29T11:01:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:420.out 2023-03-29T11:01:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:421 2023-03-29T11:01:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:421.out 2023-03-29T11:01:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:422 2023-03-29T11:01:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:422.out 2023-03-29T11:01:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:423 2023-03-29T11:01:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:423.out 2023-03-29T11:01:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:424 2023-03-29T11:01:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:424.out 2023-03-29T11:01:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:425 2023-03-29T11:01:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:425.out 2023-03-29T11:01:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:426 2023-03-29T11:01:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:426.out 2023-03-29T11:01:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:427 2023-03-29T11:01:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:427.out 2023-03-29T11:01:27.914 INFO:teuthology.orchestra.run.smithi111.stdout:428 2023-03-29T11:01:27.915 INFO:teuthology.orchestra.run.smithi111.stdout:428.out 2023-03-29T11:01:27.915 INFO:teuthology.orchestra.run.smithi111.stdout:429 2023-03-29T11:01:27.915 INFO:teuthology.orchestra.run.smithi111.stdout:429.out 2023-03-29T11:01:27.915 INFO:teuthology.orchestra.run.smithi111.stdout:430 2023-03-29T11:01:27.915 INFO:teuthology.orchestra.run.smithi111.stdout:430.out 2023-03-29T11:01:27.915 INFO:teuthology.orchestra.run.smithi111.stdout:431 2023-03-29T11:01:27.915 INFO:teuthology.orchestra.run.smithi111.stdout:431.out 2023-03-29T11:01:27.915 INFO:teuthology.orchestra.run.smithi111.stdout:432 2023-03-29T11:01:27.915 INFO:teuthology.orchestra.run.smithi111.stdout:432.out 2023-03-29T11:01:27.915 INFO:teuthology.orchestra.run.smithi111.stdout:433 2023-03-29T11:01:27.915 INFO:teuthology.orchestra.run.smithi111.stdout:433.out 2023-03-29T11:01:27.915 INFO:teuthology.orchestra.run.smithi111.stdout:434 2023-03-29T11:01:27.915 INFO:teuthology.orchestra.run.smithi111.stdout:434.out 2023-03-29T11:01:27.915 INFO:teuthology.orchestra.run.smithi111.stdout:435 2023-03-29T11:01:27.915 INFO:teuthology.orchestra.run.smithi111.stdout:435.out 2023-03-29T11:01:27.915 INFO:teuthology.orchestra.run.smithi111.stdout:436 2023-03-29T11:01:27.915 INFO:teuthology.orchestra.run.smithi111.stdout:436.out 2023-03-29T11:01:27.915 INFO:teuthology.orchestra.run.smithi111.stdout:437 2023-03-29T11:01:27.916 INFO:teuthology.orchestra.run.smithi111.stdout:437.out 2023-03-29T11:01:27.916 INFO:teuthology.orchestra.run.smithi111.stdout:438 2023-03-29T11:01:27.916 INFO:teuthology.orchestra.run.smithi111.stdout:438.out 2023-03-29T11:01:27.916 INFO:teuthology.orchestra.run.smithi111.stdout:439 2023-03-29T11:01:27.916 INFO:teuthology.orchestra.run.smithi111.stdout:439.out 2023-03-29T11:01:27.916 INFO:teuthology.orchestra.run.smithi111.stdout:440 2023-03-29T11:01:27.916 INFO:teuthology.orchestra.run.smithi111.stdout:440.out 2023-03-29T11:01:27.916 INFO:teuthology.orchestra.run.smithi111.stdout:441 2023-03-29T11:01:27.916 INFO:teuthology.orchestra.run.smithi111.stdout:441.out 2023-03-29T11:01:27.916 INFO:teuthology.orchestra.run.smithi111.stdout:442 2023-03-29T11:01:27.916 INFO:teuthology.orchestra.run.smithi111.stdout:442.out 2023-03-29T11:01:27.916 INFO:teuthology.orchestra.run.smithi111.stdout:443 2023-03-29T11:01:27.916 INFO:teuthology.orchestra.run.smithi111.stdout:443.out 2023-03-29T11:01:27.916 INFO:teuthology.orchestra.run.smithi111.stdout:444 2023-03-29T11:01:27.916 INFO:teuthology.orchestra.run.smithi111.stdout:444.out 2023-03-29T11:01:27.916 INFO:teuthology.orchestra.run.smithi111.stdout:445 2023-03-29T11:01:27.916 INFO:teuthology.orchestra.run.smithi111.stdout:445.out 2023-03-29T11:01:27.916 INFO:teuthology.orchestra.run.smithi111.stdout:446 2023-03-29T11:01:27.917 INFO:teuthology.orchestra.run.smithi111.stdout:446.out 2023-03-29T11:01:27.917 INFO:teuthology.orchestra.run.smithi111.stdout:447 2023-03-29T11:01:27.917 INFO:teuthology.orchestra.run.smithi111.stdout:447.out 2023-03-29T11:01:27.917 INFO:teuthology.orchestra.run.smithi111.stdout:448 2023-03-29T11:01:27.917 INFO:teuthology.orchestra.run.smithi111.stdout:448.out 2023-03-29T11:01:27.917 INFO:teuthology.orchestra.run.smithi111.stdout:449 2023-03-29T11:01:27.917 INFO:teuthology.orchestra.run.smithi111.stdout:449.out 2023-03-29T11:01:27.917 INFO:teuthology.orchestra.run.smithi111.stdout:450 2023-03-29T11:01:27.917 INFO:teuthology.orchestra.run.smithi111.stdout:450.out 2023-03-29T11:01:27.917 INFO:teuthology.orchestra.run.smithi111.stdout:451 2023-03-29T11:01:27.917 INFO:teuthology.orchestra.run.smithi111.stdout:451.out 2023-03-29T11:01:27.917 INFO:teuthology.orchestra.run.smithi111.stdout:452 2023-03-29T11:01:27.917 INFO:teuthology.orchestra.run.smithi111.stdout:452.out 2023-03-29T11:01:27.917 INFO:teuthology.orchestra.run.smithi111.stdout:453 2023-03-29T11:01:27.917 INFO:teuthology.orchestra.run.smithi111.stdout:453.out 2023-03-29T11:01:27.917 INFO:teuthology.orchestra.run.smithi111.stdout:454 2023-03-29T11:01:27.917 INFO:teuthology.orchestra.run.smithi111.stdout:454.out 2023-03-29T11:01:27.917 INFO:teuthology.orchestra.run.smithi111.stdout:455 2023-03-29T11:01:27.917 INFO:teuthology.orchestra.run.smithi111.stdout:455.out 2023-03-29T11:01:27.918 INFO:teuthology.orchestra.run.smithi111.stdout:456 2023-03-29T11:01:27.918 INFO:teuthology.orchestra.run.smithi111.stdout:456.out 2023-03-29T11:01:27.918 INFO:teuthology.orchestra.run.smithi111.stdout:457 2023-03-29T11:01:27.918 INFO:teuthology.orchestra.run.smithi111.stdout:457.out 2023-03-29T11:01:27.918 INFO:teuthology.orchestra.run.smithi111.stdout:458 2023-03-29T11:01:27.918 INFO:teuthology.orchestra.run.smithi111.stdout:458.out 2023-03-29T11:01:27.918 INFO:teuthology.orchestra.run.smithi111.stdout:459 2023-03-29T11:01:27.918 INFO:teuthology.orchestra.run.smithi111.stdout:459.out 2023-03-29T11:01:27.918 INFO:teuthology.orchestra.run.smithi111.stdout:460 2023-03-29T11:01:27.918 INFO:teuthology.orchestra.run.smithi111.stdout:460.out 2023-03-29T11:01:27.918 INFO:teuthology.orchestra.run.smithi111.stdout:461 2023-03-29T11:01:27.918 INFO:teuthology.orchestra.run.smithi111.stdout:461.out 2023-03-29T11:01:27.918 INFO:teuthology.orchestra.run.smithi111.stdout:462 2023-03-29T11:01:27.918 INFO:teuthology.orchestra.run.smithi111.stdout:462.out 2023-03-29T11:01:27.918 INFO:teuthology.orchestra.run.smithi111.stdout:463 2023-03-29T11:01:27.918 INFO:teuthology.orchestra.run.smithi111.stdout:463.out 2023-03-29T11:01:27.918 INFO:teuthology.orchestra.run.smithi111.stdout:464 2023-03-29T11:01:27.918 INFO:teuthology.orchestra.run.smithi111.stdout:464.out 2023-03-29T11:01:27.919 INFO:teuthology.orchestra.run.smithi111.stdout:465 2023-03-29T11:01:27.919 INFO:teuthology.orchestra.run.smithi111.stdout:465.out 2023-03-29T11:01:27.919 INFO:teuthology.orchestra.run.smithi111.stdout:466 2023-03-29T11:01:27.919 INFO:teuthology.orchestra.run.smithi111.stdout:466.out 2023-03-29T11:01:27.919 INFO:teuthology.orchestra.run.smithi111.stdout:467 2023-03-29T11:01:27.919 INFO:teuthology.orchestra.run.smithi111.stdout:467.out 2023-03-29T11:01:27.919 INFO:teuthology.orchestra.run.smithi111.stdout:468 2023-03-29T11:01:27.919 INFO:teuthology.orchestra.run.smithi111.stdout:468.out 2023-03-29T11:01:27.919 INFO:teuthology.orchestra.run.smithi111.stdout:469 2023-03-29T11:01:27.919 INFO:teuthology.orchestra.run.smithi111.stdout:469.out 2023-03-29T11:01:27.919 INFO:teuthology.orchestra.run.smithi111.stdout:470 2023-03-29T11:01:27.919 INFO:teuthology.orchestra.run.smithi111.stdout:470.out 2023-03-29T11:01:27.919 INFO:teuthology.orchestra.run.smithi111.stdout:471 2023-03-29T11:01:27.919 INFO:teuthology.orchestra.run.smithi111.stdout:471.out 2023-03-29T11:01:27.919 INFO:teuthology.orchestra.run.smithi111.stdout:472 2023-03-29T11:01:27.919 INFO:teuthology.orchestra.run.smithi111.stdout:472.out 2023-03-29T11:01:27.919 INFO:teuthology.orchestra.run.smithi111.stdout:473 2023-03-29T11:01:27.919 INFO:teuthology.orchestra.run.smithi111.stdout:473.out 2023-03-29T11:01:27.920 INFO:teuthology.orchestra.run.smithi111.stdout:474 2023-03-29T11:01:27.920 INFO:teuthology.orchestra.run.smithi111.stdout:474.out 2023-03-29T11:01:27.920 INFO:teuthology.orchestra.run.smithi111.stdout:475 2023-03-29T11:01:27.920 INFO:teuthology.orchestra.run.smithi111.stdout:475.out 2023-03-29T11:01:27.920 INFO:teuthology.orchestra.run.smithi111.stdout:476 2023-03-29T11:01:27.920 INFO:teuthology.orchestra.run.smithi111.stdout:476.out 2023-03-29T11:01:27.920 INFO:teuthology.orchestra.run.smithi111.stdout:477 2023-03-29T11:01:27.920 INFO:teuthology.orchestra.run.smithi111.stdout:477.out 2023-03-29T11:01:27.920 INFO:teuthology.orchestra.run.smithi111.stdout:478 2023-03-29T11:01:27.920 INFO:teuthology.orchestra.run.smithi111.stdout:478.out 2023-03-29T11:01:27.920 INFO:teuthology.orchestra.run.smithi111.stdout:479 2023-03-29T11:01:27.920 INFO:teuthology.orchestra.run.smithi111.stdout:479.out 2023-03-29T11:01:27.920 INFO:teuthology.orchestra.run.smithi111.stdout:480 2023-03-29T11:01:27.920 INFO:teuthology.orchestra.run.smithi111.stdout:480.out 2023-03-29T11:01:27.920 INFO:teuthology.orchestra.run.smithi111.stdout:481 2023-03-29T11:01:27.920 INFO:teuthology.orchestra.run.smithi111.stdout:481.out 2023-03-29T11:01:27.920 INFO:teuthology.orchestra.run.smithi111.stdout:482 2023-03-29T11:01:27.920 INFO:teuthology.orchestra.run.smithi111.stdout:482.out 2023-03-29T11:01:27.921 INFO:teuthology.orchestra.run.smithi111.stdout:483 2023-03-29T11:01:27.921 INFO:teuthology.orchestra.run.smithi111.stdout:483.out 2023-03-29T11:01:27.921 INFO:teuthology.orchestra.run.smithi111.stdout:484 2023-03-29T11:01:27.921 INFO:teuthology.orchestra.run.smithi111.stdout:484.out 2023-03-29T11:01:27.921 INFO:teuthology.orchestra.run.smithi111.stdout:485 2023-03-29T11:01:27.921 INFO:teuthology.orchestra.run.smithi111.stdout:485.out 2023-03-29T11:01:27.921 INFO:teuthology.orchestra.run.smithi111.stdout:486 2023-03-29T11:01:27.921 INFO:teuthology.orchestra.run.smithi111.stdout:486.out 2023-03-29T11:01:27.921 INFO:teuthology.orchestra.run.smithi111.stdout:487 2023-03-29T11:01:27.921 INFO:teuthology.orchestra.run.smithi111.stdout:487.out 2023-03-29T11:01:27.921 INFO:teuthology.orchestra.run.smithi111.stdout:488 2023-03-29T11:01:27.921 INFO:teuthology.orchestra.run.smithi111.stdout:488.out 2023-03-29T11:01:27.921 INFO:teuthology.orchestra.run.smithi111.stdout:489 2023-03-29T11:01:27.921 INFO:teuthology.orchestra.run.smithi111.stdout:489.out 2023-03-29T11:01:27.921 INFO:teuthology.orchestra.run.smithi111.stdout:490 2023-03-29T11:01:27.921 INFO:teuthology.orchestra.run.smithi111.stdout:490.out 2023-03-29T11:01:27.921 INFO:teuthology.orchestra.run.smithi111.stdout:491 2023-03-29T11:01:27.922 INFO:teuthology.orchestra.run.smithi111.stdout:491.out 2023-03-29T11:01:27.922 INFO:teuthology.orchestra.run.smithi111.stdout:492 2023-03-29T11:01:27.922 INFO:teuthology.orchestra.run.smithi111.stdout:492.out 2023-03-29T11:01:27.922 INFO:teuthology.orchestra.run.smithi111.stdout:493 2023-03-29T11:01:27.922 INFO:teuthology.orchestra.run.smithi111.stdout:493.out 2023-03-29T11:01:27.922 INFO:teuthology.orchestra.run.smithi111.stdout:494 2023-03-29T11:01:27.922 INFO:teuthology.orchestra.run.smithi111.stdout:494.out 2023-03-29T11:01:27.922 INFO:teuthology.orchestra.run.smithi111.stdout:495 2023-03-29T11:01:27.922 INFO:teuthology.orchestra.run.smithi111.stdout:495.out 2023-03-29T11:01:27.922 INFO:teuthology.orchestra.run.smithi111.stdout:496 2023-03-29T11:01:27.922 INFO:teuthology.orchestra.run.smithi111.stdout:496.out 2023-03-29T11:01:27.922 INFO:teuthology.orchestra.run.smithi111.stdout:497 2023-03-29T11:01:27.922 INFO:teuthology.orchestra.run.smithi111.stdout:497.out 2023-03-29T11:01:27.922 INFO:teuthology.orchestra.run.smithi111.stdout:498 2023-03-29T11:01:27.922 INFO:teuthology.orchestra.run.smithi111.stdout:498.out 2023-03-29T11:01:27.922 INFO:teuthology.orchestra.run.smithi111.stdout:499 2023-03-29T11:01:27.922 INFO:teuthology.orchestra.run.smithi111.stdout:499.out 2023-03-29T11:01:27.922 INFO:teuthology.orchestra.run.smithi111.stdout:500 2023-03-29T11:01:27.923 INFO:teuthology.orchestra.run.smithi111.stdout:500.out 2023-03-29T11:01:27.923 INFO:teuthology.orchestra.run.smithi111.stdout:501 2023-03-29T11:01:27.923 INFO:teuthology.orchestra.run.smithi111.stdout:501.out 2023-03-29T11:01:27.923 INFO:teuthology.orchestra.run.smithi111.stdout:502 2023-03-29T11:01:27.923 INFO:teuthology.orchestra.run.smithi111.stdout:502.out 2023-03-29T11:01:27.923 INFO:teuthology.orchestra.run.smithi111.stdout:503 2023-03-29T11:01:27.923 INFO:teuthology.orchestra.run.smithi111.stdout:503.out 2023-03-29T11:01:27.923 INFO:teuthology.orchestra.run.smithi111.stdout:504 2023-03-29T11:01:27.923 INFO:teuthology.orchestra.run.smithi111.stdout:504.out 2023-03-29T11:01:27.923 INFO:teuthology.orchestra.run.smithi111.stdout:505 2023-03-29T11:01:27.923 INFO:teuthology.orchestra.run.smithi111.stdout:505.out 2023-03-29T11:01:27.923 INFO:teuthology.orchestra.run.smithi111.stdout:506 2023-03-29T11:01:27.923 INFO:teuthology.orchestra.run.smithi111.stdout:506.out 2023-03-29T11:01:27.923 INFO:teuthology.orchestra.run.smithi111.stdout:507 2023-03-29T11:01:27.923 INFO:teuthology.orchestra.run.smithi111.stdout:507.out 2023-03-29T11:01:27.923 INFO:teuthology.orchestra.run.smithi111.stdout:508 2023-03-29T11:01:27.923 INFO:teuthology.orchestra.run.smithi111.stdout:508.out 2023-03-29T11:01:27.923 INFO:teuthology.orchestra.run.smithi111.stdout:509 2023-03-29T11:01:27.924 INFO:teuthology.orchestra.run.smithi111.stdout:509.out 2023-03-29T11:01:27.924 INFO:teuthology.orchestra.run.smithi111.stdout:510 2023-03-29T11:01:27.924 INFO:teuthology.orchestra.run.smithi111.stdout:510.out 2023-03-29T11:01:27.924 INFO:teuthology.orchestra.run.smithi111.stdout:511 2023-03-29T11:01:27.924 INFO:teuthology.orchestra.run.smithi111.stdout:511.out 2023-03-29T11:01:27.924 INFO:teuthology.orchestra.run.smithi111.stdout:512 2023-03-29T11:01:27.924 INFO:teuthology.orchestra.run.smithi111.stdout:512.out 2023-03-29T11:01:27.924 INFO:teuthology.orchestra.run.smithi111.stdout:513 2023-03-29T11:01:27.924 INFO:teuthology.orchestra.run.smithi111.stdout:513.out 2023-03-29T11:01:27.924 INFO:teuthology.orchestra.run.smithi111.stdout:514 2023-03-29T11:01:27.924 INFO:teuthology.orchestra.run.smithi111.stdout:514.out 2023-03-29T11:01:27.924 INFO:teuthology.orchestra.run.smithi111.stdout:515 2023-03-29T11:01:27.924 INFO:teuthology.orchestra.run.smithi111.stdout:515.out 2023-03-29T11:01:27.924 INFO:teuthology.orchestra.run.smithi111.stdout:516 2023-03-29T11:01:27.924 INFO:teuthology.orchestra.run.smithi111.stdout:516.out 2023-03-29T11:01:27.924 INFO:teuthology.orchestra.run.smithi111.stdout:517 2023-03-29T11:01:27.924 INFO:teuthology.orchestra.run.smithi111.stdout:517.out 2023-03-29T11:01:27.924 INFO:teuthology.orchestra.run.smithi111.stdout:518 2023-03-29T11:01:27.925 INFO:teuthology.orchestra.run.smithi111.stdout:518.out 2023-03-29T11:01:27.925 INFO:teuthology.orchestra.run.smithi111.stdout:519 2023-03-29T11:01:27.925 INFO:teuthology.orchestra.run.smithi111.stdout:519.out 2023-03-29T11:01:27.925 INFO:teuthology.orchestra.run.smithi111.stdout:520 2023-03-29T11:01:27.925 INFO:teuthology.orchestra.run.smithi111.stdout:520.out 2023-03-29T11:01:27.925 INFO:teuthology.orchestra.run.smithi111.stdout:521 2023-03-29T11:01:27.925 INFO:teuthology.orchestra.run.smithi111.stdout:521.out 2023-03-29T11:01:27.925 INFO:teuthology.orchestra.run.smithi111.stdout:522 2023-03-29T11:01:27.925 INFO:teuthology.orchestra.run.smithi111.stdout:522.out 2023-03-29T11:01:27.925 INFO:teuthology.orchestra.run.smithi111.stdout:523 2023-03-29T11:01:27.925 INFO:teuthology.orchestra.run.smithi111.stdout:523.out 2023-03-29T11:01:27.925 INFO:teuthology.orchestra.run.smithi111.stdout:524 2023-03-29T11:01:27.925 INFO:teuthology.orchestra.run.smithi111.stdout:524.out 2023-03-29T11:01:27.925 INFO:teuthology.orchestra.run.smithi111.stdout:525 2023-03-29T11:01:27.925 INFO:teuthology.orchestra.run.smithi111.stdout:525.out 2023-03-29T11:01:27.925 INFO:teuthology.orchestra.run.smithi111.stdout:526 2023-03-29T11:01:27.925 INFO:teuthology.orchestra.run.smithi111.stdout:526.out 2023-03-29T11:01:27.925 INFO:teuthology.orchestra.run.smithi111.stdout:527 2023-03-29T11:01:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:527.out 2023-03-29T11:01:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:528 2023-03-29T11:01:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:528.out 2023-03-29T11:01:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:529 2023-03-29T11:01:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:529.out 2023-03-29T11:01:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:530 2023-03-29T11:01:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:530.out 2023-03-29T11:01:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:531 2023-03-29T11:01:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:531.out 2023-03-29T11:01:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:532 2023-03-29T11:01:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:532.out 2023-03-29T11:01:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:533 2023-03-29T11:01:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:533.out 2023-03-29T11:01:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:534 2023-03-29T11:01:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:534.out 2023-03-29T11:01:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:535 2023-03-29T11:01:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:535.out 2023-03-29T11:01:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:536 2023-03-29T11:01:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:536.out 2023-03-29T11:01:27.927 INFO:teuthology.orchestra.run.smithi111.stdout:537 2023-03-29T11:01:27.927 INFO:teuthology.orchestra.run.smithi111.stdout:537.out 2023-03-29T11:01:27.927 INFO:teuthology.orchestra.run.smithi111.stdout:538 2023-03-29T11:01:27.927 INFO:teuthology.orchestra.run.smithi111.stdout:538.out 2023-03-29T11:01:27.927 INFO:teuthology.orchestra.run.smithi111.stdout:539 2023-03-29T11:01:27.927 INFO:teuthology.orchestra.run.smithi111.stdout:539.out 2023-03-29T11:01:27.927 INFO:teuthology.orchestra.run.smithi111.stdout:540 2023-03-29T11:01:27.927 INFO:teuthology.orchestra.run.smithi111.stdout:540.out 2023-03-29T11:01:27.927 INFO:teuthology.orchestra.run.smithi111.stdout:541 2023-03-29T11:01:27.927 INFO:teuthology.orchestra.run.smithi111.stdout:541.out 2023-03-29T11:01:27.927 INFO:teuthology.orchestra.run.smithi111.stdout:542 2023-03-29T11:01:27.927 INFO:teuthology.orchestra.run.smithi111.stdout:542.out 2023-03-29T11:01:27.927 INFO:teuthology.orchestra.run.smithi111.stdout:543 2023-03-29T11:01:27.927 INFO:teuthology.orchestra.run.smithi111.stdout:543.out 2023-03-29T11:01:27.927 INFO:teuthology.orchestra.run.smithi111.stdout:544 2023-03-29T11:01:27.927 INFO:teuthology.orchestra.run.smithi111.stdout:544.out 2023-03-29T11:01:27.927 INFO:teuthology.orchestra.run.smithi111.stdout:545 2023-03-29T11:01:27.927 INFO:teuthology.orchestra.run.smithi111.stdout:545.out 2023-03-29T11:01:27.928 INFO:teuthology.orchestra.run.smithi111.stdout:546 2023-03-29T11:01:27.928 INFO:teuthology.orchestra.run.smithi111.stdout:546.out 2023-03-29T11:01:27.928 INFO:teuthology.orchestra.run.smithi111.stdout:547 2023-03-29T11:01:27.928 INFO:teuthology.orchestra.run.smithi111.stdout:547.out 2023-03-29T11:01:27.928 INFO:teuthology.orchestra.run.smithi111.stdout:548 2023-03-29T11:01:27.928 INFO:teuthology.orchestra.run.smithi111.stdout:548.out 2023-03-29T11:01:27.928 INFO:teuthology.orchestra.run.smithi111.stdout:549 2023-03-29T11:01:27.928 INFO:teuthology.orchestra.run.smithi111.stdout:549.out 2023-03-29T11:01:27.928 INFO:teuthology.orchestra.run.smithi111.stdout:550 2023-03-29T11:01:27.928 INFO:teuthology.orchestra.run.smithi111.stdout:550.out 2023-03-29T11:01:27.928 INFO:teuthology.orchestra.run.smithi111.stdout:551 2023-03-29T11:01:27.928 INFO:teuthology.orchestra.run.smithi111.stdout:551.out 2023-03-29T11:01:27.928 INFO:teuthology.orchestra.run.smithi111.stdout:552 2023-03-29T11:01:27.928 INFO:teuthology.orchestra.run.smithi111.stdout:552.out 2023-03-29T11:01:27.928 INFO:teuthology.orchestra.run.smithi111.stdout:553 2023-03-29T11:01:27.928 INFO:teuthology.orchestra.run.smithi111.stdout:553.out 2023-03-29T11:01:27.928 INFO:teuthology.orchestra.run.smithi111.stdout:554 2023-03-29T11:01:27.928 INFO:teuthology.orchestra.run.smithi111.stdout:554.out 2023-03-29T11:01:27.929 INFO:teuthology.orchestra.run.smithi111.stdout:555 2023-03-29T11:01:27.929 INFO:teuthology.orchestra.run.smithi111.stdout:555.out 2023-03-29T11:01:27.929 INFO:teuthology.orchestra.run.smithi111.stdout:556 2023-03-29T11:01:27.929 INFO:teuthology.orchestra.run.smithi111.stdout:556.out 2023-03-29T11:01:27.929 INFO:teuthology.orchestra.run.smithi111.stdout:557 2023-03-29T11:01:27.929 INFO:teuthology.orchestra.run.smithi111.stdout:557.out 2023-03-29T11:01:27.929 INFO:teuthology.orchestra.run.smithi111.stdout:558 2023-03-29T11:01:27.929 INFO:teuthology.orchestra.run.smithi111.stdout:558.out 2023-03-29T11:01:27.929 INFO:teuthology.orchestra.run.smithi111.stdout:559 2023-03-29T11:01:27.929 INFO:teuthology.orchestra.run.smithi111.stdout:559.out 2023-03-29T11:01:27.929 INFO:teuthology.orchestra.run.smithi111.stdout:560 2023-03-29T11:01:27.929 INFO:teuthology.orchestra.run.smithi111.stdout:560.out 2023-03-29T11:01:27.929 INFO:teuthology.orchestra.run.smithi111.stdout:561 2023-03-29T11:01:27.929 INFO:teuthology.orchestra.run.smithi111.stdout:561.out 2023-03-29T11:01:27.929 INFO:teuthology.orchestra.run.smithi111.stdout:562 2023-03-29T11:01:27.929 INFO:teuthology.orchestra.run.smithi111.stdout:562.out 2023-03-29T11:01:27.929 INFO:teuthology.orchestra.run.smithi111.stdout:563 2023-03-29T11:01:27.929 INFO:teuthology.orchestra.run.smithi111.stdout:563.out 2023-03-29T11:01:27.930 INFO:teuthology.orchestra.run.smithi111.stdout:564 2023-03-29T11:01:27.930 INFO:teuthology.orchestra.run.smithi111.stdout:564.out 2023-03-29T11:01:27.930 INFO:teuthology.orchestra.run.smithi111.stdout:565 2023-03-29T11:01:27.930 INFO:teuthology.orchestra.run.smithi111.stdout:565.out 2023-03-29T11:01:27.930 INFO:teuthology.orchestra.run.smithi111.stdout:566 2023-03-29T11:01:27.930 INFO:teuthology.orchestra.run.smithi111.stdout:566.out 2023-03-29T11:01:27.930 INFO:teuthology.orchestra.run.smithi111.stdout:567 2023-03-29T11:01:27.930 INFO:teuthology.orchestra.run.smithi111.stdout:567.out 2023-03-29T11:01:27.930 INFO:teuthology.orchestra.run.smithi111.stdout:568 2023-03-29T11:01:27.930 INFO:teuthology.orchestra.run.smithi111.stdout:568.out 2023-03-29T11:01:27.930 INFO:teuthology.orchestra.run.smithi111.stdout:569 2023-03-29T11:01:27.930 INFO:teuthology.orchestra.run.smithi111.stdout:569.out 2023-03-29T11:01:27.930 INFO:teuthology.orchestra.run.smithi111.stdout:570 2023-03-29T11:01:27.930 INFO:teuthology.orchestra.run.smithi111.stdout:570.out 2023-03-29T11:01:27.930 INFO:teuthology.orchestra.run.smithi111.stdout:571 2023-03-29T11:01:27.930 INFO:teuthology.orchestra.run.smithi111.stdout:571.out 2023-03-29T11:01:27.930 INFO:teuthology.orchestra.run.smithi111.stdout:572 2023-03-29T11:01:27.930 INFO:teuthology.orchestra.run.smithi111.stdout:572.out 2023-03-29T11:01:27.931 INFO:teuthology.orchestra.run.smithi111.stdout:573 2023-03-29T11:01:27.931 INFO:teuthology.orchestra.run.smithi111.stdout:573.out 2023-03-29T11:01:27.931 INFO:teuthology.orchestra.run.smithi111.stdout:574 2023-03-29T11:01:27.931 INFO:teuthology.orchestra.run.smithi111.stdout:574.out 2023-03-29T11:01:27.931 INFO:teuthology.orchestra.run.smithi111.stdout:575 2023-03-29T11:01:27.931 INFO:teuthology.orchestra.run.smithi111.stdout:575.out 2023-03-29T11:01:27.931 INFO:teuthology.orchestra.run.smithi111.stdout:576 2023-03-29T11:01:27.931 INFO:teuthology.orchestra.run.smithi111.stdout:576.out 2023-03-29T11:01:27.931 INFO:teuthology.orchestra.run.smithi111.stdout:577 2023-03-29T11:01:27.931 INFO:teuthology.orchestra.run.smithi111.stdout:577.out 2023-03-29T11:01:27.931 INFO:teuthology.orchestra.run.smithi111.stdout:578 2023-03-29T11:01:27.931 INFO:teuthology.orchestra.run.smithi111.stdout:578.out 2023-03-29T11:01:27.931 INFO:teuthology.orchestra.run.smithi111.stdout:579 2023-03-29T11:01:27.931 INFO:teuthology.orchestra.run.smithi111.stdout:579.out 2023-03-29T11:01:27.931 INFO:teuthology.orchestra.run.smithi111.stdout:580 2023-03-29T11:01:27.931 INFO:teuthology.orchestra.run.smithi111.stdout:580.out 2023-03-29T11:01:27.931 INFO:teuthology.orchestra.run.smithi111.stdout:581 2023-03-29T11:01:27.931 INFO:teuthology.orchestra.run.smithi111.stdout:581.out 2023-03-29T11:01:27.931 INFO:teuthology.orchestra.run.smithi111.stdout:582 2023-03-29T11:01:27.932 INFO:teuthology.orchestra.run.smithi111.stdout:582.out 2023-03-29T11:01:27.932 INFO:teuthology.orchestra.run.smithi111.stdout:583 2023-03-29T11:01:27.932 INFO:teuthology.orchestra.run.smithi111.stdout:583.out 2023-03-29T11:01:27.932 INFO:teuthology.orchestra.run.smithi111.stdout:584 2023-03-29T11:01:27.932 INFO:teuthology.orchestra.run.smithi111.stdout:584.out 2023-03-29T11:01:27.932 INFO:teuthology.orchestra.run.smithi111.stdout:585 2023-03-29T11:01:27.932 INFO:teuthology.orchestra.run.smithi111.stdout:585.out 2023-03-29T11:01:27.932 INFO:teuthology.orchestra.run.smithi111.stdout:586 2023-03-29T11:01:27.932 INFO:teuthology.orchestra.run.smithi111.stdout:586.out 2023-03-29T11:01:27.932 INFO:teuthology.orchestra.run.smithi111.stdout:587 2023-03-29T11:01:27.932 INFO:teuthology.orchestra.run.smithi111.stdout:587.out 2023-03-29T11:01:27.932 INFO:teuthology.orchestra.run.smithi111.stdout:588 2023-03-29T11:01:27.932 INFO:teuthology.orchestra.run.smithi111.stdout:588.out 2023-03-29T11:01:27.932 INFO:teuthology.orchestra.run.smithi111.stdout:589 2023-03-29T11:01:27.932 INFO:teuthology.orchestra.run.smithi111.stdout:589.out 2023-03-29T11:01:27.932 INFO:teuthology.orchestra.run.smithi111.stdout:590 2023-03-29T11:01:27.932 INFO:teuthology.orchestra.run.smithi111.stdout:590.out 2023-03-29T11:01:27.932 INFO:teuthology.orchestra.run.smithi111.stdout:591 2023-03-29T11:01:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:591.out 2023-03-29T11:01:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:592 2023-03-29T11:01:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:592.out 2023-03-29T11:01:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:593 2023-03-29T11:01:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:593.out 2023-03-29T11:01:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:594 2023-03-29T11:01:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:594.out 2023-03-29T11:01:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:595 2023-03-29T11:01:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:595.out 2023-03-29T11:01:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:596 2023-03-29T11:01:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:596.out 2023-03-29T11:01:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:597 2023-03-29T11:01:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:597.out 2023-03-29T11:01:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:598 2023-03-29T11:01:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:598.out 2023-03-29T11:01:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:599 2023-03-29T11:01:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:599.out 2023-03-29T11:01:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:600 2023-03-29T11:01:27.934 INFO:teuthology.orchestra.run.smithi111.stdout:600.out 2023-03-29T11:01:27.934 INFO:teuthology.orchestra.run.smithi111.stdout:601 2023-03-29T11:01:27.934 INFO:teuthology.orchestra.run.smithi111.stdout:601.out 2023-03-29T11:01:27.934 INFO:teuthology.orchestra.run.smithi111.stdout:602 2023-03-29T11:01:27.934 INFO:teuthology.orchestra.run.smithi111.stdout:602.out 2023-03-29T11:01:27.934 INFO:teuthology.orchestra.run.smithi111.stdout:603 2023-03-29T11:01:27.934 INFO:teuthology.orchestra.run.smithi111.stdout:603.out 2023-03-29T11:01:27.934 INFO:teuthology.orchestra.run.smithi111.stdout:604 2023-03-29T11:01:27.934 INFO:teuthology.orchestra.run.smithi111.stdout:604.out 2023-03-29T11:01:27.934 INFO:teuthology.orchestra.run.smithi111.stdout:605 2023-03-29T11:01:27.934 INFO:teuthology.orchestra.run.smithi111.stdout:605.out 2023-03-29T11:01:27.934 INFO:teuthology.orchestra.run.smithi111.stdout:606 2023-03-29T11:01:27.934 INFO:teuthology.orchestra.run.smithi111.stdout:606.out 2023-03-29T11:01:27.934 INFO:teuthology.orchestra.run.smithi111.stdout:607 2023-03-29T11:01:27.934 INFO:teuthology.orchestra.run.smithi111.stdout:607.out 2023-03-29T11:01:27.934 INFO:teuthology.orchestra.run.smithi111.stdout:608 2023-03-29T11:01:27.934 INFO:teuthology.orchestra.run.smithi111.stdout:608.out 2023-03-29T11:01:27.935 INFO:teuthology.orchestra.run.smithi111.stdout:609 2023-03-29T11:01:27.935 INFO:teuthology.orchestra.run.smithi111.stdout:609.out 2023-03-29T11:01:27.935 INFO:teuthology.orchestra.run.smithi111.stdout:610 2023-03-29T11:01:27.935 INFO:teuthology.orchestra.run.smithi111.stdout:610.out 2023-03-29T11:01:27.935 INFO:teuthology.orchestra.run.smithi111.stdout:611 2023-03-29T11:01:27.935 INFO:teuthology.orchestra.run.smithi111.stdout:611.out 2023-03-29T11:01:27.935 INFO:teuthology.orchestra.run.smithi111.stdout:612 2023-03-29T11:01:27.935 INFO:teuthology.orchestra.run.smithi111.stdout:612.out 2023-03-29T11:01:27.935 INFO:teuthology.orchestra.run.smithi111.stdout:613 2023-03-29T11:01:27.935 INFO:teuthology.orchestra.run.smithi111.stdout:613.out 2023-03-29T11:01:27.935 INFO:teuthology.orchestra.run.smithi111.stdout:614 2023-03-29T11:01:27.935 INFO:teuthology.orchestra.run.smithi111.stdout:614.out 2023-03-29T11:01:27.935 INFO:teuthology.orchestra.run.smithi111.stdout:615 2023-03-29T11:01:27.935 INFO:teuthology.orchestra.run.smithi111.stdout:615.out 2023-03-29T11:01:27.935 INFO:teuthology.orchestra.run.smithi111.stdout:616 2023-03-29T11:01:27.935 INFO:teuthology.orchestra.run.smithi111.stdout:616.out 2023-03-29T11:01:27.935 INFO:teuthology.orchestra.run.smithi111.stdout:617 2023-03-29T11:01:27.935 INFO:teuthology.orchestra.run.smithi111.stdout:617.out 2023-03-29T11:01:27.936 INFO:teuthology.orchestra.run.smithi111.stdout:618 2023-03-29T11:01:27.936 INFO:teuthology.orchestra.run.smithi111.stdout:618.out 2023-03-29T11:01:27.936 INFO:teuthology.orchestra.run.smithi111.stdout:619 2023-03-29T11:01:27.936 INFO:teuthology.orchestra.run.smithi111.stdout:619.out 2023-03-29T11:01:27.936 INFO:teuthology.orchestra.run.smithi111.stdout:620 2023-03-29T11:01:27.936 INFO:teuthology.orchestra.run.smithi111.stdout:620.out 2023-03-29T11:01:27.936 INFO:teuthology.orchestra.run.smithi111.stdout:621 2023-03-29T11:01:27.936 INFO:teuthology.orchestra.run.smithi111.stdout:621.out 2023-03-29T11:01:27.936 INFO:teuthology.orchestra.run.smithi111.stdout:622 2023-03-29T11:01:27.936 INFO:teuthology.orchestra.run.smithi111.stdout:622.out 2023-03-29T11:01:27.936 INFO:teuthology.orchestra.run.smithi111.stdout:623 2023-03-29T11:01:27.936 INFO:teuthology.orchestra.run.smithi111.stdout:623.out 2023-03-29T11:01:27.936 INFO:teuthology.orchestra.run.smithi111.stdout:624 2023-03-29T11:01:27.936 INFO:teuthology.orchestra.run.smithi111.stdout:624.out 2023-03-29T11:01:27.936 INFO:teuthology.orchestra.run.smithi111.stdout:625 2023-03-29T11:01:27.936 INFO:teuthology.orchestra.run.smithi111.stdout:625.out 2023-03-29T11:01:27.936 INFO:teuthology.orchestra.run.smithi111.stdout:626 2023-03-29T11:01:27.936 INFO:teuthology.orchestra.run.smithi111.stdout:626.out 2023-03-29T11:01:27.936 INFO:teuthology.orchestra.run.smithi111.stdout:627 2023-03-29T11:01:27.937 INFO:teuthology.orchestra.run.smithi111.stdout:627.out 2023-03-29T11:01:27.937 INFO:teuthology.orchestra.run.smithi111.stdout:628 2023-03-29T11:01:27.937 INFO:teuthology.orchestra.run.smithi111.stdout:628.out 2023-03-29T11:01:27.937 INFO:teuthology.orchestra.run.smithi111.stdout:629 2023-03-29T11:01:27.937 INFO:teuthology.orchestra.run.smithi111.stdout:629.out 2023-03-29T11:01:27.937 INFO:teuthology.orchestra.run.smithi111.stdout:630 2023-03-29T11:01:27.937 INFO:teuthology.orchestra.run.smithi111.stdout:630.out 2023-03-29T11:01:27.937 INFO:teuthology.orchestra.run.smithi111.stdout:631 2023-03-29T11:01:27.937 INFO:teuthology.orchestra.run.smithi111.stdout:631.out 2023-03-29T11:01:27.937 INFO:teuthology.orchestra.run.smithi111.stdout:632 2023-03-29T11:01:27.937 INFO:teuthology.orchestra.run.smithi111.stdout:632.out 2023-03-29T11:01:27.937 INFO:teuthology.orchestra.run.smithi111.stdout:633 2023-03-29T11:01:27.937 INFO:teuthology.orchestra.run.smithi111.stdout:633.out 2023-03-29T11:01:27.937 INFO:teuthology.orchestra.run.smithi111.stdout:634 2023-03-29T11:01:27.937 INFO:teuthology.orchestra.run.smithi111.stdout:634.out 2023-03-29T11:01:27.937 INFO:teuthology.orchestra.run.smithi111.stdout:635 2023-03-29T11:01:27.937 INFO:teuthology.orchestra.run.smithi111.stdout:635.out 2023-03-29T11:01:27.938 INFO:teuthology.orchestra.run.smithi111.stdout:636 2023-03-29T11:01:27.938 INFO:teuthology.orchestra.run.smithi111.stdout:636.out 2023-03-29T11:01:27.938 INFO:teuthology.orchestra.run.smithi111.stdout:637 2023-03-29T11:01:27.938 INFO:teuthology.orchestra.run.smithi111.stdout:637.out 2023-03-29T11:01:27.938 INFO:teuthology.orchestra.run.smithi111.stdout:638 2023-03-29T11:01:27.938 INFO:teuthology.orchestra.run.smithi111.stdout:638.out 2023-03-29T11:01:27.938 INFO:teuthology.orchestra.run.smithi111.stdout:639 2023-03-29T11:01:27.938 INFO:teuthology.orchestra.run.smithi111.stdout:639.out 2023-03-29T11:01:27.938 INFO:teuthology.orchestra.run.smithi111.stdout:640 2023-03-29T11:01:27.938 INFO:teuthology.orchestra.run.smithi111.stdout:640.out 2023-03-29T11:01:27.938 INFO:teuthology.orchestra.run.smithi111.stdout:641 2023-03-29T11:01:27.938 INFO:teuthology.orchestra.run.smithi111.stdout:641.out 2023-03-29T11:01:27.938 INFO:teuthology.orchestra.run.smithi111.stdout:642 2023-03-29T11:01:27.938 INFO:teuthology.orchestra.run.smithi111.stdout:642.out 2023-03-29T11:01:27.938 INFO:teuthology.orchestra.run.smithi111.stdout:643 2023-03-29T11:01:27.938 INFO:teuthology.orchestra.run.smithi111.stdout:643.out 2023-03-29T11:01:27.938 INFO:teuthology.orchestra.run.smithi111.stdout:644 2023-03-29T11:01:27.938 INFO:teuthology.orchestra.run.smithi111.stdout:644.out 2023-03-29T11:01:27.939 INFO:teuthology.orchestra.run.smithi111.stdout:645 2023-03-29T11:01:27.939 INFO:teuthology.orchestra.run.smithi111.stdout:645.out 2023-03-29T11:01:27.939 INFO:teuthology.orchestra.run.smithi111.stdout:646 2023-03-29T11:01:27.939 INFO:teuthology.orchestra.run.smithi111.stdout:646.out 2023-03-29T11:01:27.939 INFO:teuthology.orchestra.run.smithi111.stdout:647 2023-03-29T11:01:27.939 INFO:teuthology.orchestra.run.smithi111.stdout:647.out 2023-03-29T11:01:27.939 INFO:teuthology.orchestra.run.smithi111.stdout:648 2023-03-29T11:01:27.939 INFO:teuthology.orchestra.run.smithi111.stdout:648.out 2023-03-29T11:01:27.939 INFO:teuthology.orchestra.run.smithi111.stdout:649 2023-03-29T11:01:27.939 INFO:teuthology.orchestra.run.smithi111.stdout:649.out 2023-03-29T11:01:27.939 INFO:teuthology.orchestra.run.smithi111.stdout:650 2023-03-29T11:01:27.939 INFO:teuthology.orchestra.run.smithi111.stdout:650.out 2023-03-29T11:01:27.939 INFO:teuthology.orchestra.run.smithi111.stdout:651 2023-03-29T11:01:27.939 INFO:teuthology.orchestra.run.smithi111.stdout:651.out 2023-03-29T11:01:27.939 INFO:teuthology.orchestra.run.smithi111.stdout:652 2023-03-29T11:01:27.939 INFO:teuthology.orchestra.run.smithi111.stdout:652.out 2023-03-29T11:01:27.939 INFO:teuthology.orchestra.run.smithi111.stdout:653 2023-03-29T11:01:27.939 INFO:teuthology.orchestra.run.smithi111.stdout:653.out 2023-03-29T11:01:27.940 INFO:teuthology.orchestra.run.smithi111.stdout:654 2023-03-29T11:01:27.940 INFO:teuthology.orchestra.run.smithi111.stdout:654.out 2023-03-29T11:01:27.940 INFO:teuthology.orchestra.run.smithi111.stdout:655 2023-03-29T11:01:27.940 INFO:teuthology.orchestra.run.smithi111.stdout:655.out 2023-03-29T11:01:27.940 INFO:teuthology.orchestra.run.smithi111.stdout:656 2023-03-29T11:01:27.940 INFO:teuthology.orchestra.run.smithi111.stdout:656.out 2023-03-29T11:01:27.940 INFO:teuthology.orchestra.run.smithi111.stdout:657 2023-03-29T11:01:27.940 INFO:teuthology.orchestra.run.smithi111.stdout:657.out 2023-03-29T11:01:27.940 INFO:teuthology.orchestra.run.smithi111.stdout:658 2023-03-29T11:01:27.940 INFO:teuthology.orchestra.run.smithi111.stdout:658.out 2023-03-29T11:01:27.940 INFO:teuthology.orchestra.run.smithi111.stdout:659 2023-03-29T11:01:27.940 INFO:teuthology.orchestra.run.smithi111.stdout:659.out 2023-03-29T11:01:27.940 INFO:teuthology.orchestra.run.smithi111.stdout:660 2023-03-29T11:01:27.940 INFO:teuthology.orchestra.run.smithi111.stdout:660.out 2023-03-29T11:01:27.940 INFO:teuthology.orchestra.run.smithi111.stdout:661 2023-03-29T11:01:27.940 INFO:teuthology.orchestra.run.smithi111.stdout:661.out 2023-03-29T11:01:27.940 INFO:teuthology.orchestra.run.smithi111.stdout:662 2023-03-29T11:01:27.941 INFO:teuthology.orchestra.run.smithi111.stdout:662.out 2023-03-29T11:01:27.941 INFO:teuthology.orchestra.run.smithi111.stdout:663 2023-03-29T11:01:27.941 INFO:teuthology.orchestra.run.smithi111.stdout:663.out 2023-03-29T11:01:27.941 INFO:teuthology.orchestra.run.smithi111.stdout:664 2023-03-29T11:01:27.941 INFO:teuthology.orchestra.run.smithi111.stdout:664.out 2023-03-29T11:01:27.941 INFO:teuthology.orchestra.run.smithi111.stdout:665 2023-03-29T11:01:27.941 INFO:teuthology.orchestra.run.smithi111.stdout:665.out 2023-03-29T11:01:27.941 INFO:teuthology.orchestra.run.smithi111.stdout:666 2023-03-29T11:01:27.941 INFO:teuthology.orchestra.run.smithi111.stdout:666.out 2023-03-29T11:01:27.941 INFO:teuthology.orchestra.run.smithi111.stdout:667 2023-03-29T11:01:27.941 INFO:teuthology.orchestra.run.smithi111.stdout:667.out 2023-03-29T11:01:27.941 INFO:teuthology.orchestra.run.smithi111.stdout:668 2023-03-29T11:01:27.941 INFO:teuthology.orchestra.run.smithi111.stdout:668.out 2023-03-29T11:01:27.941 INFO:teuthology.orchestra.run.smithi111.stdout:669 2023-03-29T11:01:27.941 INFO:teuthology.orchestra.run.smithi111.stdout:669.out 2023-03-29T11:01:27.941 INFO:teuthology.orchestra.run.smithi111.stdout:670 2023-03-29T11:01:27.941 INFO:teuthology.orchestra.run.smithi111.stdout:670.out 2023-03-29T11:01:27.941 INFO:teuthology.orchestra.run.smithi111.stdout:671 2023-03-29T11:01:27.942 INFO:teuthology.orchestra.run.smithi111.stdout:671.out 2023-03-29T11:01:27.942 INFO:teuthology.orchestra.run.smithi111.stdout:672 2023-03-29T11:01:27.942 INFO:teuthology.orchestra.run.smithi111.stdout:672.out 2023-03-29T11:01:27.942 INFO:teuthology.orchestra.run.smithi111.stdout:673 2023-03-29T11:01:27.942 INFO:teuthology.orchestra.run.smithi111.stdout:673.out 2023-03-29T11:01:27.942 INFO:teuthology.orchestra.run.smithi111.stdout:674 2023-03-29T11:01:27.942 INFO:teuthology.orchestra.run.smithi111.stdout:674.out 2023-03-29T11:01:27.942 INFO:teuthology.orchestra.run.smithi111.stdout:675 2023-03-29T11:01:27.942 INFO:teuthology.orchestra.run.smithi111.stdout:675.out 2023-03-29T11:01:27.942 INFO:teuthology.orchestra.run.smithi111.stdout:676 2023-03-29T11:01:27.942 INFO:teuthology.orchestra.run.smithi111.stdout:676.out 2023-03-29T11:01:27.942 INFO:teuthology.orchestra.run.smithi111.stdout:677 2023-03-29T11:01:27.942 INFO:teuthology.orchestra.run.smithi111.stdout:677.out 2023-03-29T11:01:27.942 INFO:teuthology.orchestra.run.smithi111.stdout:678 2023-03-29T11:01:27.942 INFO:teuthology.orchestra.run.smithi111.stdout:678.out 2023-03-29T11:01:27.942 INFO:teuthology.orchestra.run.smithi111.stdout:679 2023-03-29T11:01:27.942 INFO:teuthology.orchestra.run.smithi111.stdout:679.out 2023-03-29T11:01:27.942 INFO:teuthology.orchestra.run.smithi111.stdout:680 2023-03-29T11:01:27.943 INFO:teuthology.orchestra.run.smithi111.stdout:680.out 2023-03-29T11:01:27.943 INFO:teuthology.orchestra.run.smithi111.stdout:681 2023-03-29T11:01:27.943 INFO:teuthology.orchestra.run.smithi111.stdout:681.out 2023-03-29T11:01:27.943 INFO:teuthology.orchestra.run.smithi111.stdout:682 2023-03-29T11:01:27.943 INFO:teuthology.orchestra.run.smithi111.stdout:682.out 2023-03-29T11:01:27.943 INFO:teuthology.orchestra.run.smithi111.stdout:683 2023-03-29T11:01:27.943 INFO:teuthology.orchestra.run.smithi111.stdout:683.out 2023-03-29T11:01:27.943 INFO:teuthology.orchestra.run.smithi111.stdout:684 2023-03-29T11:01:27.943 INFO:teuthology.orchestra.run.smithi111.stdout:684.out 2023-03-29T11:01:27.943 INFO:teuthology.orchestra.run.smithi111.stdout:685 2023-03-29T11:01:27.943 INFO:teuthology.orchestra.run.smithi111.stdout:685.out 2023-03-29T11:01:27.943 INFO:teuthology.orchestra.run.smithi111.stdout:686 2023-03-29T11:01:27.943 INFO:teuthology.orchestra.run.smithi111.stdout:686.out 2023-03-29T11:01:27.943 INFO:teuthology.orchestra.run.smithi111.stdout:687 2023-03-29T11:01:27.943 INFO:teuthology.orchestra.run.smithi111.stdout:687.out 2023-03-29T11:01:27.943 INFO:teuthology.orchestra.run.smithi111.stdout:688 2023-03-29T11:01:27.943 INFO:teuthology.orchestra.run.smithi111.stdout:688.out 2023-03-29T11:01:27.943 INFO:teuthology.orchestra.run.smithi111.stdout:689 2023-03-29T11:01:27.944 INFO:teuthology.orchestra.run.smithi111.stdout:689.out 2023-03-29T11:01:27.944 INFO:teuthology.orchestra.run.smithi111.stdout:690 2023-03-29T11:01:27.944 INFO:teuthology.orchestra.run.smithi111.stdout:690.out 2023-03-29T11:01:27.944 INFO:teuthology.orchestra.run.smithi111.stdout:691 2023-03-29T11:01:27.944 INFO:teuthology.orchestra.run.smithi111.stdout:691.out 2023-03-29T11:01:27.944 INFO:teuthology.orchestra.run.smithi111.stdout:692 2023-03-29T11:01:27.944 INFO:teuthology.orchestra.run.smithi111.stdout:692.out 2023-03-29T11:01:27.944 INFO:teuthology.orchestra.run.smithi111.stdout:693 2023-03-29T11:01:27.944 INFO:teuthology.orchestra.run.smithi111.stdout:693.out 2023-03-29T11:01:27.944 INFO:teuthology.orchestra.run.smithi111.stdout:694 2023-03-29T11:01:27.944 INFO:teuthology.orchestra.run.smithi111.stdout:694.out 2023-03-29T11:01:27.944 INFO:teuthology.orchestra.run.smithi111.stdout:695 2023-03-29T11:01:27.944 INFO:teuthology.orchestra.run.smithi111.stdout:695.out 2023-03-29T11:01:27.944 INFO:teuthology.orchestra.run.smithi111.stdout:696 2023-03-29T11:01:27.944 INFO:teuthology.orchestra.run.smithi111.stdout:696.out 2023-03-29T11:01:27.944 INFO:teuthology.orchestra.run.smithi111.stdout:697 2023-03-29T11:01:27.944 INFO:teuthology.orchestra.run.smithi111.stdout:697.out 2023-03-29T11:01:27.944 INFO:teuthology.orchestra.run.smithi111.stdout:698 2023-03-29T11:01:27.945 INFO:teuthology.orchestra.run.smithi111.stdout:698.out 2023-03-29T11:01:27.945 INFO:teuthology.orchestra.run.smithi111.stdout:699 2023-03-29T11:01:27.945 INFO:teuthology.orchestra.run.smithi111.stdout:699.out 2023-03-29T11:01:27.945 INFO:teuthology.orchestra.run.smithi111.stdout:700 2023-03-29T11:01:27.945 INFO:teuthology.orchestra.run.smithi111.stdout:700.out 2023-03-29T11:01:27.945 INFO:teuthology.orchestra.run.smithi111.stdout:701 2023-03-29T11:01:27.945 INFO:teuthology.orchestra.run.smithi111.stdout:701.out 2023-03-29T11:01:27.945 INFO:teuthology.orchestra.run.smithi111.stdout:702 2023-03-29T11:01:27.945 INFO:teuthology.orchestra.run.smithi111.stdout:702.out 2023-03-29T11:01:27.945 INFO:teuthology.orchestra.run.smithi111.stdout:703 2023-03-29T11:01:27.945 INFO:teuthology.orchestra.run.smithi111.stdout:703.out 2023-03-29T11:01:27.945 INFO:teuthology.orchestra.run.smithi111.stdout:704 2023-03-29T11:01:27.945 INFO:teuthology.orchestra.run.smithi111.stdout:704.out 2023-03-29T11:01:27.945 INFO:teuthology.orchestra.run.smithi111.stdout:705 2023-03-29T11:01:27.945 INFO:teuthology.orchestra.run.smithi111.stdout:705.out 2023-03-29T11:01:27.945 INFO:teuthology.orchestra.run.smithi111.stdout:706 2023-03-29T11:01:27.945 INFO:teuthology.orchestra.run.smithi111.stdout:706.out 2023-03-29T11:01:27.945 INFO:teuthology.orchestra.run.smithi111.stdout:707 2023-03-29T11:01:27.946 INFO:teuthology.orchestra.run.smithi111.stdout:707.out 2023-03-29T11:01:27.946 INFO:teuthology.orchestra.run.smithi111.stdout:708 2023-03-29T11:01:27.946 INFO:teuthology.orchestra.run.smithi111.stdout:708.out 2023-03-29T11:01:27.946 INFO:teuthology.orchestra.run.smithi111.stdout:709 2023-03-29T11:01:27.946 INFO:teuthology.orchestra.run.smithi111.stdout:709.out 2023-03-29T11:01:27.946 INFO:teuthology.orchestra.run.smithi111.stdout:710 2023-03-29T11:01:27.946 INFO:teuthology.orchestra.run.smithi111.stdout:710.out 2023-03-29T11:01:27.946 INFO:teuthology.orchestra.run.smithi111.stdout:711 2023-03-29T11:01:27.946 INFO:teuthology.orchestra.run.smithi111.stdout:711.out 2023-03-29T11:01:27.946 INFO:teuthology.orchestra.run.smithi111.stdout:712 2023-03-29T11:01:27.946 INFO:teuthology.orchestra.run.smithi111.stdout:712.out 2023-03-29T11:01:27.946 INFO:teuthology.orchestra.run.smithi111.stdout:713 2023-03-29T11:01:27.946 INFO:teuthology.orchestra.run.smithi111.stdout:713.out 2023-03-29T11:01:27.946 INFO:teuthology.orchestra.run.smithi111.stdout:714 2023-03-29T11:01:27.946 INFO:teuthology.orchestra.run.smithi111.stdout:714.out 2023-03-29T11:01:27.946 INFO:teuthology.orchestra.run.smithi111.stdout:715 2023-03-29T11:01:27.946 INFO:teuthology.orchestra.run.smithi111.stdout:715.out 2023-03-29T11:01:27.946 INFO:teuthology.orchestra.run.smithi111.stdout:716 2023-03-29T11:01:27.947 INFO:teuthology.orchestra.run.smithi111.stdout:716.out 2023-03-29T11:01:27.947 INFO:teuthology.orchestra.run.smithi111.stdout:717 2023-03-29T11:01:27.947 INFO:teuthology.orchestra.run.smithi111.stdout:717.out 2023-03-29T11:01:27.947 INFO:teuthology.orchestra.run.smithi111.stdout:718 2023-03-29T11:01:27.947 INFO:teuthology.orchestra.run.smithi111.stdout:718.out 2023-03-29T11:01:27.947 INFO:teuthology.orchestra.run.smithi111.stdout:719 2023-03-29T11:01:27.947 INFO:teuthology.orchestra.run.smithi111.stdout:719.out 2023-03-29T11:01:27.947 INFO:teuthology.orchestra.run.smithi111.stdout:720 2023-03-29T11:01:27.947 INFO:teuthology.orchestra.run.smithi111.stdout:720.out 2023-03-29T11:01:27.947 INFO:teuthology.orchestra.run.smithi111.stdout:721 2023-03-29T11:01:27.947 INFO:teuthology.orchestra.run.smithi111.stdout:721.out 2023-03-29T11:01:27.947 INFO:teuthology.orchestra.run.smithi111.stdout:722 2023-03-29T11:01:27.947 INFO:teuthology.orchestra.run.smithi111.stdout:722.out 2023-03-29T11:01:27.947 INFO:teuthology.orchestra.run.smithi111.stdout:723 2023-03-29T11:01:27.947 INFO:teuthology.orchestra.run.smithi111.stdout:723.out 2023-03-29T11:01:27.947 INFO:teuthology.orchestra.run.smithi111.stdout:724 2023-03-29T11:01:27.947 INFO:teuthology.orchestra.run.smithi111.stdout:724.out 2023-03-29T11:01:27.947 INFO:teuthology.orchestra.run.smithi111.stdout:725 2023-03-29T11:01:27.948 INFO:teuthology.orchestra.run.smithi111.stdout:725.out 2023-03-29T11:01:27.948 INFO:teuthology.orchestra.run.smithi111.stdout:726 2023-03-29T11:01:27.948 INFO:teuthology.orchestra.run.smithi111.stdout:726.out 2023-03-29T11:01:27.948 INFO:teuthology.orchestra.run.smithi111.stdout:727 2023-03-29T11:01:27.948 INFO:teuthology.orchestra.run.smithi111.stdout:727.out 2023-03-29T11:01:27.948 INFO:teuthology.orchestra.run.smithi111.stdout:group.list 2023-03-29T11:01:27.948 INFO:teuthology.orchestra.run.smithi111.stdout:Makefile 2023-03-29T11:01:27.949 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/001) 2023-03-29T11:01:28.347 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:01:28.358 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:01:28.358 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:01:33.428 INFO:teuthology.orchestra.run.smithi111.stdout:generic/001 5s 2023-03-29T11:01:33.439 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/001 2023-03-29T11:01:33.442 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:01:33.445 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:01:33.448 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi111.stderr:ceph-fuse[130136]: fuse finished with error 0 and tester_r 0 2023-03-29T11:01:33.466 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:01:33.466 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/002) 2023-03-29T11:01:33.849 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:01:33.860 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:01:33.860 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:01:34.638 INFO:teuthology.orchestra.run.smithi111.stdout:generic/002 1s 2023-03-29T11:01:34.650 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/002 2023-03-29T11:01:34.650 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:01:34.655 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:01:34.676 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:01:34.676 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/003) 2023-03-29T11:01:35.092 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:01:35.103 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:01:35.103 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:01:35.375 INFO:teuthology.orchestra.run.smithi111.stdout:generic/003 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:01:35.383 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/003 2023-03-29T11:01:35.383 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/003 2023-03-29T11:01:35.383 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:01:35.389 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:01:35.410 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:01:35.411 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/004) 2023-03-29T11:01:35.795 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:01:35.806 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:01:35.806 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:01:36.104 INFO:teuthology.orchestra.run.smithi111.stdout:generic/004 [not run] O_TMPFILE is not supported 2023-03-29T11:01:36.112 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/004 2023-03-29T11:01:36.112 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/004 2023-03-29T11:01:36.112 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:01:36.117 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:01:36.136 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:01:36.136 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/005) 2023-03-29T11:01:36.518 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:01:36.529 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:01:36.529 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:01:37.513 INFO:teuthology.orchestra.run.smithi111.stdout:generic/005 1s 2023-03-29T11:01:37.524 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/005 2023-03-29T11:01:37.525 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:01:37.530 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:01:37.552 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:01:37.552 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/006) 2023-03-29T11:01:37.968 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:01:37.980 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:01:37.980 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:01:53.036 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:01:53.046 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:02:15.303 INFO:teuthology.orchestra.run.smithi111.stdout:generic/006 37s 2023-03-29T11:02:15.314 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/006 2023-03-29T11:02:15.314 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:02:15.319 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:02:15.342 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:02:15.342 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/007) 2023-03-29T11:02:15.776 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:02:15.788 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:02:15.789 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:02:23.217 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:02:23.225 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:02:53.327 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:02:53.332 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:03:25.282 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:03:25.287 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:03:45.458 INFO:teuthology.orchestra.run.smithi111.stdout:generic/007 90s 2023-03-29T11:03:45.525 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/007 2023-03-29T11:03:45.525 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:03:45.530 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:03:45.550 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:03:45.551 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/008) 2023-03-29T11:03:46.081 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:03:46.097 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:03:46.097 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:03:47.651 INFO:teuthology.orchestra.run.smithi111.stdout:generic/008 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-03-29T11:03:49.827 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/008 2023-03-29T11:03:49.828 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/008 2023-03-29T11:03:49.828 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:03:49.834 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:03:49.856 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:03:49.856 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/009) 2023-03-29T11:03:50.252 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:03:50.264 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:03:50.264 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:03:51.790 INFO:teuthology.orchestra.run.smithi111.stdout:generic/009 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-03-29T11:03:51.798 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/009 2023-03-29T11:03:51.798 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/009 2023-03-29T11:03:51.798 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:03:51.803 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:03:51.822 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:03:51.823 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/010) 2023-03-29T11:03:52.202 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:03:52.213 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:03:52.214 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:03:52.491 INFO:teuthology.orchestra.run.smithi111.stdout:generic/010 [not run] /tmp/tmp.CzuMqE4LuJxfstests-dev/src/dbtest not built 2023-03-29T11:03:52.499 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/010 2023-03-29T11:03:52.499 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/010 2023-03-29T11:03:52.499 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:03:52.504 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:03:52.523 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:03:52.524 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/011) 2023-03-29T11:03:52.906 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:03:52.917 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:03:52.917 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:03:55.471 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:03:55.475 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:04:25.856 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:04:25.863 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:04:39.202 INFO:teuthology.orchestra.run.smithi111.stdout:generic/011 47s 2023-03-29T11:04:39.214 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/011 2023-03-29T11:04:39.215 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:04:39.222 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:04:39.263 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:04:39.263 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/012) 2023-03-29T11:04:39.689 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:04:39.700 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:04:39.701 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:04:41.380 INFO:teuthology.orchestra.run.smithi111.stdout:generic/012 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-03-29T11:04:41.390 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/012 2023-03-29T11:04:41.390 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/012 2023-03-29T11:04:41.390 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:04:41.394 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:04:41.420 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:04:41.420 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/013) 2023-03-29T11:04:41.835 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:04:41.846 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:04:41.846 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:05:01.763 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:05:01.770 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:05:32.319 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:05:32.324 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:06:03.323 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:06:03.330 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:06:35.582 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:06:35.588 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:06:41.381 INFO:teuthology.orchestra.run.smithi111.stdout:generic/013 120s 2023-03-29T11:06:41.400 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/013 2023-03-29T11:06:41.400 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:06:41.408 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:06:41.436 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:06:41.436 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/014) 2023-03-29T11:06:42.036 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:06:42.053 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:06:42.054 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:07:10.529 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:07:10.534 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:07:43.029 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:07:43.038 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:07:43.847 INFO:tasks.ceph.mds.a.smithi111.stderr:2023-03-29T11:07:43.848+0000 7f3a763f9700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 175741) UID: 0 2023-03-29T11:07:43.848 INFO:tasks.ceph.mds.c.smithi111.stderr:2023-03-29T11:07:43.849+0000 7f155ddf9700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 175741) UID: 0 2023-03-29T11:11:03.715 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:11:03.736 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:11:34.113 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:11:34.139 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:11:36.425 INFO:teuthology.orchestra.run.smithi111.stdout:generic/014 294s 2023-03-29T11:11:36.453 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/014 2023-03-29T11:11:36.453 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:11:36.461 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:11:36.507 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:11:36.507 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/015) 2023-03-29T11:11:37.235 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:11:37.251 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:11:37.251 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:11:37.685 INFO:teuthology.orchestra.run.smithi111.stdout:generic/015 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:11:37.696 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/015 2023-03-29T11:11:37.696 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/015 2023-03-29T11:11:37.696 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:11:37.708 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:11:37.743 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:11:37.743 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/016) 2023-03-29T11:11:38.345 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:11:38.363 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:11:38.363 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:11:41.151 INFO:teuthology.orchestra.run.smithi111.stdout:generic/016 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-03-29T11:11:41.160 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/016 2023-03-29T11:11:41.160 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/016 2023-03-29T11:11:41.161 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:11:41.170 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:11:41.228 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:11:41.229 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/017) 2023-03-29T11:11:42.024 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:11:42.039 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:11:42.039 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:11:42.448 INFO:teuthology.orchestra.run.smithi111.stdout:generic/017 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:11:42.460 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/017 2023-03-29T11:11:42.460 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/017 2023-03-29T11:11:42.460 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:11:42.468 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:11:42.495 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:11:42.496 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/018) 2023-03-29T11:11:43.105 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:11:43.123 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:11:43.123 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:11:43.566 INFO:teuthology.orchestra.run.smithi111.stdout:generic/018 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:11:43.578 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/018 2023-03-29T11:11:43.578 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/018 2023-03-29T11:11:43.579 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:11:43.586 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:11:43.612 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:11:43.612 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/019) 2023-03-29T11:11:44.214 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:11:44.236 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:11:44.236 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:11:44.551 INFO:teuthology.orchestra.run.smithi111.stdout:generic/019 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:11:44.558 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/019 2023-03-29T11:11:44.559 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/019 2023-03-29T11:11:44.559 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:11:44.564 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:11:44.582 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:11:44.582 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/020) 2023-03-29T11:11:45.192 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:11:45.206 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:11:45.207 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:12:04.290 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:12:04.304 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:12:10.152 INFO:teuthology.orchestra.run.smithi111.stdout:generic/020 - output mismatch (see /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/020.out.bad) 2023-03-29T11:12:10.161 INFO:teuthology.orchestra.run.smithi111.stdout: --- tests/generic/020.out 2023-03-29 10:59:21.737747327 +0000 2023-03-29T11:12:10.161 INFO:teuthology.orchestra.run.smithi111.stdout: +++ /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/020.out.bad 2023-03-29 11:12:10.149137743 +0000 2023-03-29T11:12:10.161 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -39,27 +39,6 @@ 2023-03-29T11:12:10.161 INFO:teuthology.orchestra.run.smithi111.stdout: user.snrub="fish2\012" 2023-03-29T11:12:10.161 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:12:10.162 INFO:teuthology.orchestra.run.smithi111.stdout: *** add lots of attributes 2023-03-29T11:12:10.162 INFO:teuthology.orchestra.run.smithi111.stdout: -*** check 2023-03-29T11:12:10.162 INFO:teuthology.orchestra.run.smithi111.stdout: - *** MAX_ATTRS attribute(s) 2023-03-29T11:12:10.162 INFO:teuthology.orchestra.run.smithi111.stdout: -*** remove lots of attributes 2023-03-29T11:12:10.162 INFO:teuthology.orchestra.run.smithi111.stdout: - *** print attributes 2023-03-29T11:12:10.162 INFO:teuthology.orchestra.run.smithi111.stdout: -# file: 2023-03-29T11:12:10.162 INFO:teuthology.orchestra.run.smithi111.stdout: -user.snrub="fish2\012" 2023-03-29T11:12:10.162 INFO:teuthology.orchestra.run.smithi111.stdout: - 2023-03-29T11:12:10.162 INFO:teuthology.orchestra.run.smithi111.stdout: -*** really long value 2023-03-29T11:12:10.162 INFO:teuthology.orchestra.run.smithi111.stdout: -0000000 00 2023-03-29T11:12:10.162 INFO:teuthology.orchestra.run.smithi111.stdout: -* 2023-03-29T11:12:10.162 INFO:teuthology.orchestra.run.smithi111.stdout: -ATTRSIZE 2023-03-29T11:12:10.162 INFO:teuthology.orchestra.run.smithi111.stdout: -*** set/get/remove really long names (expect failure) 2023-03-29T11:12:10.162 INFO:teuthology.orchestra.run.smithi111.stdout: -attr_set: Invalid argument 2023-03-29T11:12:10.162 INFO:teuthology.orchestra.run.smithi111.stdout: -Could not set "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" for 2023-03-29T11:12:10.162 INFO:teuthology.orchestra.run.smithi111.stdout: -attr_get: Invalid argument 2023-03-29T11:12:10.162 INFO:teuthology.orchestra.run.smithi111.stdout: -Could not get "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" for 2023-03-29T11:12:10.163 INFO:teuthology.orchestra.run.smithi111.stdout: -attr_remove: Invalid argument 2023-03-29T11:12:10.163 INFO:teuthology.orchestra.run.smithi111.stdout: -Could not remove "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" for 2023-03-29T11:12:10.163 INFO:teuthology.orchestra.run.smithi111.stdout: -*** check final 2023-03-29T11:12:10.163 INFO:teuthology.orchestra.run.smithi111.stdout: - *** print attributes 2023-03-29T11:12:10.163 INFO:teuthology.orchestra.run.smithi111.stdout: -# file: 2023-03-29T11:12:10.163 INFO:teuthology.orchestra.run.smithi111.stdout: -user.snrub="fish2\012" 2023-03-29T11:12:10.163 INFO:teuthology.orchestra.run.smithi111.stdout: - 2023-03-29T11:12:10.163 INFO:teuthology.orchestra.run.smithi111.stdout: -*** delete 2023-03-29T11:12:10.163 INFO:teuthology.orchestra.run.smithi111.stdout: +attr_set: No space left on device 2023-03-29T11:12:10.163 INFO:teuthology.orchestra.run.smithi111.stdout: +Could not set "attribute_2335" for /tmp/tmp.6dtVtxiLHFtest/attribute_178441 2023-03-29T11:12:10.163 INFO:teuthology.orchestra.run.smithi111.stdout: +!!! failed to add "attribute_2335" 2023-03-29T11:12:10.172 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/020 2023-03-29T11:12:10.172 INFO:teuthology.orchestra.run.smithi111.stdout:Failures: generic/020 2023-03-29T11:12:10.172 INFO:teuthology.orchestra.run.smithi111.stdout:Failed 1 of 1 tests 2023-03-29T11:12:10.179 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:12:10.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T11:12:10.212 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-03-29T11:12:10.212 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-03-29T11:12:10.212 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/021) 2023-03-29T11:12:11.063 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:12:11.081 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:12:11.081 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:12:14.033 INFO:teuthology.orchestra.run.smithi111.stdout:generic/021 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-03-29T11:12:14.046 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/021 2023-03-29T11:12:14.046 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/021 2023-03-29T11:12:14.047 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:12:14.060 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:12:14.089 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:12:14.090 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/022) 2023-03-29T11:12:14.911 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:12:14.928 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:12:14.928 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:12:17.991 INFO:teuthology.orchestra.run.smithi111.stdout:generic/022 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-03-29T11:12:18.002 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/022 2023-03-29T11:12:18.002 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/022 2023-03-29T11:12:18.002 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:12:18.013 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:12:18.044 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:12:18.044 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/023) 2023-03-29T11:12:18.911 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:12:18.934 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:12:18.934 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:12:36.057 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:12:36.066 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:13:06.206 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:13:06.211 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:13:34.725 INFO:teuthology.orchestra.run.smithi111.stdout:generic/023 76s 2023-03-29T11:13:34.736 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/023 2023-03-29T11:13:34.736 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:13:34.742 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:34.835 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:13:34.835 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/024) 2023-03-29T11:13:35.419 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:13:35.430 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:13:35.430 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:35.735 INFO:teuthology.orchestra.run.smithi111.stdout:generic/024 [not run] kernel doesn't support renameat2 syscall 2023-03-29T11:13:35.743 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/024 2023-03-29T11:13:35.743 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/024 2023-03-29T11:13:35.743 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:13:35.748 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:35.810 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:13:35.811 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/025) 2023-03-29T11:13:36.311 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:13:36.316 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:13:36.425 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:13:36.436 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:13:36.436 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:36.746 INFO:teuthology.orchestra.run.smithi111.stdout:generic/025 [not run] kernel doesn't support renameat2 syscall 2023-03-29T11:13:36.754 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/025 2023-03-29T11:13:36.755 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/025 2023-03-29T11:13:36.755 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:13:36.760 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:36.853 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:13:36.854 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/026) 2023-03-29T11:13:37.464 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:13:37.475 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:13:37.475 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:37.820 INFO:teuthology.orchestra.run.smithi111.stdout:generic/026 [not run] ACLs not supported by this filesystem type: ceph-fuse 2023-03-29T11:13:37.828 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/026 2023-03-29T11:13:37.828 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/026 2023-03-29T11:13:37.829 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:13:37.834 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:37.852 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:13:37.852 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/027) 2023-03-29T11:13:38.238 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:13:38.250 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:13:38.250 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:38.517 INFO:teuthology.orchestra.run.smithi111.stdout:generic/027 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:13:38.525 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/027 2023-03-29T11:13:38.525 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/027 2023-03-29T11:13:38.525 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:13:38.531 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:38.551 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:13:38.551 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/028) 2023-03-29T11:13:38.937 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:13:38.948 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:13:38.949 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:44.261 INFO:teuthology.orchestra.run.smithi111.stdout:generic/028 6s 2023-03-29T11:13:44.272 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/028 2023-03-29T11:13:44.272 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:13:44.278 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:44.297 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:13:44.298 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/029) 2023-03-29T11:13:44.673 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:13:44.684 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:13:44.684 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:44.951 INFO:teuthology.orchestra.run.smithi111.stdout:generic/029 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:13:44.959 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/029 2023-03-29T11:13:44.959 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/029 2023-03-29T11:13:44.959 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:13:44.965 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:44.985 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:13:44.985 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/030) 2023-03-29T11:13:45.373 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:13:45.384 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:13:45.385 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:45.658 INFO:teuthology.orchestra.run.smithi111.stdout:generic/030 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:13:45.666 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/030 2023-03-29T11:13:45.666 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/030 2023-03-29T11:13:45.667 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:13:45.672 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:45.691 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:13:45.691 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/031) 2023-03-29T11:13:46.073 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:13:46.084 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:13:46.084 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:46.356 INFO:teuthology.orchestra.run.smithi111.stdout:generic/031 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:13:46.364 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/031 2023-03-29T11:13:46.364 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/031 2023-03-29T11:13:46.364 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:13:46.369 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:46.389 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:13:46.390 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/032) 2023-03-29T11:13:46.765 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:13:46.776 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:13:46.777 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:47.055 INFO:teuthology.orchestra.run.smithi111.stdout:generic/032 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:13:47.063 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/032 2023-03-29T11:13:47.063 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/032 2023-03-29T11:13:47.063 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:13:47.069 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:47.088 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:13:47.088 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/033) 2023-03-29T11:13:47.465 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:13:47.476 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:13:47.476 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:47.744 INFO:teuthology.orchestra.run.smithi111.stdout:generic/033 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:13:47.752 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/033 2023-03-29T11:13:47.752 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/033 2023-03-29T11:13:47.752 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:13:47.758 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:47.778 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:13:47.778 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/034) 2023-03-29T11:13:48.159 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:13:48.170 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:13:48.170 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:48.493 INFO:teuthology.orchestra.run.smithi111.stdout:generic/034 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:13:48.501 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/034 2023-03-29T11:13:48.501 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/034 2023-03-29T11:13:48.501 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:13:48.507 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:48.526 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:13:48.526 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/035) 2023-03-29T11:13:48.902 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:13:48.913 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:13:48.913 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:49.287 INFO:teuthology.orchestra.run.smithi111.stdout:generic/035 1s 2023-03-29T11:13:49.299 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/035 2023-03-29T11:13:49.299 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:13:49.304 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:13:49.382 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:13:49.382 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/036) 2023-03-29T11:13:49.977 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:13:49.989 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:13:49.989 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:00.402 INFO:teuthology.orchestra.run.smithi111.stdout:generic/036 10s 2023-03-29T11:14:00.414 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/036 2023-03-29T11:14:00.414 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:00.419 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:00.461 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:00.462 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/037) 2023-03-29T11:14:01.091 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:01.102 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:01.102 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:01.375 INFO:teuthology.orchestra.run.smithi111.stdout:generic/037 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:01.383 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/037 2023-03-29T11:14:01.384 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/037 2023-03-29T11:14:01.384 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:01.389 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:01.409 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:01.409 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/038) 2023-03-29T11:14:01.786 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:01.797 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:01.797 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:02.089 INFO:teuthology.orchestra.run.smithi111.stdout:generic/038 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:02.097 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/038 2023-03-29T11:14:02.097 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/038 2023-03-29T11:14:02.097 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:02.102 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:02.122 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:02.122 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/039) 2023-03-29T11:14:02.500 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:02.511 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:02.511 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:02.808 INFO:teuthology.orchestra.run.smithi111.stdout:generic/039 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:02.817 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/039 2023-03-29T11:14:02.817 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/039 2023-03-29T11:14:02.817 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:02.822 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:02.842 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:02.842 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/040) 2023-03-29T11:14:03.224 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:03.235 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:03.235 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:03.528 INFO:teuthology.orchestra.run.smithi111.stdout:generic/040 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:03.536 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/040 2023-03-29T11:14:03.537 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/040 2023-03-29T11:14:03.537 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:03.542 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:03.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:03.561 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/041) 2023-03-29T11:14:03.942 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:03.953 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:03.953 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:04.243 INFO:teuthology.orchestra.run.smithi111.stdout:generic/041 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:04.251 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/041 2023-03-29T11:14:04.252 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/041 2023-03-29T11:14:04.252 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:04.257 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:04.276 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:04.276 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/042) 2023-03-29T11:14:04.660 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:04.672 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:04.672 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:04.943 INFO:teuthology.orchestra.run.smithi111.stdout:generic/042 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:04.951 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/042 2023-03-29T11:14:04.951 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/042 2023-03-29T11:14:04.951 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:04.956 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:04.976 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:04.976 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/043) 2023-03-29T11:14:05.352 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:05.363 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:05.363 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:05.632 INFO:teuthology.orchestra.run.smithi111.stdout:generic/043 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:05.640 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/043 2023-03-29T11:14:05.640 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/043 2023-03-29T11:14:05.640 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:05.645 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:05.665 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:05.665 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/044) 2023-03-29T11:14:06.041 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:06.052 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:06.052 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:06.331 INFO:teuthology.orchestra.run.smithi111.stdout:generic/044 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:06.339 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/044 2023-03-29T11:14:06.339 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/044 2023-03-29T11:14:06.339 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:06.344 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:06.364 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:06.364 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/045) 2023-03-29T11:14:06.486 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:14:06.489 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:14:06.750 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:06.761 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:06.762 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:07.031 INFO:teuthology.orchestra.run.smithi111.stdout:generic/045 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:07.039 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/045 2023-03-29T11:14:07.039 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/045 2023-03-29T11:14:07.039 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:07.044 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:07.065 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:07.065 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/046) 2023-03-29T11:14:07.442 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:07.453 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:07.453 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:07.721 INFO:teuthology.orchestra.run.smithi111.stdout:generic/046 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:07.728 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/046 2023-03-29T11:14:07.729 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/046 2023-03-29T11:14:07.729 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:07.734 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:07.753 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:07.753 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/047) 2023-03-29T11:14:08.128 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:08.139 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:08.140 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:08.417 INFO:teuthology.orchestra.run.smithi111.stdout:generic/047 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:08.425 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/047 2023-03-29T11:14:08.425 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/047 2023-03-29T11:14:08.425 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:08.431 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:08.452 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:08.452 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/048) 2023-03-29T11:14:08.832 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:08.843 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:08.843 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:09.114 INFO:teuthology.orchestra.run.smithi111.stdout:generic/048 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:09.122 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/048 2023-03-29T11:14:09.122 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/048 2023-03-29T11:14:09.122 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:09.128 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:09.147 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:09.147 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/049) 2023-03-29T11:14:09.521 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:09.532 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:09.532 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:09.803 INFO:teuthology.orchestra.run.smithi111.stdout:generic/049 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:09.811 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/049 2023-03-29T11:14:09.811 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/049 2023-03-29T11:14:09.811 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:09.816 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:09.835 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:09.836 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/050) 2023-03-29T11:14:10.212 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:10.223 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:10.223 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:11.471 INFO:teuthology.orchestra.run.smithi111.stdout:generic/050 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:11.480 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/050 2023-03-29T11:14:11.481 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/050 2023-03-29T11:14:11.481 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:11.487 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:11.509 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:11.509 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/051) 2023-03-29T11:14:11.897 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:11.908 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:11.908 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:12.179 INFO:teuthology.orchestra.run.smithi111.stdout:generic/051 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:12.187 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/051 2023-03-29T11:14:12.187 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/051 2023-03-29T11:14:12.187 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:12.193 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:12.212 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:12.212 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/052) 2023-03-29T11:14:12.590 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:12.601 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:12.601 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:12.880 INFO:teuthology.orchestra.run.smithi111.stdout:generic/052 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:12.888 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/052 2023-03-29T11:14:12.888 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/052 2023-03-29T11:14:12.888 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:12.893 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:12.913 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:12.913 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/053) 2023-03-29T11:14:13.292 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:13.303 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:13.303 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:13.578 INFO:teuthology.orchestra.run.smithi111.stdout:generic/053 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:13.586 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/053 2023-03-29T11:14:13.587 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/053 2023-03-29T11:14:13.587 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:13.592 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:13.611 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:13.612 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/054) 2023-03-29T11:14:13.991 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:14.002 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:14.002 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:14.278 INFO:teuthology.orchestra.run.smithi111.stdout:generic/054 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:14.286 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/054 2023-03-29T11:14:14.286 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/054 2023-03-29T11:14:14.286 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:14.292 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:14.311 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:14.311 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/055) 2023-03-29T11:14:14.682 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:14.694 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:14.694 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:14.979 INFO:teuthology.orchestra.run.smithi111.stdout:generic/055 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:14.987 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/055 2023-03-29T11:14:14.987 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/055 2023-03-29T11:14:14.987 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:14.992 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:15.013 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:15.013 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/056) 2023-03-29T11:14:15.398 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:15.409 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:15.409 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:15.706 INFO:teuthology.orchestra.run.smithi111.stdout:generic/056 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:15.714 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/056 2023-03-29T11:14:15.714 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/056 2023-03-29T11:14:15.714 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:15.719 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:15.739 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:15.739 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/057) 2023-03-29T11:14:16.118 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:16.129 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:16.129 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:16.426 INFO:teuthology.orchestra.run.smithi111.stdout:generic/057 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:16.434 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/057 2023-03-29T11:14:16.434 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/057 2023-03-29T11:14:16.434 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:16.439 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:16.459 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:16.460 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/058) 2023-03-29T11:14:16.840 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:16.851 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:16.851 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:18.476 INFO:teuthology.orchestra.run.smithi111.stdout:generic/058 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-03-29T11:14:18.484 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/058 2023-03-29T11:14:18.485 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/058 2023-03-29T11:14:18.485 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:18.490 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:18.550 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:18.550 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/059) 2023-03-29T11:14:19.160 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:19.171 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:19.172 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:19.481 INFO:teuthology.orchestra.run.smithi111.stdout:generic/059 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:19.489 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/059 2023-03-29T11:14:19.489 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/059 2023-03-29T11:14:19.489 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:19.494 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:19.517 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:19.517 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/060) 2023-03-29T11:14:19.900 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:19.911 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:19.911 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:21.509 INFO:teuthology.orchestra.run.smithi111.stdout:generic/060 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-03-29T11:14:21.517 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/060 2023-03-29T11:14:21.517 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/060 2023-03-29T11:14:21.517 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:21.523 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:21.565 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:21.565 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/061) 2023-03-29T11:14:22.193 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:22.204 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:22.204 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:23.807 INFO:teuthology.orchestra.run.smithi111.stdout:generic/061 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-03-29T11:14:23.815 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/061 2023-03-29T11:14:23.815 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/061 2023-03-29T11:14:23.815 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:23.820 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:23.877 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:23.878 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/062) 2023-03-29T11:14:24.489 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:24.507 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:24.507 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:24.786 INFO:teuthology.orchestra.run.smithi111.stdout:generic/062 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:24.795 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/062 2023-03-29T11:14:24.795 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/062 2023-03-29T11:14:24.795 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:24.800 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:24.819 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:24.820 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/063) 2023-03-29T11:14:25.193 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:25.204 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:25.204 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:26.766 INFO:teuthology.orchestra.run.smithi111.stdout:generic/063 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-03-29T11:14:26.775 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/063 2023-03-29T11:14:26.775 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/063 2023-03-29T11:14:26.775 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:26.779 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:26.807 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:26.807 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/064) 2023-03-29T11:14:27.466 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:27.477 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:27.477 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:27.749 INFO:teuthology.orchestra.run.smithi111.stdout:generic/064 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:27.756 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/064 2023-03-29T11:14:27.757 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/064 2023-03-29T11:14:27.757 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:27.762 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:27.782 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:27.782 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/065) 2023-03-29T11:14:28.162 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:28.173 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:28.173 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:28.471 INFO:teuthology.orchestra.run.smithi111.stdout:generic/065 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:28.479 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/065 2023-03-29T11:14:28.479 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/065 2023-03-29T11:14:28.479 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:28.485 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:28.504 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:28.504 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/066) 2023-03-29T11:14:28.884 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:28.895 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:28.895 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:29.197 INFO:teuthology.orchestra.run.smithi111.stdout:generic/066 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:29.205 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/066 2023-03-29T11:14:29.205 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/066 2023-03-29T11:14:29.205 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:29.210 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:29.230 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:29.231 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/067) 2023-03-29T11:14:29.602 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:29.614 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:29.614 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:29.922 INFO:teuthology.orchestra.run.smithi111.stdout:generic/067 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:29.930 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/067 2023-03-29T11:14:29.931 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/067 2023-03-29T11:14:29.931 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:29.936 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:29.956 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:29.956 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/068) 2023-03-29T11:14:30.337 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:30.349 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:30.349 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:30.635 INFO:teuthology.orchestra.run.smithi111.stdout:generic/068 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:30.643 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/068 2023-03-29T11:14:30.643 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/068 2023-03-29T11:14:30.644 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:30.649 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:30.668 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:30.669 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/069) 2023-03-29T11:14:31.048 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:31.059 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:31.059 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:31.332 INFO:teuthology.orchestra.run.smithi111.stdout:generic/069 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:14:31.340 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/069 2023-03-29T11:14:31.340 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/069 2023-03-29T11:14:31.340 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:14:31.345 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:31.366 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:14:31.366 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/070) 2023-03-29T11:14:31.745 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:14:31.756 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:14:31.756 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:14:36.666 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:14:36.671 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:15:06.823 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:15:06.827 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:15:36.984 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:15:36.990 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:15:45.868 INFO:teuthology.orchestra.run.smithi111.stdout:generic/070 74s 2023-03-29T11:15:45.880 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/070 2023-03-29T11:15:45.880 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:15:45.885 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:15:45.929 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:15:45.929 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/071) 2023-03-29T11:15:46.564 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:15:46.576 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:15:46.576 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:15:46.971 INFO:teuthology.orchestra.run.smithi111.stdout:generic/071 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:15:46.987 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/071 2023-03-29T11:15:46.987 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/071 2023-03-29T11:15:46.987 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:15:46.995 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:15:47.032 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:15:47.032 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/072) 2023-03-29T11:15:47.633 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:15:47.653 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:15:47.653 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:15:50.244 INFO:teuthology.orchestra.run.smithi111.stdout:generic/072 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2023-03-29T11:15:50.256 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/072 2023-03-29T11:15:50.256 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/072 2023-03-29T11:15:50.257 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:15:50.265 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:15:50.295 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:15:50.296 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/073) 2023-03-29T11:15:50.906 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:15:50.924 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:15:50.924 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:15:51.386 INFO:teuthology.orchestra.run.smithi111.stdout:generic/073 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:15:51.399 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/073 2023-03-29T11:15:51.399 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/073 2023-03-29T11:15:51.399 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:15:51.407 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:15:51.431 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:15:51.432 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/074) 2023-03-29T11:15:52.026 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:15:52.040 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:15:52.040 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:16:07.151 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:16:07.157 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:16:38.835 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:16:38.841 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:17:09.060 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:17:09.068 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:17:39.263 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:17:39.273 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:18:10.590 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:18:10.596 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:18:40.863 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:18:40.878 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:19:11.060 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:19:11.064 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:19:42.166 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:19:42.171 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:20:16.747 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:20:16.758 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:20:50.341 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:20:50.347 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:21:20.550 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:21:20.559 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:21:50.790 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:21:50.801 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:22:21.115 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:22:21.129 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:22:51.448 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:22:51.467 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:22:51.619 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-03-29T11:22:51.619+0000 7fd2277f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120050) UID: 0 2023-03-29T11:22:51.619 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-03-29T11:22:51.619+0000 7fabb15f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120050) UID: 0 2023-03-29T11:22:51.620 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-03-29T11:22:51.619+0000 7f0c30df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120050) UID: 0 2023-03-29T11:22:51.620 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-03-29T11:22:51.619+0000 7f9655df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120050) UID: 0 2023-03-29T11:26:29.466 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:26:29.472 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:26:30.611 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-03-29T11:26:30.609+0000 7f046aff9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 209558) UID: 0 2023-03-29T11:26:30.611 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-03-29T11:26:30.609+0000 7f22b21f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 209558) UID: 0 2023-03-29T11:26:30.613 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-03-29T11:26:30.614+0000 7f93ce7f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 209558) UID: 0 2023-03-29T11:26:30.642 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T11:26:30.611+0000 7f7a699f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 209558) UID: 0 2023-03-29T11:29:46.473 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T11:29:46.473+0000 7f7a4c1d2700 -1 bdev(0x55ed48a1dc00 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 4 2023-03-29T11:29:46.475 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T11:29:46.475+0000 7f7a4d1d4700 -1 bdev(0x55ed48a1dc00 /var/lib/ceph/osd/ceph-0/block) aio_submit retries 2 2023-03-29T11:35:37.299 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:35:37.315 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:35:37.482 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-03-29T11:35:37.483+0000 7f046aff9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 210322) UID: 0 2023-03-29T11:35:37.482 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T11:35:37.483+0000 7f7a699f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 210322) UID: 0 2023-03-29T11:35:37.483 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-03-29T11:35:37.483+0000 7f22b21f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 210322) UID: 0 2023-03-29T11:35:37.483 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-03-29T11:35:37.483+0000 7f93ce7f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 210322) UID: 0 2023-03-29T11:35:37.494 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-03-29T11:35:37.494+0000 7fabb15f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120244) UID: 0 2023-03-29T11:35:37.494 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-03-29T11:35:37.494+0000 7f9655df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120244) UID: 0 2023-03-29T11:35:37.494 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-03-29T11:35:37.494+0000 7fd2277f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120244) UID: 0 2023-03-29T11:35:37.495 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-03-29T11:35:37.494+0000 7f0c30df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120244) UID: 0 2023-03-29T11:42:01.629 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-03-29T11:42:01.623+0000 7fd2277f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120494) UID: 0 2023-03-29T11:42:01.630 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-03-29T11:42:01.623+0000 7f0c30df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120494) UID: 0 2023-03-29T11:42:01.630 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-03-29T11:42:01.623+0000 7fabb15f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120494) UID: 0 2023-03-29T11:42:01.630 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-03-29T11:42:01.623+0000 7f9655df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 120494) UID: 0 2023-03-29T11:44:54.518 INFO:teuthology.orchestra.run.smithi111.stdout:generic/074 1742s 2023-03-29T11:44:54.547 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/074 2023-03-29T11:44:54.548 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:44:54.553 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:44:54.609 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:44:54.609 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/075) 2023-03-29T11:44:55.410 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:44:55.450 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:44:55.450 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:04.350 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:45:04.355 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:45:34.684 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:45:34.688 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:45:34.789 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-03-29T11:45:34.790+0000 7f046aff9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 212502) UID: 0 2023-03-29T11:45:34.789 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T11:45:34.790+0000 7f7a699f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 212502) UID: 0 2023-03-29T11:45:34.790 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-03-29T11:45:34.790+0000 7f22b21f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 212502) UID: 0 2023-03-29T11:45:34.790 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-03-29T11:45:34.790+0000 7f93ce7f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 212502) UID: 0 2023-03-29T11:45:42.055 INFO:teuthology.orchestra.run.smithi111.stdout:generic/075 47s 2023-03-29T11:45:42.067 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/075 2023-03-29T11:45:42.067 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:45:42.072 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:42.155 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:45:42.155 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/076) 2023-03-29T11:45:42.816 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:45:42.827 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:45:42.828 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:43.105 INFO:teuthology.orchestra.run.smithi111.stdout:generic/076 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:45:43.113 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/076 2023-03-29T11:45:43.113 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/076 2023-03-29T11:45:43.113 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:45:43.118 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:43.138 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:45:43.138 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/077) 2023-03-29T11:45:43.522 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:45:43.532 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:45:43.533 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:47.237 INFO:teuthology.orchestra.run.smithi111.stdout:generic/077 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:45:47.245 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/077 2023-03-29T11:45:47.245 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/077 2023-03-29T11:45:47.246 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:45:47.250 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:47.270 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:45:47.271 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/078) 2023-03-29T11:45:47.655 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:45:47.666 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:45:47.667 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:48.056 INFO:teuthology.orchestra.run.smithi111.stdout:generic/078 [not run] kernel doesn't support renameat2 syscall 2023-03-29T11:45:48.065 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/078 2023-03-29T11:45:48.065 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/078 2023-03-29T11:45:48.065 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:45:48.070 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:48.146 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:45:48.147 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/079) 2023-03-29T11:45:48.842 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:45:48.854 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:45:48.854 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:50.367 INFO:teuthology.orchestra.run.smithi111.stdout:generic/079 [not run] file system doesn't support chattr +ia 2023-03-29T11:45:50.375 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/079 2023-03-29T11:45:50.376 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/079 2023-03-29T11:45:50.376 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:45:50.381 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:50.403 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:45:50.403 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/080) 2023-03-29T11:45:50.792 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:45:50.803 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:45:50.803 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:53.172 INFO:teuthology.orchestra.run.smithi111.stdout:generic/080 3s 2023-03-29T11:45:53.184 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/080 2023-03-29T11:45:53.184 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:45:53.189 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:53.209 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:45:53.209 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/081) 2023-03-29T11:45:53.589 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:45:53.600 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:45:53.600 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:53.870 INFO:teuthology.orchestra.run.smithi111.stdout:generic/081 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:45:53.878 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/081 2023-03-29T11:45:53.878 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/081 2023-03-29T11:45:53.878 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:45:53.883 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:53.903 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:45:53.903 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/082) 2023-03-29T11:45:54.294 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:45:54.306 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:45:54.306 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:54.623 INFO:teuthology.orchestra.run.smithi111.stdout:generic/082 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:45:54.632 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/082 2023-03-29T11:45:54.632 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/082 2023-03-29T11:45:54.632 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:45:54.637 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:54.657 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:45:54.657 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/083) 2023-03-29T11:45:55.051 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:45:55.062 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:45:55.063 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:55.338 INFO:teuthology.orchestra.run.smithi111.stdout:generic/083 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:45:55.346 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/083 2023-03-29T11:45:55.346 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/083 2023-03-29T11:45:55.346 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:45:55.351 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:55.372 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:45:55.372 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/084) 2023-03-29T11:45:55.758 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:45:55.769 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:45:55.769 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:56.040 INFO:teuthology.orchestra.run.smithi111.stdout:generic/084 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:45:56.048 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/084 2023-03-29T11:45:56.048 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/084 2023-03-29T11:45:56.048 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:45:56.061 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:56.074 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:45:56.074 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/085) 2023-03-29T11:45:56.454 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:45:56.465 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:45:56.465 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:56.885 INFO:teuthology.orchestra.run.smithi111.stdout:generic/085 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T11:45:56.893 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/085 2023-03-29T11:45:56.893 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/085 2023-03-29T11:45:56.894 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:45:56.899 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:45:56.919 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:45:56.919 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/086) 2023-03-29T11:45:57.299 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:45:57.310 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:45:57.310 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:46:17.517 INFO:teuthology.orchestra.run.smithi111.stdout:generic/086 20s 2023-03-29T11:46:17.789 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/086 2023-03-29T11:46:17.789 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:46:17.794 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:46:18.214 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:46:18.214 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/087) 2023-03-29T11:46:21.708 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:46:21.719 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:46:21.719 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:46:22.780 INFO:teuthology.orchestra.run.smithi111.stdout:generic/087 1s 2023-03-29T11:46:22.791 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/087 2023-03-29T11:46:22.791 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:46:22.796 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:46:22.816 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:46:22.816 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/088) 2023-03-29T11:46:23.577 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:46:23.588 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:46:23.588 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:46:24.051 INFO:teuthology.orchestra.run.smithi111.stdout:generic/088 1s 2023-03-29T11:46:24.063 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/088 2023-03-29T11:46:24.063 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T11:46:24.068 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:46:24.094 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T11:46:24.095 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/089) 2023-03-29T11:46:24.839 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T11:46:24.850 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T11:46:24.851 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T11:49:06.255 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:49:06.262 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:49:06.415 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-03-29T11:49:06.415+0000 7f046aff9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 219056) UID: 0 2023-03-29T11:49:06.416 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T11:49:06.415+0000 7f7a699f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 219056) UID: 0 2023-03-29T11:49:06.416 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-03-29T11:49:06.415+0000 7f22b21f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 219056) UID: 0 2023-03-29T11:49:06.416 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-03-29T11:49:06.415+0000 7f93ce7f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 219056) UID: 0 2023-03-29T11:53:05.189 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:53:05.194 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:53:05.521 INFO:tasks.ceph.mds.a.smithi111.stderr:2023-03-29T11:53:05.521+0000 7f3a763f9700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 219327) UID: 0 2023-03-29T11:53:05.521 INFO:tasks.ceph.mds.c.smithi111.stderr:2023-03-29T11:53:05.521+0000 7f155ddf9700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 219327) UID: 0 2023-03-29T11:59:35.269 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:59:35.274 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T11:59:35.584 INFO:tasks.ceph.mds.a.smithi111.stderr:2023-03-29T11:59:35.585+0000 7f3a763f9700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 220777) UID: 0 2023-03-29T11:59:35.585 INFO:tasks.ceph.mds.c.smithi111.stderr:2023-03-29T11:59:35.585+0000 7f155ddf9700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 220777) UID: 0 2023-03-29T12:03:50.881 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:03:50.887 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:04:21.010 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:04:21.016 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:04:21.464 INFO:tasks.ceph.mds.a.smithi111.stderr:2023-03-29T12:04:21.465+0000 7f3a763f9700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 221529) UID: 0 2023-03-29T12:04:21.464 INFO:tasks.ceph.mds.c.smithi111.stderr:2023-03-29T12:04:21.465+0000 7f155ddf9700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 221529) UID: 0 2023-03-29T12:06:30.191 INFO:teuthology.orchestra.run.smithi111.stdout:generic/089 1206s 2023-03-29T12:06:30.227 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/089 2023-03-29T12:06:30.227 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:06:30.233 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:06:30.259 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:06:30.259 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/090) 2023-03-29T12:06:31.225 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:06:31.236 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:06:31.236 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:06:31.642 INFO:teuthology.orchestra.run.smithi111.stdout:generic/090 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:06:31.651 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/090 2023-03-29T12:06:31.651 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/090 2023-03-29T12:06:31.651 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:06:31.658 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:06:31.679 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:06:31.679 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/091) 2023-03-29T12:06:32.065 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:06:32.076 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:06:32.076 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:07:13.465 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:07:13.470 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:07:43.576 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:07:43.581 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:08:14.352 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:08:14.357 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:08:44.453 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:08:44.458 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:08:51.139 INFO:teuthology.orchestra.run.smithi111.stdout:generic/091 139s 2023-03-29T12:08:51.151 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/091 2023-03-29T12:08:51.151 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:08:51.155 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:08:51.210 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:08:51.210 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/092) 2023-03-29T12:08:51.898 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:08:51.909 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:08:51.910 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:08:52.278 INFO:teuthology.orchestra.run.smithi111.stdout:generic/092 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-03-29T12:08:52.287 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/092 2023-03-29T12:08:52.287 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/092 2023-03-29T12:08:52.287 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:08:52.292 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:08:52.332 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:08:52.332 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/093) 2023-03-29T12:08:53.019 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:08:53.030 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:08:53.030 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:08:54.080 INFO:teuthology.orchestra.run.smithi111.stdout:generic/093 1s 2023-03-29T12:08:54.091 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/093 2023-03-29T12:08:54.091 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:08:54.096 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:08:54.147 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:08:54.148 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/094) 2023-03-29T12:08:54.830 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:08:54.841 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:08:54.841 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:08:55.109 INFO:teuthology.orchestra.run.smithi111.stdout:generic/094 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:08:55.117 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/094 2023-03-29T12:08:55.118 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/094 2023-03-29T12:08:55.118 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:08:55.123 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:08:55.143 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:08:55.143 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/095) 2023-03-29T12:08:55.541 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:08:55.553 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:08:55.553 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:08:55.920 INFO:teuthology.orchestra.run.smithi111.stdout:generic/095 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:08:55.928 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/095 2023-03-29T12:08:55.929 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/095 2023-03-29T12:08:55.929 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:08:55.934 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:08:55.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:08:55.955 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/096) 2023-03-29T12:08:56.339 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:08:56.350 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:08:56.350 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:08:56.623 INFO:teuthology.orchestra.run.smithi111.stdout:generic/096 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:08:56.632 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/096 2023-03-29T12:08:56.632 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/096 2023-03-29T12:08:56.632 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:08:56.637 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:08:56.658 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:08:56.658 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/097) 2023-03-29T12:08:57.038 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:08:57.050 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:08:57.050 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:08:57.964 INFO:teuthology.orchestra.run.smithi111.stdout:generic/097 0s 2023-03-29T12:08:57.975 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/097 2023-03-29T12:08:57.975 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:08:57.980 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:08:57.997 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:08:57.998 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/098) 2023-03-29T12:08:58.376 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:08:58.387 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:08:58.387 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:08:58.657 INFO:teuthology.orchestra.run.smithi111.stdout:generic/098 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:08:58.665 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/098 2023-03-29T12:08:58.665 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/098 2023-03-29T12:08:58.665 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:08:58.670 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:08:58.691 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:08:58.691 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2023-03-29T12:08:59.069 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:08:59.081 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:08:59.081 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:08:59.426 INFO:teuthology.orchestra.run.smithi111.stdout:generic/099 [not run] ACLs not supported by this filesystem type: ceph-fuse 2023-03-29T12:08:59.434 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/099 2023-03-29T12:08:59.434 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/099 2023-03-29T12:08:59.434 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:08:59.439 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:08:59.457 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:08:59.458 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/100) 2023-03-29T12:08:59.841 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:08:59.853 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:08:59.854 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:14.563 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:09:14.567 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:09:44.687 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:09:44.691 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:09:49.663 INFO:teuthology.orchestra.run.smithi111.stdout:generic/100 50s 2023-03-29T12:09:49.674 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/100 2023-03-29T12:09:49.675 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:09:49.680 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:49.734 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:09:49.735 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/101) 2023-03-29T12:09:50.475 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:09:50.487 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:09:50.487 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:50.782 INFO:teuthology.orchestra.run.smithi111.stdout:generic/101 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:09:50.790 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/101 2023-03-29T12:09:50.790 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/101 2023-03-29T12:09:50.790 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:09:50.796 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:50.815 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:09:50.815 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/102) 2023-03-29T12:09:51.198 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:09:51.209 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:09:51.209 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:51.482 INFO:teuthology.orchestra.run.smithi111.stdout:generic/102 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:09:51.490 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/102 2023-03-29T12:09:51.490 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/102 2023-03-29T12:09:51.490 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:09:51.496 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:51.517 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:09:51.517 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/103) 2023-03-29T12:09:51.899 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:09:51.910 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:09:51.910 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:52.181 INFO:teuthology.orchestra.run.smithi111.stdout:generic/103 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:09:52.189 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/103 2023-03-29T12:09:52.189 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/103 2023-03-29T12:09:52.189 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:09:52.194 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:52.214 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:09:52.214 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/104) 2023-03-29T12:09:52.602 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:09:52.613 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:09:52.613 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:52.917 INFO:teuthology.orchestra.run.smithi111.stdout:generic/104 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:09:52.926 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/104 2023-03-29T12:09:52.926 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/104 2023-03-29T12:09:52.926 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:09:52.931 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:52.950 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:09:52.950 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/105) 2023-03-29T12:09:53.326 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:09:53.337 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:09:53.337 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:53.604 INFO:teuthology.orchestra.run.smithi111.stdout:generic/105 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:09:53.613 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/105 2023-03-29T12:09:53.613 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/105 2023-03-29T12:09:53.613 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:09:53.618 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:53.637 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:09:53.637 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/106) 2023-03-29T12:09:54.013 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:09:54.024 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:09:54.025 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:54.322 INFO:teuthology.orchestra.run.smithi111.stdout:generic/106 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:09:54.331 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/106 2023-03-29T12:09:54.331 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/106 2023-03-29T12:09:54.331 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:09:54.336 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:54.355 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:09:54.356 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/107) 2023-03-29T12:09:54.745 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:09:54.756 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:09:54.756 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:55.070 INFO:teuthology.orchestra.run.smithi111.stdout:generic/107 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:09:55.079 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/107 2023-03-29T12:09:55.079 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/107 2023-03-29T12:09:55.079 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:09:55.084 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:55.103 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:09:55.103 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/108) 2023-03-29T12:09:55.489 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:09:55.500 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:09:55.500 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:55.940 INFO:teuthology.orchestra.run.smithi111.stdout:generic/108 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:09:55.948 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/108 2023-03-29T12:09:55.948 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/108 2023-03-29T12:09:55.948 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:09:55.954 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:55.972 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:09:55.973 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/109) 2023-03-29T12:09:56.362 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:09:56.373 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:09:56.373 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:56.647 INFO:teuthology.orchestra.run.smithi111.stdout:generic/109 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:09:56.655 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/109 2023-03-29T12:09:56.655 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/109 2023-03-29T12:09:56.655 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:09:56.660 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:56.679 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:09:56.680 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/110) 2023-03-29T12:09:57.066 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:09:57.077 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:09:57.077 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:57.436 INFO:teuthology.orchestra.run.smithi111.stdout:generic/110 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:09:57.444 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/110 2023-03-29T12:09:57.445 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/110 2023-03-29T12:09:57.445 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:09:57.450 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:57.492 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:09:57.493 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/111) 2023-03-29T12:09:58.177 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:09:58.188 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:09:58.188 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:58.536 INFO:teuthology.orchestra.run.smithi111.stdout:generic/111 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:09:58.545 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/111 2023-03-29T12:09:58.545 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/111 2023-03-29T12:09:58.545 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:09:58.550 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:09:58.613 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:09:58.613 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/112) 2023-03-29T12:09:59.277 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:09:59.288 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:09:59.288 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:10:14.827 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:10:14.831 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:10:44.957 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:10:44.961 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:10:50.188 INFO:teuthology.orchestra.run.smithi111.stdout:generic/112 51s 2023-03-29T12:10:50.199 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/112 2023-03-29T12:10:50.199 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:10:50.205 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:10:50.264 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:10:50.264 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/113) 2023-03-29T12:10:50.928 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:10:50.939 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:10:50.939 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:15.057 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:11:15.062 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:11:38.931 INFO:teuthology.orchestra.run.smithi111.stdout:generic/113 48s 2023-03-29T12:11:40.210 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/113 2023-03-29T12:11:40.210 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:11:40.216 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:40.305 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:11:40.306 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/114) 2023-03-29T12:11:41.020 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:11:41.031 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:11:41.031 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:41.415 INFO:teuthology.orchestra.run.smithi111.stdout:generic/114 [not run] device block size: 4096 greater than 512 2023-03-29T12:11:41.423 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/114 2023-03-29T12:11:41.424 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/114 2023-03-29T12:11:41.424 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:11:41.429 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:41.448 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:11:41.449 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/115) 2023-03-29T12:11:41.834 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:11:41.845 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:11:41.845 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:42.193 INFO:teuthology.orchestra.run.smithi111.stdout:generic/115 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:11:42.201 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/115 2023-03-29T12:11:42.202 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/115 2023-03-29T12:11:42.202 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:11:42.207 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:42.250 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:11:42.250 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/116) 2023-03-29T12:11:42.923 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:11:42.934 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:11:42.934 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:43.280 INFO:teuthology.orchestra.run.smithi111.stdout:generic/116 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:11:43.288 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/116 2023-03-29T12:11:43.289 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/116 2023-03-29T12:11:43.289 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:11:43.294 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:43.346 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:11:43.346 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/117) 2023-03-29T12:11:44.017 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:11:44.028 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:11:44.028 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:44.298 INFO:teuthology.orchestra.run.smithi111.stdout:generic/117 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:11:44.306 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/117 2023-03-29T12:11:44.306 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/117 2023-03-29T12:11:44.306 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:11:44.311 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:44.330 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:11:44.330 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/118) 2023-03-29T12:11:44.708 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:11:44.719 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:11:44.719 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:45.068 INFO:teuthology.orchestra.run.smithi111.stdout:generic/118 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:11:45.076 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/118 2023-03-29T12:11:45.076 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/118 2023-03-29T12:11:45.076 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:11:45.081 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:45.135 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:11:45.135 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/119) 2023-03-29T12:11:45.173 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:11:45.208 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:11:45.975 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:11:45.986 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:11:45.986 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:46.457 INFO:teuthology.orchestra.run.smithi111.stdout:generic/119 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:11:46.465 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/119 2023-03-29T12:11:46.465 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/119 2023-03-29T12:11:46.465 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:11:46.470 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:46.509 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:11:46.510 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/120) 2023-03-29T12:11:47.185 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:11:47.196 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:11:47.196 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:47.664 INFO:teuthology.orchestra.run.smithi111.stdout:generic/120 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:11:47.672 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/120 2023-03-29T12:11:47.672 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/120 2023-03-29T12:11:47.672 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:11:47.677 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:47.703 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:11:47.703 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/121) 2023-03-29T12:11:48.087 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:11:48.098 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:11:48.099 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:48.460 INFO:teuthology.orchestra.run.smithi111.stdout:generic/121 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2023-03-29T12:11:48.468 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/121 2023-03-29T12:11:48.469 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/121 2023-03-29T12:11:48.469 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:11:48.474 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:48.492 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:11:48.492 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/122) 2023-03-29T12:11:49.203 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:11:49.214 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:11:49.214 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:49.586 INFO:teuthology.orchestra.run.smithi111.stdout:generic/122 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2023-03-29T12:11:49.594 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/122 2023-03-29T12:11:49.594 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/122 2023-03-29T12:11:49.594 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:11:49.600 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:49.618 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:11:49.619 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/123) 2023-03-29T12:11:50.320 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:11:50.333 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:11:50.333 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:51.033 INFO:teuthology.orchestra.run.smithi111.stdout:generic/123 1s 2023-03-29T12:11:51.044 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/123 2023-03-29T12:11:51.044 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:11:51.049 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:51.077 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:11:51.077 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/124) 2023-03-29T12:11:51.760 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:11:51.771 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:11:51.771 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:52.046 INFO:teuthology.orchestra.run.smithi111.stdout:generic/124 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:11:52.054 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/124 2023-03-29T12:11:52.054 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/124 2023-03-29T12:11:52.054 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:11:52.059 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:11:52.079 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:11:52.080 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/125) 2023-03-29T12:11:52.462 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:11:52.473 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:11:52.473 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:12:16.810 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:12:16.815 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:12:47.004 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:12:47.009 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:12:53.136 INFO:teuthology.orchestra.run.smithi111.stdout:generic/125 61s 2023-03-29T12:12:53.148 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/125 2023-03-29T12:12:53.148 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:12:53.154 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:12:53.247 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:12:53.248 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/126) 2023-03-29T12:12:53.869 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:12:53.880 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:12:53.880 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:12:54.911 INFO:teuthology.orchestra.run.smithi111.stdout:generic/126 - output mismatch (see /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/126.out.bad) 2023-03-29T12:12:54.917 INFO:teuthology.orchestra.run.smithi111.stdout: --- tests/generic/126.out 2023-03-29 10:59:21.751747098 +0000 2023-03-29T12:12:54.917 INFO:teuthology.orchestra.run.smithi111.stdout: +++ /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/126.out.bad 2023-03-29 12:12:54.911351312 +0000 2023-03-29T12:12:54.917 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -1,7 +1,7 @@ 2023-03-29T12:12:54.918 INFO:teuthology.orchestra.run.smithi111.stdout: QA output created by 126 2023-03-29T12:12:54.918 INFO:teuthology.orchestra.run.smithi111.stdout: -x a 001 file owned by (99/99) as user/group(12/100) PASS 2023-03-29T12:12:54.918 INFO:teuthology.orchestra.run.smithi111.stdout: -x a 010 file owned by (99/99) as user/group(200/99) PASS 2023-03-29T12:12:54.918 INFO:teuthology.orchestra.run.smithi111.stdout: -x a 100 file owned by (99/99) as user/group(99/500) PASS 2023-03-29T12:12:54.918 INFO:teuthology.orchestra.run.smithi111.stdout: +x a 001 file owned by (99/99) as user/group(12/100) FAIL 2023-03-29T12:12:54.918 INFO:teuthology.orchestra.run.smithi111.stdout: +x a 010 file owned by (99/99) as user/group(200/99) FAIL 2023-03-29T12:12:54.918 INFO:teuthology.orchestra.run.smithi111.stdout: +x a 100 file owned by (99/99) as user/group(99/500) FAIL 2023-03-29T12:12:54.918 INFO:teuthology.orchestra.run.smithi111.stdout: w a 002 file owned by (99/99) as user/group(12/100) PASS 2023-03-29T12:12:54.919 INFO:teuthology.orchestra.run.smithi111.stdout: w a 020 file owned by (99/99) as user/group(200/99) PASS 2023-03-29T12:12:54.919 INFO:teuthology.orchestra.run.smithi111.stdout: w a 200 file owned by (99/99) as user/group(99/500) PASS 2023-03-29T12:12:54.922 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/126 2023-03-29T12:12:54.922 INFO:teuthology.orchestra.run.smithi111.stdout:Failures: generic/126 2023-03-29T12:12:54.922 INFO:teuthology.orchestra.run.smithi111.stdout:Failed 1 of 1 tests 2023-03-29T12:12:54.927 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:12:54.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T12:12:54.946 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-03-29T12:12:54.947 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-03-29T12:12:54.947 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/127) 2023-03-29T12:12:55.682 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:12:55.693 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:12:55.694 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:13:17.161 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:13:17.167 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:13:47.347 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:13:47.352 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:14:17.555 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:14:17.559 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:14:47.678 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:14:47.682 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:15:17.841 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:15:17.846 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:15:49.773 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:15:49.778 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:16:19.955 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:16:19.960 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:16:50.069 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:16:50.073 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:17:20.183 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:17:20.187 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:17:50.329 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:17:50.333 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:18:20.437 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:18:20.442 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:18:50.625 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:18:50.630 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:19:20.763 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:19:20.767 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:19:50.870 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:19:50.874 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:20:21.748 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:20:21.753 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:20:51.863 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:20:51.869 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:21:22.062 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:21:22.067 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:21:52.208 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:21:52.213 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:22:22.372 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:22:22.377 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:22:53.520 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:22:53.524 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:23:23.624 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:23:23.629 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:23:53.737 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:23:53.742 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:24:23.991 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:24:23.994 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:24:54.324 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:24:54.328 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:25:24.421 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:25:24.426 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:25:54.536 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:25:54.540 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:26:24.653 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:26:24.657 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:26:54.788 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:26:54.793 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:27:24.896 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:27:24.900 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:27:55.043 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:27:55.048 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:28:25.164 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:28:25.169 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:28:55.286 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:28:55.291 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:29:26.442 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:29:26.447 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:29:48.068 INFO:teuthology.orchestra.run.smithi111.stdout:generic/127 1013s 2023-03-29T12:29:48.079 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/127 2023-03-29T12:29:48.079 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:29:48.085 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:29:48.126 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:29:48.126 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/128) 2023-03-29T12:29:48.823 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:29:48.833 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:29:48.833 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:29:49.130 INFO:teuthology.orchestra.run.smithi111.stdout:generic/128 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:29:49.138 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/128 2023-03-29T12:29:49.138 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/128 2023-03-29T12:29:49.138 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:29:49.143 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:29:49.163 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:29:49.163 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/129) 2023-03-29T12:29:49.541 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:29:49.553 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:29:49.553 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:29:49.822 INFO:teuthology.orchestra.run.smithi111.stdout:generic/129 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:29:49.830 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/129 2023-03-29T12:29:49.830 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/129 2023-03-29T12:29:49.830 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:29:49.836 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:29:49.854 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:29:49.854 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/130) 2023-03-29T12:29:50.232 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:29:50.243 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:29:50.244 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:29:50.522 INFO:teuthology.orchestra.run.smithi111.stdout:generic/130 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:29:50.530 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/130 2023-03-29T12:29:50.530 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/130 2023-03-29T12:29:50.530 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:29:50.536 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:29:50.554 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:29:50.555 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/131) 2023-03-29T12:29:50.940 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:29:50.951 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:29:50.951 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:29:52.449 INFO:teuthology.orchestra.run.smithi111.stdout:generic/131 2s 2023-03-29T12:29:52.461 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/131 2023-03-29T12:29:52.461 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:29:52.466 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:29:52.487 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:29:52.487 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/132) 2023-03-29T12:29:53.168 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:29:53.179 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:29:53.180 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:29:53.454 INFO:teuthology.orchestra.run.smithi111.stdout:generic/132 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:29:53.462 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/132 2023-03-29T12:29:53.462 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/132 2023-03-29T12:29:53.462 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:29:53.467 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:29:53.487 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:29:53.487 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/133) 2023-03-29T12:29:53.877 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:29:53.888 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:29:53.888 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:29:56.571 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:29:56.575 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:30:27.763 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:30:27.767 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:30:57.882 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:30:57.886 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:31:28.041 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:31:28.045 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:31:58.155 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:31:58.160 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:32:28.264 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:32:28.269 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:32:58.432 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:32:58.449 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:33:28.545 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:33:28.549 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:33:58.730 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:33:58.734 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:34:28.849 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:34:28.854 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:34:56.196 INFO:teuthology.orchestra.run.smithi111.stdout:generic/133 303s 2023-03-29T12:34:56.207 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/133 2023-03-29T12:34:56.207 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:34:56.212 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:34:56.233 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:34:56.234 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/134) 2023-03-29T12:34:57.051 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:34:57.062 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:34:57.062 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:34:57.433 INFO:teuthology.orchestra.run.smithi111.stdout:generic/134 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:34:57.441 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/134 2023-03-29T12:34:57.441 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/134 2023-03-29T12:34:57.441 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:34:57.447 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:34:57.466 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:34:57.466 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/135) 2023-03-29T12:34:58.170 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:34:58.181 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:34:58.181 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:34:58.517 INFO:teuthology.orchestra.run.smithi111.stdout:generic/135 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:34:58.525 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/135 2023-03-29T12:34:58.525 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/135 2023-03-29T12:34:58.526 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:34:58.531 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:34:58.551 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:34:58.552 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/136) 2023-03-29T12:34:58.933 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:34:58.945 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:34:58.946 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:34:58.953 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:34:58.957 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:34:59.326 INFO:teuthology.orchestra.run.smithi111.stdout:generic/136 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2023-03-29T12:34:59.334 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/136 2023-03-29T12:34:59.334 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/136 2023-03-29T12:34:59.334 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:34:59.340 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:34:59.358 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:34:59.359 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/137) 2023-03-29T12:35:00.057 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:00.069 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:00.069 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:00.442 INFO:teuthology.orchestra.run.smithi111.stdout:generic/137 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:00.451 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/137 2023-03-29T12:35:00.451 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/137 2023-03-29T12:35:00.451 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:00.456 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:00.497 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:00.497 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/138) 2023-03-29T12:35:01.193 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:01.204 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:01.204 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:01.559 INFO:teuthology.orchestra.run.smithi111.stdout:generic/138 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:01.566 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/138 2023-03-29T12:35:01.567 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/138 2023-03-29T12:35:01.567 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:01.572 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:01.618 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:01.618 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/139) 2023-03-29T12:35:02.275 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:02.286 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:02.286 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:02.636 INFO:teuthology.orchestra.run.smithi111.stdout:generic/139 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:02.644 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/139 2023-03-29T12:35:02.645 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/139 2023-03-29T12:35:02.645 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:02.650 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:02.669 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:02.670 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/140) 2023-03-29T12:35:03.354 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:03.365 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:03.365 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:03.716 INFO:teuthology.orchestra.run.smithi111.stdout:generic/140 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:03.724 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/140 2023-03-29T12:35:03.724 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/140 2023-03-29T12:35:03.724 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:03.729 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:03.749 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:03.749 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/141) 2023-03-29T12:35:04.425 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:04.436 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:04.436 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:04.710 INFO:teuthology.orchestra.run.smithi111.stdout:generic/141 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:04.718 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/141 2023-03-29T12:35:04.719 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/141 2023-03-29T12:35:04.719 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:04.724 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:04.743 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:04.743 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/142) 2023-03-29T12:35:05.121 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:05.132 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:05.133 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:05.481 INFO:teuthology.orchestra.run.smithi111.stdout:generic/142 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:05.489 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/142 2023-03-29T12:35:05.489 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/142 2023-03-29T12:35:05.489 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:05.494 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:05.545 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:05.546 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/143) 2023-03-29T12:35:06.215 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:06.227 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:06.227 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:06.577 INFO:teuthology.orchestra.run.smithi111.stdout:generic/143 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:06.585 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/143 2023-03-29T12:35:06.585 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/143 2023-03-29T12:35:06.585 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:06.591 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:06.633 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:06.634 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/144) 2023-03-29T12:35:07.315 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:07.326 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:07.327 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:07.674 INFO:teuthology.orchestra.run.smithi111.stdout:generic/144 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:07.682 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/144 2023-03-29T12:35:07.682 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/144 2023-03-29T12:35:07.682 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:07.687 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:07.718 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:07.718 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/145) 2023-03-29T12:35:08.403 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:08.414 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:08.414 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:08.763 INFO:teuthology.orchestra.run.smithi111.stdout:generic/145 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:08.771 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/145 2023-03-29T12:35:08.771 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/145 2023-03-29T12:35:08.771 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:08.777 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:08.796 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:08.796 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/146) 2023-03-29T12:35:09.487 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:09.498 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:09.498 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:09.850 INFO:teuthology.orchestra.run.smithi111.stdout:generic/146 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:09.858 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/146 2023-03-29T12:35:09.858 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/146 2023-03-29T12:35:09.858 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:09.863 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:09.892 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:09.892 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/147) 2023-03-29T12:35:10.574 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:10.586 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:10.586 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:11.336 INFO:teuthology.orchestra.run.smithi111.stdout:generic/147 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:11.344 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/147 2023-03-29T12:35:11.344 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/147 2023-03-29T12:35:11.344 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:11.349 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:11.370 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:11.370 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/148) 2023-03-29T12:35:12.066 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:12.077 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:12.077 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:12.624 INFO:teuthology.orchestra.run.smithi111.stdout:generic/148 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:12.632 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/148 2023-03-29T12:35:12.633 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/148 2023-03-29T12:35:12.633 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:12.638 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:12.658 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:12.659 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/149) 2023-03-29T12:35:13.350 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:13.361 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:13.361 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:13.707 INFO:teuthology.orchestra.run.smithi111.stdout:generic/149 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:13.715 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/149 2023-03-29T12:35:13.715 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/149 2023-03-29T12:35:13.715 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:13.720 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:13.741 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:13.741 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/150) 2023-03-29T12:35:14.446 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:14.457 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:14.457 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:14.833 INFO:teuthology.orchestra.run.smithi111.stdout:generic/150 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:14.841 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/150 2023-03-29T12:35:14.841 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/150 2023-03-29T12:35:14.841 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:14.921 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:14.947 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:14.947 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/151) 2023-03-29T12:35:15.639 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:15.650 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:15.650 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:16.013 INFO:teuthology.orchestra.run.smithi111.stdout:generic/151 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:16.021 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/151 2023-03-29T12:35:16.021 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/151 2023-03-29T12:35:16.022 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:16.027 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:16.053 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:16.053 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/152) 2023-03-29T12:35:16.750 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:16.762 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:16.762 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:17.114 INFO:teuthology.orchestra.run.smithi111.stdout:generic/152 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:17.122 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/152 2023-03-29T12:35:17.122 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/152 2023-03-29T12:35:17.122 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:17.128 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:17.180 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:17.180 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/153) 2023-03-29T12:35:17.939 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:17.950 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:17.950 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:18.297 INFO:teuthology.orchestra.run.smithi111.stdout:generic/153 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:18.306 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/153 2023-03-29T12:35:18.306 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/153 2023-03-29T12:35:18.306 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:18.311 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:18.348 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:18.349 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/154) 2023-03-29T12:35:19.029 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:19.041 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:19.041 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:19.390 INFO:teuthology.orchestra.run.smithi111.stdout:generic/154 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:19.398 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/154 2023-03-29T12:35:19.398 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/154 2023-03-29T12:35:19.398 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:19.404 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:19.428 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:19.429 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/155) 2023-03-29T12:35:20.135 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:20.146 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:20.146 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:20.502 INFO:teuthology.orchestra.run.smithi111.stdout:generic/155 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:20.510 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/155 2023-03-29T12:35:20.510 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/155 2023-03-29T12:35:20.510 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:20.515 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:20.544 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:20.544 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/156) 2023-03-29T12:35:21.237 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:21.248 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:21.248 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:21.604 INFO:teuthology.orchestra.run.smithi111.stdout:generic/156 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:21.612 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/156 2023-03-29T12:35:21.612 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/156 2023-03-29T12:35:21.612 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:21.618 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:21.673 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:21.676 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/157) 2023-03-29T12:35:22.330 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:22.341 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:22.341 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:22.702 INFO:teuthology.orchestra.run.smithi111.stdout:generic/157 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:22.711 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/157 2023-03-29T12:35:22.711 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/157 2023-03-29T12:35:22.711 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:22.716 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:22.796 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:22.796 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/158) 2023-03-29T12:35:23.448 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:23.459 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:23.459 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:23.826 INFO:teuthology.orchestra.run.smithi111.stdout:generic/158 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:23.834 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/158 2023-03-29T12:35:23.835 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/158 2023-03-29T12:35:23.835 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:23.840 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:23.858 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:23.859 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/159) 2023-03-29T12:35:24.572 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:24.584 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:24.584 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:24.932 INFO:teuthology.orchestra.run.smithi111.stdout:generic/159 [not run] file system doesn't support chattr +i 2023-03-29T12:35:24.940 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/159 2023-03-29T12:35:24.940 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/159 2023-03-29T12:35:24.940 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:24.945 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:24.964 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:24.964 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/160) 2023-03-29T12:35:25.353 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:25.365 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:25.365 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:25.684 INFO:teuthology.orchestra.run.smithi111.stdout:generic/160 [not run] file system doesn't support chattr +i 2023-03-29T12:35:25.692 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/160 2023-03-29T12:35:25.692 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/160 2023-03-29T12:35:25.692 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:25.697 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:25.716 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:25.716 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/161) 2023-03-29T12:35:26.100 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:26.111 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:26.111 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:26.382 INFO:teuthology.orchestra.run.smithi111.stdout:generic/161 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:26.390 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/161 2023-03-29T12:35:26.390 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/161 2023-03-29T12:35:26.391 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:26.396 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:26.415 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:26.415 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/162) 2023-03-29T12:35:26.794 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:26.805 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:26.805 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:27.072 INFO:teuthology.orchestra.run.smithi111.stdout:generic/162 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:27.080 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/162 2023-03-29T12:35:27.080 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/162 2023-03-29T12:35:27.080 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:27.085 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:27.103 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:27.103 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/163) 2023-03-29T12:35:27.483 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:27.494 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:27.494 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:27.769 INFO:teuthology.orchestra.run.smithi111.stdout:generic/163 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:27.777 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/163 2023-03-29T12:35:27.777 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/163 2023-03-29T12:35:27.778 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:27.783 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:27.801 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:27.801 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/164) 2023-03-29T12:35:28.187 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:28.198 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:28.199 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:28.472 INFO:teuthology.orchestra.run.smithi111.stdout:generic/164 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:28.480 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/164 2023-03-29T12:35:28.480 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/164 2023-03-29T12:35:28.480 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:28.485 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:28.504 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:28.504 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/165) 2023-03-29T12:35:28.890 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:28.902 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:28.902 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:29.070 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:35:29.074 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:35:29.175 INFO:teuthology.orchestra.run.smithi111.stdout:generic/165 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:29.195 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/165 2023-03-29T12:35:29.196 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/165 2023-03-29T12:35:29.196 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:29.201 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:29.219 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:29.220 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/166) 2023-03-29T12:35:29.647 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:29.658 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:29.658 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:29.934 INFO:teuthology.orchestra.run.smithi111.stdout:generic/166 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:29.942 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/166 2023-03-29T12:35:29.942 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/166 2023-03-29T12:35:29.942 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:29.947 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:29.966 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:29.966 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/167) 2023-03-29T12:35:30.348 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:30.360 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:30.360 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:30.632 INFO:teuthology.orchestra.run.smithi111.stdout:generic/167 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:30.640 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/167 2023-03-29T12:35:30.640 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/167 2023-03-29T12:35:30.640 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:30.646 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:30.665 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:30.665 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/168) 2023-03-29T12:35:31.051 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:31.062 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:31.063 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:31.337 INFO:teuthology.orchestra.run.smithi111.stdout:generic/168 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:31.345 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/168 2023-03-29T12:35:31.345 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/168 2023-03-29T12:35:31.345 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:31.350 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:31.369 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:31.369 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/169) 2023-03-29T12:35:31.753 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:31.764 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:31.764 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:32.033 INFO:teuthology.orchestra.run.smithi111.stdout:generic/169 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:32.041 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/169 2023-03-29T12:35:32.041 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/169 2023-03-29T12:35:32.041 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:32.046 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:32.065 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:32.065 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/170) 2023-03-29T12:35:32.447 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:32.458 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:32.458 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:32.729 INFO:teuthology.orchestra.run.smithi111.stdout:generic/170 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:32.738 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/170 2023-03-29T12:35:32.738 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/170 2023-03-29T12:35:32.738 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:32.743 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:32.761 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:32.762 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/171) 2023-03-29T12:35:33.143 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:33.155 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:33.155 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:33.442 INFO:teuthology.orchestra.run.smithi111.stdout:generic/171 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:33.450 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/171 2023-03-29T12:35:33.450 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/171 2023-03-29T12:35:33.450 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:33.456 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:33.473 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:33.474 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/172) 2023-03-29T12:35:33.854 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:33.865 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:33.865 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:34.148 INFO:teuthology.orchestra.run.smithi111.stdout:generic/172 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:34.156 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/172 2023-03-29T12:35:34.157 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/172 2023-03-29T12:35:34.157 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:34.162 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:34.180 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:34.181 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/173) 2023-03-29T12:35:34.566 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:34.578 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:34.578 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:34.856 INFO:teuthology.orchestra.run.smithi111.stdout:generic/173 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:34.865 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/173 2023-03-29T12:35:34.865 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/173 2023-03-29T12:35:34.865 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:34.870 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:34.888 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:34.888 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/174) 2023-03-29T12:35:35.267 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:35.278 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:35.278 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:35.557 INFO:teuthology.orchestra.run.smithi111.stdout:generic/174 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:35.566 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/174 2023-03-29T12:35:35.566 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/174 2023-03-29T12:35:35.566 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:35.571 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:35.589 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:35.590 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/175) 2023-03-29T12:35:35.975 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:35.986 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:35.986 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:36.266 INFO:teuthology.orchestra.run.smithi111.stdout:generic/175 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:36.274 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/175 2023-03-29T12:35:36.274 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/175 2023-03-29T12:35:36.274 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:36.279 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:36.298 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:36.298 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/176) 2023-03-29T12:35:36.683 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:36.694 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:36.695 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:36.968 INFO:teuthology.orchestra.run.smithi111.stdout:generic/176 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:36.976 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/176 2023-03-29T12:35:36.976 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/176 2023-03-29T12:35:36.976 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:36.981 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:37.000 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:37.000 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/177) 2023-03-29T12:35:37.388 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:37.400 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:37.400 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:37.773 INFO:teuthology.orchestra.run.smithi111.stdout:generic/177 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:37.782 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/177 2023-03-29T12:35:37.782 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/177 2023-03-29T12:35:37.782 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:37.787 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:37.805 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:37.806 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/178) 2023-03-29T12:35:38.185 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:38.197 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:38.197 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:38.566 INFO:teuthology.orchestra.run.smithi111.stdout:generic/178 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:38.574 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/178 2023-03-29T12:35:38.574 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/178 2023-03-29T12:35:38.575 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:38.579 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:38.615 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:38.615 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/179) 2023-03-29T12:35:39.293 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:39.304 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:39.304 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:39.654 INFO:teuthology.orchestra.run.smithi111.stdout:generic/179 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:39.662 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/179 2023-03-29T12:35:39.662 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/179 2023-03-29T12:35:39.662 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:39.668 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:39.687 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:39.687 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/180) 2023-03-29T12:35:40.385 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:40.396 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:40.396 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:40.746 INFO:teuthology.orchestra.run.smithi111.stdout:generic/180 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:40.754 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/180 2023-03-29T12:35:40.755 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/180 2023-03-29T12:35:40.755 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:40.759 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:40.805 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:40.805 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/181) 2023-03-29T12:35:41.480 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:41.492 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:41.492 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:41.896 INFO:teuthology.orchestra.run.smithi111.stdout:generic/181 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:41.904 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/181 2023-03-29T12:35:41.904 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/181 2023-03-29T12:35:41.904 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:41.910 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:41.930 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:41.930 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/182) 2023-03-29T12:35:42.658 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:42.669 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:42.670 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:43.045 INFO:teuthology.orchestra.run.smithi111.stdout:generic/182 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2023-03-29T12:35:43.053 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/182 2023-03-29T12:35:43.053 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/182 2023-03-29T12:35:43.053 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:43.058 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:43.076 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:43.076 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/183) 2023-03-29T12:35:43.777 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:43.789 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:43.789 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:44.065 INFO:teuthology.orchestra.run.smithi111.stdout:generic/183 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:44.073 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/183 2023-03-29T12:35:44.073 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/183 2023-03-29T12:35:44.073 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:44.079 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:44.097 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:44.097 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/184) 2023-03-29T12:35:44.478 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:44.489 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:44.490 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:44.853 INFO:teuthology.orchestra.run.smithi111.stdout:generic/184 [failed, exit status 1]- output mismatch (see /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/184.out.bad) 2023-03-29T12:35:44.857 INFO:teuthology.orchestra.run.smithi111.stdout: --- tests/generic/184.out 2023-03-29 10:59:21.760746951 +0000 2023-03-29T12:35:44.858 INFO:teuthology.orchestra.run.smithi111.stdout: +++ /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/184.out.bad 2023-03-29 12:35:44.852827096 +0000 2023-03-29T12:35:44.858 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -1 +1,2 @@ 2023-03-29T12:35:44.858 INFO:teuthology.orchestra.run.smithi111.stdout: QA output created by 184 2023-03-29T12:35:44.858 INFO:teuthology.orchestra.run.smithi111.stdout: +/tmp/tmp.CzuMqE4LuJxfstests-dev/tests/generic/184: line 23: /tmp/tmp.6dtVtxiLHFtest/null: Permission denied 2023-03-29T12:35:44.863 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/184 2023-03-29T12:35:44.863 INFO:teuthology.orchestra.run.smithi111.stdout:Failures: generic/184 2023-03-29T12:35:44.863 INFO:teuthology.orchestra.run.smithi111.stdout:Failed 1 of 1 tests 2023-03-29T12:35:44.868 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:44.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T12:35:44.886 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-03-29T12:35:44.886 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-03-29T12:35:44.886 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/185) 2023-03-29T12:35:45.272 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:45.284 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:45.284 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:45.561 INFO:teuthology.orchestra.run.smithi111.stdout:generic/185 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:45.569 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/185 2023-03-29T12:35:45.569 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/185 2023-03-29T12:35:45.569 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:45.574 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:45.593 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:45.593 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/186) 2023-03-29T12:35:45.978 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:45.989 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:45.989 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:46.265 INFO:teuthology.orchestra.run.smithi111.stdout:generic/186 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:46.273 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/186 2023-03-29T12:35:46.273 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/186 2023-03-29T12:35:46.273 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:46.278 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:46.297 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:46.297 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/187) 2023-03-29T12:35:46.683 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:46.694 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:46.694 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:46.964 INFO:teuthology.orchestra.run.smithi111.stdout:generic/187 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:46.972 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/187 2023-03-29T12:35:46.973 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/187 2023-03-29T12:35:46.973 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:46.978 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:46.997 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:46.997 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/188) 2023-03-29T12:35:47.387 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:47.398 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:47.398 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:47.679 INFO:teuthology.orchestra.run.smithi111.stdout:generic/188 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:47.688 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/188 2023-03-29T12:35:47.688 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/188 2023-03-29T12:35:47.688 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:47.693 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:47.711 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:47.711 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/189) 2023-03-29T12:35:48.100 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:48.111 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:48.111 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:48.386 INFO:teuthology.orchestra.run.smithi111.stdout:generic/189 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:48.394 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/189 2023-03-29T12:35:48.394 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/189 2023-03-29T12:35:48.394 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:48.399 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:48.417 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:48.417 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/190) 2023-03-29T12:35:48.804 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:48.814 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:48.815 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:49.088 INFO:teuthology.orchestra.run.smithi111.stdout:generic/190 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:49.096 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/190 2023-03-29T12:35:49.096 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/190 2023-03-29T12:35:49.096 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:49.101 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:49.119 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:49.120 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/191) 2023-03-29T12:35:49.507 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:49.518 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:49.518 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:49.790 INFO:teuthology.orchestra.run.smithi111.stdout:generic/191 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:49.798 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/191 2023-03-29T12:35:49.798 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/191 2023-03-29T12:35:49.798 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:49.803 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:49.821 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:49.821 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/192) 2023-03-29T12:35:50.211 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:50.222 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:50.222 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:55.887 INFO:teuthology.orchestra.run.smithi111.stdout:generic/192 - output mismatch (see /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/192.out.bad) 2023-03-29T12:35:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: --- tests/generic/192.out 2023-03-29 10:59:21.761746935 +0000 2023-03-29T12:35:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: +++ /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/192.out.bad 2023-03-29 12:35:55.887644820 +0000 2023-03-29T12:35:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -1,5 +1,6 @@ 2023-03-29T12:35:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: QA output created by 192 2023-03-29T12:35:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: sleep for 5 seconds 2023-03-29T12:35:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: test 2023-03-29T12:35:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: -delta1 is in range 2023-03-29T12:35:55.892 INFO:teuthology.orchestra.run.smithi111.stdout: +delta1 has value of 0 2023-03-29T12:35:55.893 INFO:teuthology.orchestra.run.smithi111.stdout: +delta1 is NOT in range 5 .. 7 2023-03-29T12:35:55.893 INFO:teuthology.orchestra.run.smithi111.stdout: delta2 is in range 2023-03-29T12:35:55.897 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/192 2023-03-29T12:35:55.897 INFO:teuthology.orchestra.run.smithi111.stdout:Failures: generic/192 2023-03-29T12:35:55.898 INFO:teuthology.orchestra.run.smithi111.stdout:Failed 1 of 1 tests 2023-03-29T12:35:55.903 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:55.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T12:35:55.921 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-03-29T12:35:55.921 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-03-29T12:35:55.921 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/193) 2023-03-29T12:35:56.307 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:56.318 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:56.318 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:58.159 INFO:teuthology.orchestra.run.smithi111.stdout:generic/193 - output mismatch (see /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/193.out.bad) 2023-03-29T12:35:58.164 INFO:teuthology.orchestra.run.smithi111.stdout: --- tests/generic/193.out 2023-03-29 10:59:21.761746935 +0000 2023-03-29T12:35:58.164 INFO:teuthology.orchestra.run.smithi111.stdout: +++ /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/193.out.bad 2023-03-29 12:35:58.159607290 +0000 2023-03-29T12:35:58.164 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -24,7 +24,6 @@ 2023-03-29T12:35:58.164 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:58.164 INFO:teuthology.orchestra.run.smithi111.stdout: user: chmod a+r on qa_user owned file (should succeed) 2023-03-29T12:35:58.164 INFO:teuthology.orchestra.run.smithi111.stdout: user: chmod a+r on root owned file (should fail) 2023-03-29T12:35:58.164 INFO:teuthology.orchestra.run.smithi111.stdout: -chmod: changing permissions of 'test.root': Operation not permitted 2023-03-29T12:35:58.164 INFO:teuthology.orchestra.run.smithi111.stdout: check that the sgid bit is cleared 2023-03-29T12:35:58.164 INFO:teuthology.orchestra.run.smithi111.stdout: -rw-rw-rw- 2023-03-29T12:35:58.165 INFO:teuthology.orchestra.run.smithi111.stdout: check that suid bit is not cleared 2023-03-29T12:35:58.169 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/193 2023-03-29T12:35:58.169 INFO:teuthology.orchestra.run.smithi111.stdout:Failures: generic/193 2023-03-29T12:35:58.169 INFO:teuthology.orchestra.run.smithi111.stdout:Failed 1 of 1 tests 2023-03-29T12:35:58.175 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:58.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T12:35:58.233 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-03-29T12:35:58.233 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-03-29T12:35:58.234 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/194) 2023-03-29T12:35:58.922 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:58.933 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:58.933 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:59.203 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:35:59.208 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:35:59.218 INFO:teuthology.orchestra.run.smithi111.stdout:generic/194 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:59.226 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/194 2023-03-29T12:35:59.226 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/194 2023-03-29T12:35:59.227 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:59.233 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:59.253 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:35:59.253 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/195) 2023-03-29T12:35:59.681 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:35:59.692 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:35:59.692 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:35:59.980 INFO:teuthology.orchestra.run.smithi111.stdout:generic/195 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:35:59.989 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/195 2023-03-29T12:35:59.989 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/195 2023-03-29T12:35:59.989 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:35:59.994 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:00.013 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:36:00.013 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/196) 2023-03-29T12:36:00.400 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:36:00.411 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:36:00.411 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:00.683 INFO:teuthology.orchestra.run.smithi111.stdout:generic/196 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:36:00.691 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/196 2023-03-29T12:36:00.691 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/196 2023-03-29T12:36:00.691 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:36:00.696 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:00.715 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:36:00.715 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/197) 2023-03-29T12:36:01.099 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:36:01.110 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:36:01.110 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:01.382 INFO:teuthology.orchestra.run.smithi111.stdout:generic/197 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:36:01.390 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/197 2023-03-29T12:36:01.390 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/197 2023-03-29T12:36:01.390 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:36:01.396 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:01.415 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:36:01.415 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/198) 2023-03-29T12:36:01.802 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:36:01.813 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:36:01.814 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:03.019 INFO:teuthology.orchestra.run.smithi111.stdout:generic/198 2s 2023-03-29T12:36:03.030 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/198 2023-03-29T12:36:03.030 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:36:03.035 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:03.054 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:36:03.054 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/199) 2023-03-29T12:36:03.810 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:36:03.822 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:36:03.822 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:04.101 INFO:teuthology.orchestra.run.smithi111.stdout:generic/199 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:36:04.109 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/199 2023-03-29T12:36:04.109 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/199 2023-03-29T12:36:04.110 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:36:04.115 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:04.133 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:36:04.133 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/200) 2023-03-29T12:36:04.519 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:36:04.530 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:36:04.531 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:04.804 INFO:teuthology.orchestra.run.smithi111.stdout:generic/200 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:36:04.812 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/200 2023-03-29T12:36:04.812 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/200 2023-03-29T12:36:04.812 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:36:04.817 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:04.837 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:36:04.837 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/201) 2023-03-29T12:36:05.229 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:36:05.240 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:36:05.240 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:05.514 INFO:teuthology.orchestra.run.smithi111.stdout:generic/201 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:36:05.522 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/201 2023-03-29T12:36:05.522 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/201 2023-03-29T12:36:05.522 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:36:05.527 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:05.546 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:36:05.546 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/202) 2023-03-29T12:36:05.932 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:36:05.943 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:36:05.943 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:06.217 INFO:teuthology.orchestra.run.smithi111.stdout:generic/202 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:36:06.225 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/202 2023-03-29T12:36:06.225 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/202 2023-03-29T12:36:06.225 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:36:06.231 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:06.249 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:36:06.250 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/203) 2023-03-29T12:36:06.634 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:36:06.645 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:36:06.645 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:06.916 INFO:teuthology.orchestra.run.smithi111.stdout:generic/203 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:36:06.924 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/203 2023-03-29T12:36:06.924 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/203 2023-03-29T12:36:06.924 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:36:06.929 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:06.949 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:36:06.949 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/204) 2023-03-29T12:36:07.334 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:36:07.345 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:36:07.345 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:08.004 INFO:teuthology.orchestra.run.smithi111.stdout:generic/204 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:36:08.012 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/204 2023-03-29T12:36:08.013 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/204 2023-03-29T12:36:08.013 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:36:08.018 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:08.037 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:36:08.037 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/205) 2023-03-29T12:36:08.430 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:36:08.442 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:36:08.442 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:08.721 INFO:teuthology.orchestra.run.smithi111.stdout:generic/205 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:36:08.729 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/205 2023-03-29T12:36:08.729 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/205 2023-03-29T12:36:08.729 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:36:08.735 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:08.753 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:36:08.753 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/206) 2023-03-29T12:36:09.136 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:36:09.177 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:36:09.177 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:09.436 INFO:teuthology.orchestra.run.smithi111.stdout:generic/206 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:36:09.444 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/206 2023-03-29T12:36:09.444 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/206 2023-03-29T12:36:09.444 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:36:09.449 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:09.468 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:36:09.469 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/207) 2023-03-29T12:36:09.856 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:36:09.867 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:36:09.867 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:29.313 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:36:29.318 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:36:30.803 INFO:teuthology.orchestra.run.smithi111.stdout:generic/207 21s 2023-03-29T12:36:30.815 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/207 2023-03-29T12:36:30.816 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:36:30.821 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:30.857 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:36:30.857 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/208) 2023-03-29T12:36:31.544 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:36:31.555 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:36:31.555 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:36:59.442 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:36:59.446 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:37:29.548 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:37:29.552 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:37:59.646 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:37:59.650 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:38:29.767 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:38:29.771 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:38:59.864 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:38:59.868 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:39:30.004 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:39:30.008 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:39:52.261 INFO:teuthology.orchestra.run.smithi111.stdout:generic/208 201s 2023-03-29T12:39:52.272 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/208 2023-03-29T12:39:52.273 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:39:52.280 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:39:52.348 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:39:52.348 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/209) 2023-03-29T12:39:52.997 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:39:53.008 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:39:53.008 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:00.117 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:40:00.121 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:40:04.938 INFO:teuthology.orchestra.run.smithi111.stdout:generic/209 [failed, exit status 1]- output mismatch (see /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/209.out.bad) 2023-03-29T12:40:04.965 INFO:teuthology.orchestra.run.smithi111.stdout: --- tests/generic/209.out 2023-03-29 10:59:21.764746886 +0000 2023-03-29T12:40:04.966 INFO:teuthology.orchestra.run.smithi111.stdout: +++ /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/209.out.bad 2023-03-29 12:40:04.928523887 +0000 2023-03-29T12:40:04.966 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -1,2 +1,2 @@ 2023-03-29T12:40:04.966 INFO:teuthology.orchestra.run.smithi111.stdout: QA output created by 209 2023-03-29T12:40:04.966 INFO:teuthology.orchestra.run.smithi111.stdout: -test ran for 30 seconds without error 2023-03-29T12:40:04.966 INFO:teuthology.orchestra.run.smithi111.stdout: +reader found old byte at pos 4096 2023-03-29T12:40:04.966 INFO:teuthology.orchestra.run.smithi111.stdout: \ No newline at end of file 2023-03-29T12:40:05.000 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/209 2023-03-29T12:40:05.000 INFO:teuthology.orchestra.run.smithi111.stdout:Failures: generic/209 2023-03-29T12:40:05.000 INFO:teuthology.orchestra.run.smithi111.stdout:Failed 1 of 1 tests 2023-03-29T12:40:05.026 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:05.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T12:40:05.078 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-03-29T12:40:05.078 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-03-29T12:40:05.078 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/210) 2023-03-29T12:40:05.634 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:05.646 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:05.646 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:06.092 INFO:teuthology.orchestra.run.smithi111.stdout:generic/210 1s 2023-03-29T12:40:06.103 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/210 2023-03-29T12:40:06.103 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:06.109 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:06.166 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:06.166 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/211) 2023-03-29T12:40:06.814 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:06.825 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:06.826 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:07.594 INFO:teuthology.orchestra.run.smithi111.stdout:generic/211 1s 2023-03-29T12:40:07.606 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/211 2023-03-29T12:40:07.606 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:07.611 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:07.631 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:07.631 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/212) 2023-03-29T12:40:08.014 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:08.025 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:08.025 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:08.422 INFO:teuthology.orchestra.run.smithi111.stdout:generic/212 0s 2023-03-29T12:40:08.433 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/212 2023-03-29T12:40:08.433 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:08.438 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:08.457 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:08.457 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/213) 2023-03-29T12:40:08.837 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:08.849 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:08.849 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:09.505 INFO:teuthology.orchestra.run.smithi111.stdout:generic/213 - output mismatch (see /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/213.out.bad) 2023-03-29T12:40:09.510 INFO:teuthology.orchestra.run.smithi111.stdout: --- tests/generic/213.out 2023-03-29 10:59:21.764746886 +0000 2023-03-29T12:40:09.510 INFO:teuthology.orchestra.run.smithi111.stdout: +++ /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/213.out.bad 2023-03-29 12:40:09.506447976 +0000 2023-03-29T12:40:09.510 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -1,4 +1,3 @@ 2023-03-29T12:40:09.510 INFO:teuthology.orchestra.run.smithi111.stdout: QA output created by 213 2023-03-29T12:40:09.510 INFO:teuthology.orchestra.run.smithi111.stdout: We should get: fallocate: No space left on device 2023-03-29T12:40:09.510 INFO:teuthology.orchestra.run.smithi111.stdout: Strangely, xfs_io sometimes says "Success" when something went wrong, FYI 2023-03-29T12:40:09.510 INFO:teuthology.orchestra.run.smithi111.stdout: -fallocate: No space left on device 2023-03-29T12:40:09.515 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/213 2023-03-29T12:40:09.515 INFO:teuthology.orchestra.run.smithi111.stdout:Failures: generic/213 2023-03-29T12:40:09.515 INFO:teuthology.orchestra.run.smithi111.stdout:Failed 1 of 1 tests 2023-03-29T12:40:09.521 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:09.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T12:40:09.541 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-03-29T12:40:09.541 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-03-29T12:40:09.541 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/214) 2023-03-29T12:40:10.423 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:10.437 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:10.437 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:11.506 INFO:teuthology.orchestra.run.smithi111.stdout:generic/214 1s 2023-03-29T12:40:11.525 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/214 2023-03-29T12:40:11.525 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:11.534 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:11.567 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:11.567 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/215) 2023-03-29T12:40:12.398 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:12.414 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:12.414 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:16.519 INFO:teuthology.orchestra.run.smithi111.stdout:generic/215 4s 2023-03-29T12:40:16.541 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/215 2023-03-29T12:40:16.541 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:16.549 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:16.578 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:16.578 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/216) 2023-03-29T12:40:17.180 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:17.197 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:17.197 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:17.621 INFO:teuthology.orchestra.run.smithi111.stdout:generic/216 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:40:17.633 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/216 2023-03-29T12:40:17.633 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/216 2023-03-29T12:40:17.633 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:17.641 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:17.670 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:17.670 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/217) 2023-03-29T12:40:18.269 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:18.293 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:18.293 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:18.706 INFO:teuthology.orchestra.run.smithi111.stdout:generic/217 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:40:18.718 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/217 2023-03-29T12:40:18.718 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/217 2023-03-29T12:40:18.718 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:18.725 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:18.753 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:18.753 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/218) 2023-03-29T12:40:19.362 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:19.378 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:19.378 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:20.185 INFO:teuthology.orchestra.run.smithi111.stdout:generic/218 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:40:20.202 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/218 2023-03-29T12:40:20.203 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/218 2023-03-29T12:40:20.203 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:20.212 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:20.245 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:20.245 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/219) 2023-03-29T12:40:21.046 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:21.066 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:21.066 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:21.758 INFO:teuthology.orchestra.run.smithi111.stdout:generic/219 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:40:21.769 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/219 2023-03-29T12:40:21.770 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/219 2023-03-29T12:40:21.770 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:21.782 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:21.813 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:21.813 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/220) 2023-03-29T12:40:22.408 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:22.428 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:22.428 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:22.854 INFO:teuthology.orchestra.run.smithi111.stdout:generic/220 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:40:22.867 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/220 2023-03-29T12:40:22.868 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/220 2023-03-29T12:40:22.868 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:22.874 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:22.905 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:22.905 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/221) 2023-03-29T12:40:23.490 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:23.512 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:23.512 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:25.160 INFO:teuthology.orchestra.run.smithi111.stdout:generic/221 2s 2023-03-29T12:40:25.181 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/221 2023-03-29T12:40:25.181 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:25.191 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:25.227 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:25.228 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/222) 2023-03-29T12:40:26.137 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:26.153 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:26.153 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:26.664 INFO:teuthology.orchestra.run.smithi111.stdout:generic/222 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:40:26.677 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/222 2023-03-29T12:40:26.677 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/222 2023-03-29T12:40:26.677 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:26.689 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:26.722 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:26.723 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/223) 2023-03-29T12:40:27.325 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:27.342 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:27.342 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:27.765 INFO:teuthology.orchestra.run.smithi111.stdout:generic/223 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:40:27.781 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/223 2023-03-29T12:40:27.781 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/223 2023-03-29T12:40:27.781 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:27.789 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:27.823 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:27.823 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/224) 2023-03-29T12:40:28.490 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:28.504 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:28.504 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:28.936 INFO:teuthology.orchestra.run.smithi111.stdout:generic/224 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:40:28.950 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/224 2023-03-29T12:40:28.950 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/224 2023-03-29T12:40:28.950 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:28.957 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:28.981 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:28.981 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/225) 2023-03-29T12:40:29.572 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:29.590 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:29.590 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:30.005 INFO:teuthology.orchestra.run.smithi111.stdout:generic/225 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:40:30.020 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/225 2023-03-29T12:40:30.020 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/225 2023-03-29T12:40:30.020 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:30.028 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:30.057 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:30.057 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/226) 2023-03-29T12:40:30.238 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:40:30.243 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:40:30.785 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:30.804 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:30.804 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:31.222 INFO:teuthology.orchestra.run.smithi111.stdout:generic/226 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:40:31.233 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/226 2023-03-29T12:40:31.233 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/226 2023-03-29T12:40:31.233 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:31.240 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:31.275 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:31.275 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/227) 2023-03-29T12:40:31.945 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:31.964 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:31.965 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:32.384 INFO:teuthology.orchestra.run.smithi111.stdout:generic/227 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:40:32.396 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/227 2023-03-29T12:40:32.396 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/227 2023-03-29T12:40:32.396 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:32.405 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:32.434 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:32.434 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/228) 2023-03-29T12:40:33.021 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:33.037 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:33.037 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:33.780 INFO:teuthology.orchestra.run.smithi111.stdout:generic/228 0s 2023-03-29T12:40:33.796 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/228 2023-03-29T12:40:33.796 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:33.806 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:33.868 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:33.868 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/229) 2023-03-29T12:40:34.712 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:34.726 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:34.726 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:35.157 INFO:teuthology.orchestra.run.smithi111.stdout:generic/229 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:40:35.171 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/229 2023-03-29T12:40:35.171 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/229 2023-03-29T12:40:35.171 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:35.179 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:35.212 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:35.213 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/230) 2023-03-29T12:40:35.806 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:35.821 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:35.822 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:36.230 INFO:teuthology.orchestra.run.smithi111.stdout:generic/230 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:40:36.240 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/230 2023-03-29T12:40:36.240 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/230 2023-03-29T12:40:36.240 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:36.248 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:36.285 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:36.285 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/231) 2023-03-29T12:40:36.880 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:36.898 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:36.898 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:37.337 INFO:teuthology.orchestra.run.smithi111.stdout:generic/231 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:40:37.350 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/231 2023-03-29T12:40:37.350 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/231 2023-03-29T12:40:37.350 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:37.360 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:37.391 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:37.391 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/232) 2023-03-29T12:40:38.041 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:38.072 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:38.072 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:38.508 INFO:teuthology.orchestra.run.smithi111.stdout:generic/232 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:40:38.523 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/232 2023-03-29T12:40:38.523 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/232 2023-03-29T12:40:38.523 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:38.538 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:38.569 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:38.570 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/233) 2023-03-29T12:40:39.179 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:39.194 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:39.194 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:39.690 INFO:teuthology.orchestra.run.smithi111.stdout:generic/233 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:40:39.712 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/233 2023-03-29T12:40:39.712 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/233 2023-03-29T12:40:39.712 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:39.719 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:39.751 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:39.751 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/234) 2023-03-29T12:40:40.351 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:40.369 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:40.369 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:40.809 INFO:teuthology.orchestra.run.smithi111.stdout:generic/234 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:40:40.830 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/234 2023-03-29T12:40:40.830 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/234 2023-03-29T12:40:40.830 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:40.841 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:40.869 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:40.869 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/235) 2023-03-29T12:40:41.503 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:41.520 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:41.520 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:41.955 INFO:teuthology.orchestra.run.smithi111.stdout:generic/235 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:40:41.969 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/235 2023-03-29T12:40:41.969 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/235 2023-03-29T12:40:41.969 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:41.979 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:42.004 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:42.004 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/236) 2023-03-29T12:40:42.619 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:42.635 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:42.636 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:44.261 INFO:teuthology.orchestra.run.smithi111.stdout:generic/236 2s 2023-03-29T12:40:44.283 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/236 2023-03-29T12:40:44.283 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:44.290 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:44.325 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:44.325 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/237) 2023-03-29T12:40:45.224 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:45.246 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:45.246 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:45.735 INFO:teuthology.orchestra.run.smithi111.stdout:generic/237 [not run] ACLs not supported by this filesystem type: ceph-fuse 2023-03-29T12:40:45.745 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/237 2023-03-29T12:40:45.745 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/237 2023-03-29T12:40:45.746 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:45.755 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:45.786 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:45.786 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/238) 2023-03-29T12:40:46.358 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:46.371 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:46.372 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:46.789 INFO:teuthology.orchestra.run.smithi111.stdout:generic/238 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:40:46.802 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/238 2023-03-29T12:40:46.802 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/238 2023-03-29T12:40:46.802 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:40:46.809 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:40:46.842 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:40:46.842 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/239) 2023-03-29T12:40:47.457 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:40:47.472 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:40:47.473 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:41:00.436 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:41:00.461 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:41:30.764 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:41:30.776 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:41:31.075 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-03-29T12:41:31.075+0000 7f046aff9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 310603) UID: 0 2023-03-29T12:41:31.076 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-03-29T12:41:31.078+0000 7f93ce7f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 310603) UID: 0 2023-03-29T12:41:31.081 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T12:41:31.080+0000 7f7a699f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 310603) UID: 0 2023-03-29T12:41:31.089 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-03-29T12:41:31.072+0000 7f22b21f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 310603) UID: 0 2023-03-29T12:43:22.402 INFO:teuthology.orchestra.run.smithi111.stdout:generic/239 155s 2023-03-29T12:43:22.423 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/239 2023-03-29T12:43:22.423 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:43:22.434 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:43:22.470 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:43:22.471 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/240) 2023-03-29T12:43:23.525 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:43:23.539 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:43:23.539 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:43:27.901 INFO:teuthology.orchestra.run.smithi111.stdout:generic/240 4s 2023-03-29T12:43:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/240 2023-03-29T12:43:27.926 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:43:27.939 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:43:27.973 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:43:27.974 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/241) 2023-03-29T12:43:28.968 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:43:28.987 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:43:28.988 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:44:43.964 INFO:teuthology.orchestra.run.smithi111.stdout:generic/241 74s 2023-03-29T12:44:43.988 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/241 2023-03-29T12:44:43.988 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:44:44.001 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:44:44.048 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:44:44.048 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/242) 2023-03-29T12:44:45.053 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:44:45.071 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:44:45.072 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:44:45.593 INFO:teuthology.orchestra.run.smithi111.stdout:generic/242 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:44:45.606 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/242 2023-03-29T12:44:45.606 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/242 2023-03-29T12:44:45.606 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:44:45.615 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:44:45.657 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:44:45.657 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/243) 2023-03-29T12:44:46.386 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:44:46.409 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:44:46.409 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:44:47.606 INFO:teuthology.orchestra.run.smithi111.stdout:generic/243 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:44:47.619 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/243 2023-03-29T12:44:47.620 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/243 2023-03-29T12:44:47.620 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:44:47.629 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:44:47.666 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:44:47.666 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/244) 2023-03-29T12:44:48.349 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:44:48.370 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:44:48.370 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:44:49.267 INFO:teuthology.orchestra.run.smithi111.stdout:generic/244 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2023-03-29T12:44:49.284 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/244 2023-03-29T12:44:49.284 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/244 2023-03-29T12:44:49.284 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:44:49.292 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:44:49.323 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:44:49.323 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/245) 2023-03-29T12:44:50.124 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:44:50.143 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:44:50.143 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:44:51.869 INFO:teuthology.orchestra.run.smithi111.stdout:generic/245 1s 2023-03-29T12:44:51.891 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/245 2023-03-29T12:44:51.891 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:44:51.902 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:44:51.971 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:44:51.971 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/246) 2023-03-29T12:44:52.963 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:44:52.985 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:44:52.985 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:44:53.605 INFO:teuthology.orchestra.run.smithi111.stdout:generic/246 0s 2023-03-29T12:44:53.624 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/246 2023-03-29T12:44:53.624 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:44:53.636 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:44:53.675 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:44:53.676 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/247) 2023-03-29T12:44:54.660 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:44:54.679 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:44:54.679 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:45:33.370 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:45:33.377 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:45:33.667 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-03-29T12:45:33.671+0000 7fd2277f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123658) UID: 0 2023-03-29T12:45:33.668 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-03-29T12:45:33.672+0000 7fabb15f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123658) UID: 0 2023-03-29T12:45:33.668 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-03-29T12:45:33.671+0000 7f9655df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123658) UID: 0 2023-03-29T12:45:33.670 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-03-29T12:45:33.671+0000 7f0c30df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123658) UID: 0 2023-03-29T12:45:46.063 INFO:teuthology.orchestra.run.smithi111.stdout:generic/247 52s 2023-03-29T12:45:46.082 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/247 2023-03-29T12:45:46.082 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:45:46.090 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:45:46.113 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:45:46.114 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/248) 2023-03-29T12:45:46.688 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:45:46.702 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:45:46.702 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:45:47.248 INFO:teuthology.orchestra.run.smithi111.stdout:generic/248 1s 2023-03-29T12:45:47.266 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/248 2023-03-29T12:45:47.266 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:45:47.278 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:45:47.310 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:45:47.310 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/249) 2023-03-29T12:45:47.933 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:45:47.949 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:45:47.950 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:45:52.333 INFO:teuthology.orchestra.run.smithi111.stdout:generic/249 5s 2023-03-29T12:45:52.354 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/249 2023-03-29T12:45:52.354 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:45:52.361 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:45:52.403 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:45:52.403 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/250) 2023-03-29T12:45:53.328 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:45:53.344 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:45:53.344 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:45:53.857 INFO:teuthology.orchestra.run.smithi111.stdout:generic/250 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:45:53.869 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/250 2023-03-29T12:45:53.869 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/250 2023-03-29T12:45:53.869 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:45:53.880 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:45:53.912 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:45:53.913 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/251) 2023-03-29T12:45:54.473 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:45:54.486 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:45:54.486 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:45:54.920 INFO:teuthology.orchestra.run.smithi111.stdout:generic/251 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:45:54.931 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/251 2023-03-29T12:45:54.931 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/251 2023-03-29T12:45:54.932 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:45:54.940 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:45:54.973 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:45:54.974 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/252) 2023-03-29T12:45:55.607 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:45:55.625 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:45:55.626 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:45:56.082 INFO:teuthology.orchestra.run.smithi111.stdout:generic/252 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:45:56.094 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/252 2023-03-29T12:45:56.094 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/252 2023-03-29T12:45:56.094 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:45:56.102 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:45:56.136 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:45:56.136 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/253) 2023-03-29T12:45:56.724 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:45:56.737 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:45:56.738 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:45:57.176 INFO:teuthology.orchestra.run.smithi111.stdout:generic/253 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:45:57.187 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/253 2023-03-29T12:45:57.188 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/253 2023-03-29T12:45:57.188 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:45:57.193 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:45:57.223 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:45:57.223 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/254) 2023-03-29T12:45:57.814 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:45:57.836 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:45:57.836 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:45:58.253 INFO:teuthology.orchestra.run.smithi111.stdout:generic/254 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:45:58.264 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/254 2023-03-29T12:45:58.264 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/254 2023-03-29T12:45:58.264 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:45:58.272 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:45:58.320 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:45:58.321 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/255) 2023-03-29T12:45:58.941 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:45:58.958 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:45:58.959 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:46:01.746 INFO:teuthology.orchestra.run.smithi111.stdout:generic/255 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-03-29T12:46:01.757 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/255 2023-03-29T12:46:01.757 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/255 2023-03-29T12:46:01.757 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:46:01.765 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:46:01.795 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:46:01.796 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/256) 2023-03-29T12:46:02.700 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:46:02.727 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:46:02.727 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:46:05.330 INFO:teuthology.orchestra.run.smithi111.stdout:generic/256 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:46:05.343 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/256 2023-03-29T12:46:05.344 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/256 2023-03-29T12:46:05.344 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:46:05.354 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:46:05.386 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:46:05.386 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/257) 2023-03-29T12:46:05.988 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:46:06.009 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:46:06.009 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:46:08.664 INFO:teuthology.orchestra.run.smithi111.stdout:generic/257 2s 2023-03-29T12:46:08.680 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/257 2023-03-29T12:46:08.680 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:46:08.689 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:46:08.729 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:46:08.729 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/258) 2023-03-29T12:46:09.622 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:46:09.639 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:46:09.639 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:46:10.158 INFO:teuthology.orchestra.run.smithi111.stdout:generic/258 [failed, exit status 1]- output mismatch (see /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/258.out.bad) 2023-03-29T12:46:10.170 INFO:teuthology.orchestra.run.smithi111.stdout: --- tests/generic/258.out 2023-03-29 10:59:21.771746771 +0000 2023-03-29T12:46:10.170 INFO:teuthology.orchestra.run.smithi111.stdout: +++ /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/258.out.bad 2023-03-29 12:46:10.158461805 +0000 2023-03-29T12:46:10.170 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -1,5 +1,6 @@ 2023-03-29T12:46:10.170 INFO:teuthology.orchestra.run.smithi111.stdout: QA output created by 258 2023-03-29T12:46:10.171 INFO:teuthology.orchestra.run.smithi111.stdout: Creating file with timestamp of Jan 1, 1960 2023-03-29T12:46:10.171 INFO:teuthology.orchestra.run.smithi111.stdout: Testing for negative seconds since epoch 2023-03-29T12:46:10.171 INFO:teuthology.orchestra.run.smithi111.stdout: -Remounting to flush cache 2023-03-29T12:46:10.171 INFO:teuthology.orchestra.run.smithi111.stdout: -Testing for negative seconds since epoch 2023-03-29T12:46:10.171 INFO:teuthology.orchestra.run.smithi111.stdout: +Timestamp wrapped: 3979351756 2023-03-29T12:46:10.171 INFO:teuthology.orchestra.run.smithi111.stdout: +Timestamp wrapped 2023-03-29T12:46:10.171 INFO:teuthology.orchestra.run.smithi111.stdout: +(see /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/258.full for details) 2023-03-29T12:46:10.176 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/258 2023-03-29T12:46:10.176 INFO:teuthology.orchestra.run.smithi111.stdout:Failures: generic/258 2023-03-29T12:46:10.176 INFO:teuthology.orchestra.run.smithi111.stdout:Failed 1 of 1 tests 2023-03-29T12:46:10.183 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:46:10.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T12:46:10.213 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-03-29T12:46:10.213 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-03-29T12:46:10.213 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/259) 2023-03-29T12:46:10.794 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:46:10.816 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:46:10.817 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:46:11.237 INFO:teuthology.orchestra.run.smithi111.stdout:generic/259 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:46:11.252 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/259 2023-03-29T12:46:11.252 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/259 2023-03-29T12:46:11.252 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:46:11.261 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:46:11.294 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:46:11.294 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/260) 2023-03-29T12:46:11.890 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:46:11.906 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:46:11.906 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:46:12.316 INFO:teuthology.orchestra.run.smithi111.stdout:generic/260 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:46:12.330 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/260 2023-03-29T12:46:12.330 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/260 2023-03-29T12:46:12.330 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:46:12.340 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:46:12.369 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:46:12.369 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/261) 2023-03-29T12:46:12.955 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:46:12.971 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:46:12.971 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:46:13.396 INFO:teuthology.orchestra.run.smithi111.stdout:generic/261 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:46:13.414 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/261 2023-03-29T12:46:13.414 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/261 2023-03-29T12:46:13.414 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:46:13.421 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:46:13.448 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:46:13.449 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/262) 2023-03-29T12:46:14.036 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:46:14.059 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:46:14.059 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:46:14.472 INFO:teuthology.orchestra.run.smithi111.stdout:generic/262 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:46:14.486 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/262 2023-03-29T12:46:14.487 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/262 2023-03-29T12:46:14.487 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:46:14.494 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:46:14.525 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:46:14.525 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/263) 2023-03-29T12:46:15.083 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:46:15.103 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:46:15.103 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:22.226 INFO:teuthology.orchestra.run.smithi111.stdout:generic/263 67s 2023-03-29T12:47:22.237 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/263 2023-03-29T12:47:22.238 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:22.243 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:22.309 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:22.310 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/264) 2023-03-29T12:47:22.962 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:22.974 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:22.974 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:23.266 INFO:teuthology.orchestra.run.smithi111.stdout:generic/264 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:23.274 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/264 2023-03-29T12:47:23.274 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/264 2023-03-29T12:47:23.274 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:23.279 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:23.299 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:23.299 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/265) 2023-03-29T12:47:23.682 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:23.693 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:23.694 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:23.997 INFO:teuthology.orchestra.run.smithi111.stdout:generic/265 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:24.005 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/265 2023-03-29T12:47:24.005 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/265 2023-03-29T12:47:24.005 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:24.010 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:24.030 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:24.030 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/266) 2023-03-29T12:47:24.407 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:24.419 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:24.419 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:24.721 INFO:teuthology.orchestra.run.smithi111.stdout:generic/266 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:24.729 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/266 2023-03-29T12:47:24.730 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/266 2023-03-29T12:47:24.730 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:24.734 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:24.753 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:24.753 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/267) 2023-03-29T12:47:25.132 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:25.143 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:25.143 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:25.448 INFO:teuthology.orchestra.run.smithi111.stdout:generic/267 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:25.457 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/267 2023-03-29T12:47:25.458 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/267 2023-03-29T12:47:25.458 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:25.461 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:25.481 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:25.481 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/268) 2023-03-29T12:47:25.859 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:25.871 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:25.871 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:26.167 INFO:teuthology.orchestra.run.smithi111.stdout:generic/268 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:26.175 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/268 2023-03-29T12:47:26.175 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/268 2023-03-29T12:47:26.176 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:26.181 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:26.200 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:26.200 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/269) 2023-03-29T12:47:26.586 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:26.598 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:26.598 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:26.868 INFO:teuthology.orchestra.run.smithi111.stdout:generic/269 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:26.876 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/269 2023-03-29T12:47:26.876 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/269 2023-03-29T12:47:26.876 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:26.881 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:26.900 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:26.900 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/270) 2023-03-29T12:47:27.283 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:27.295 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:27.295 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:27.569 INFO:teuthology.orchestra.run.smithi111.stdout:generic/270 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2023-03-29T12:47:27.577 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/270 2023-03-29T12:47:27.578 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/270 2023-03-29T12:47:27.578 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:27.583 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:27.602 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:27.602 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/271) 2023-03-29T12:47:27.981 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:27.993 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:27.993 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:28.523 INFO:teuthology.orchestra.run.smithi111.stdout:generic/271 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:28.531 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/271 2023-03-29T12:47:28.531 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/271 2023-03-29T12:47:28.531 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:28.536 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:28.556 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:28.556 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/272) 2023-03-29T12:47:28.935 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:28.947 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:28.947 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:29.251 INFO:teuthology.orchestra.run.smithi111.stdout:generic/272 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:29.259 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/272 2023-03-29T12:47:29.260 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/272 2023-03-29T12:47:29.260 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:29.265 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:29.286 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:29.286 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/273) 2023-03-29T12:47:29.665 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:29.677 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:29.677 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:29.963 INFO:teuthology.orchestra.run.smithi111.stdout:generic/273 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:29.972 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/273 2023-03-29T12:47:29.972 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/273 2023-03-29T12:47:29.972 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:29.977 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:29.996 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:29.996 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/274) 2023-03-29T12:47:30.379 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:30.392 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:30.392 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:30.659 INFO:teuthology.orchestra.run.smithi111.stdout:generic/274 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:30.667 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/274 2023-03-29T12:47:30.667 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/274 2023-03-29T12:47:30.667 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:30.672 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:30.691 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:30.691 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/275) 2023-03-29T12:47:31.074 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:31.086 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:31.086 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:31.362 INFO:teuthology.orchestra.run.smithi111.stdout:generic/275 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:31.370 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/275 2023-03-29T12:47:31.370 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/275 2023-03-29T12:47:31.370 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:31.375 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:31.394 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:31.395 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/276) 2023-03-29T12:47:31.782 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:31.792 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:31.793 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:32.089 INFO:teuthology.orchestra.run.smithi111.stdout:generic/276 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:32.097 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/276 2023-03-29T12:47:32.097 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/276 2023-03-29T12:47:32.097 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:32.102 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:32.122 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:32.122 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/277) 2023-03-29T12:47:32.500 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:32.512 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:32.512 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:32.794 INFO:teuthology.orchestra.run.smithi111.stdout:generic/277 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:32.802 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/277 2023-03-29T12:47:32.803 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/277 2023-03-29T12:47:32.803 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:32.808 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:32.827 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:32.827 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/278) 2023-03-29T12:47:33.208 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:33.220 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:33.220 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:33.579 INFO:teuthology.orchestra.run.smithi111.stdout:generic/278 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:33.587 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/278 2023-03-29T12:47:33.587 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/278 2023-03-29T12:47:33.587 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:33.592 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:33.611 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:33.612 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/279) 2023-03-29T12:47:33.991 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:34.001 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:34.002 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:34.416 INFO:teuthology.orchestra.run.smithi111.stdout:generic/279 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:34.424 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/279 2023-03-29T12:47:34.424 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/279 2023-03-29T12:47:34.425 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:34.430 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:34.449 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:34.450 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/280) 2023-03-29T12:47:34.833 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:34.845 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:34.845 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:35.376 INFO:teuthology.orchestra.run.smithi111.stdout:generic/280 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:35.385 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/280 2023-03-29T12:47:35.385 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/280 2023-03-29T12:47:35.385 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:35.390 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:35.409 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:35.409 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/281) 2023-03-29T12:47:35.792 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:35.803 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:35.803 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:36.120 INFO:teuthology.orchestra.run.smithi111.stdout:generic/281 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:36.128 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/281 2023-03-29T12:47:36.128 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/281 2023-03-29T12:47:36.128 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:36.133 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:36.153 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:36.154 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/282) 2023-03-29T12:47:36.533 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:36.544 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:36.544 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:36.858 INFO:teuthology.orchestra.run.smithi111.stdout:generic/282 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:36.866 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/282 2023-03-29T12:47:36.866 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/282 2023-03-29T12:47:36.866 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:36.872 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:36.892 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:36.893 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/283) 2023-03-29T12:47:37.273 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:37.284 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:37.284 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:37.583 INFO:teuthology.orchestra.run.smithi111.stdout:generic/283 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:37.591 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/283 2023-03-29T12:47:37.592 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/283 2023-03-29T12:47:37.592 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:37.597 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:37.617 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:37.618 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/284) 2023-03-29T12:47:38.001 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:38.012 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:38.013 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:38.281 INFO:teuthology.orchestra.run.smithi111.stdout:generic/284 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:47:38.289 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/284 2023-03-29T12:47:38.289 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/284 2023-03-29T12:47:38.289 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:38.294 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:38.314 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:38.314 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/285) 2023-03-29T12:47:38.690 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:38.701 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:38.702 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:41.161 INFO:teuthology.orchestra.run.smithi111.stdout:generic/285 3s 2023-03-29T12:47:41.172 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/285 2023-03-29T12:47:41.173 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:47:41.178 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:47:41.197 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:47:41.197 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/286) 2023-03-29T12:47:41.887 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:47:41.898 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:47:41.898 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:34.374 INFO:teuthology.orchestra.run.smithi111.stdout:generic/286 53s 2023-03-29T12:48:34.386 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/286 2023-03-29T12:48:34.387 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:34.392 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:34.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:34.414 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/287) 2023-03-29T12:48:35.173 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:35.192 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:35.192 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:35.505 INFO:teuthology.orchestra.run.smithi111.stdout:generic/287 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:48:35.513 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/287 2023-03-29T12:48:35.513 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/287 2023-03-29T12:48:35.513 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:35.519 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:35.538 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:35.538 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/288) 2023-03-29T12:48:35.922 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:35.934 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:35.934 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:36.205 INFO:teuthology.orchestra.run.smithi111.stdout:generic/288 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:48:36.213 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/288 2023-03-29T12:48:36.213 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/288 2023-03-29T12:48:36.213 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:36.218 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:36.237 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:36.238 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/289) 2023-03-29T12:48:36.621 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:36.633 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:36.633 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:36.905 INFO:teuthology.orchestra.run.smithi111.stdout:generic/289 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:48:36.913 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/289 2023-03-29T12:48:36.914 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/289 2023-03-29T12:48:36.914 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:36.919 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:36.938 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:36.938 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/290) 2023-03-29T12:48:37.323 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:37.334 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:37.334 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:37.607 INFO:teuthology.orchestra.run.smithi111.stdout:generic/290 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:48:37.615 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/290 2023-03-29T12:48:37.615 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/290 2023-03-29T12:48:37.615 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:37.621 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:37.642 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:37.642 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/291) 2023-03-29T12:48:38.023 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:38.034 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:38.034 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:38.290 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:48:38.294 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:48:38.312 INFO:teuthology.orchestra.run.smithi111.stdout:generic/291 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:48:38.320 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/291 2023-03-29T12:48:38.320 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/291 2023-03-29T12:48:38.320 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:38.325 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:38.345 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:38.346 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/292) 2023-03-29T12:48:38.375 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-03-29T12:48:38.378+0000 7f046aff9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 336823) UID: 0 2023-03-29T12:48:38.375 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T12:48:38.378+0000 7f7a699f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 336823) UID: 0 2023-03-29T12:48:38.376 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-03-29T12:48:38.378+0000 7f22b21f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 336823) UID: 0 2023-03-29T12:48:38.376 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-03-29T12:48:38.378+0000 7f93ce7f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 336823) UID: 0 2023-03-29T12:48:38.760 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:38.771 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:38.771 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:39.052 INFO:teuthology.orchestra.run.smithi111.stdout:generic/292 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:48:39.060 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/292 2023-03-29T12:48:39.060 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/292 2023-03-29T12:48:39.060 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:39.066 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:39.088 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:39.088 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/293) 2023-03-29T12:48:39.481 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:39.493 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:39.493 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:39.780 INFO:teuthology.orchestra.run.smithi111.stdout:generic/293 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:48:39.789 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/293 2023-03-29T12:48:39.789 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/293 2023-03-29T12:48:39.789 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:39.795 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:39.815 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:39.816 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/294) 2023-03-29T12:48:40.195 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:40.207 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:40.207 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:40.513 INFO:teuthology.orchestra.run.smithi111.stdout:generic/294 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:48:40.521 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/294 2023-03-29T12:48:40.522 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/294 2023-03-29T12:48:40.522 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:40.527 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:40.547 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:40.547 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/295) 2023-03-29T12:48:40.932 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:40.944 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:40.944 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:41.218 INFO:teuthology.orchestra.run.smithi111.stdout:generic/295 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:48:41.226 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/295 2023-03-29T12:48:41.227 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/295 2023-03-29T12:48:41.227 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:41.232 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:41.251 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:41.252 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/296) 2023-03-29T12:48:41.642 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:41.655 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:41.655 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:41.951 INFO:teuthology.orchestra.run.smithi111.stdout:generic/296 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:48:41.959 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/296 2023-03-29T12:48:41.960 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/296 2023-03-29T12:48:41.960 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:41.964 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:41.984 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:41.985 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/297) 2023-03-29T12:48:42.364 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:42.375 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:42.375 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:42.652 INFO:teuthology.orchestra.run.smithi111.stdout:generic/297 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:48:42.660 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/297 2023-03-29T12:48:42.660 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/297 2023-03-29T12:48:42.660 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:42.665 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:42.685 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:42.685 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/298) 2023-03-29T12:48:43.100 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:43.109 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:43.109 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:43.398 INFO:teuthology.orchestra.run.smithi111.stdout:generic/298 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:48:43.407 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/298 2023-03-29T12:48:43.407 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/298 2023-03-29T12:48:43.407 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:43.412 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:43.432 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:43.432 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/299) 2023-03-29T12:48:43.850 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:43.861 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:43.861 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:44.189 INFO:teuthology.orchestra.run.smithi111.stdout:generic/299 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:48:44.198 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/299 2023-03-29T12:48:44.198 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/299 2023-03-29T12:48:44.198 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:44.204 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:44.225 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:44.225 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/300) 2023-03-29T12:48:44.612 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:44.624 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:44.624 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:44.913 INFO:teuthology.orchestra.run.smithi111.stdout:generic/300 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:48:44.921 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/300 2023-03-29T12:48:44.921 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/300 2023-03-29T12:48:44.921 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:44.927 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:44.947 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:44.947 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/301) 2023-03-29T12:48:45.328 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:45.341 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:45.341 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:45.635 INFO:teuthology.orchestra.run.smithi111.stdout:generic/301 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:48:45.644 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/301 2023-03-29T12:48:45.645 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/301 2023-03-29T12:48:45.645 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:45.651 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:45.671 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:45.671 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/302) 2023-03-29T12:48:46.059 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:46.070 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:46.070 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:46.355 INFO:teuthology.orchestra.run.smithi111.stdout:generic/302 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:48:46.363 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/302 2023-03-29T12:48:46.363 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/302 2023-03-29T12:48:46.363 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:46.368 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:46.388 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:46.388 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/303) 2023-03-29T12:48:46.770 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:46.781 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:46.781 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:47.146 INFO:teuthology.orchestra.run.smithi111.stdout:generic/303 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:48:47.154 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/303 2023-03-29T12:48:47.154 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/303 2023-03-29T12:48:47.155 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:47.160 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:47.225 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:47.226 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/304) 2023-03-29T12:48:47.945 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:47.956 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:47.957 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:48.352 INFO:teuthology.orchestra.run.smithi111.stdout:generic/304 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2023-03-29T12:48:48.361 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/304 2023-03-29T12:48:48.361 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/304 2023-03-29T12:48:48.361 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:48.366 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:48.384 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:48.384 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/305) 2023-03-29T12:48:49.161 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:49.171 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:49.172 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:49.454 INFO:teuthology.orchestra.run.smithi111.stdout:generic/305 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:48:49.462 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/305 2023-03-29T12:48:49.462 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/305 2023-03-29T12:48:49.462 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:49.467 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:49.486 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:49.487 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/306) 2023-03-29T12:48:49.869 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:49.880 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:49.880 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:50.173 INFO:teuthology.orchestra.run.smithi111.stdout:generic/306 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:48:50.181 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/306 2023-03-29T12:48:50.181 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/306 2023-03-29T12:48:50.181 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:50.186 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:50.206 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:50.206 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/307) 2023-03-29T12:48:50.590 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:50.601 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:50.604 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:50.886 INFO:teuthology.orchestra.run.smithi111.stdout:generic/307 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:48:50.894 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/307 2023-03-29T12:48:50.895 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/307 2023-03-29T12:48:50.895 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:50.900 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:50.920 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:50.920 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/308) 2023-03-29T12:48:51.305 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:51.316 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:51.317 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:51.663 INFO:teuthology.orchestra.run.smithi111.stdout:generic/308 0s 2023-03-29T12:48:51.674 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/308 2023-03-29T12:48:51.675 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:51.680 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:51.701 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:51.702 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/309) 2023-03-29T12:48:52.088 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:52.099 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:52.099 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:54.850 INFO:teuthology.orchestra.run.smithi111.stdout:generic/309 2s 2023-03-29T12:48:54.862 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/309 2023-03-29T12:48:54.862 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:48:54.867 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:48:54.914 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:48:54.915 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/310) 2023-03-29T12:48:55.632 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:48:55.643 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:48:55.643 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:26.883 INFO:teuthology.orchestra.run.smithi111.stdout:generic/310 91s 2023-03-29T12:50:26.895 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/310 2023-03-29T12:50:26.895 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:50:26.901 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:26.947 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:50:26.947 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/311) 2023-03-29T12:50:27.901 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:50:27.986 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:50:27.986 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:28.610 INFO:teuthology.orchestra.run.smithi111.stdout:generic/311 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:50:28.618 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/311 2023-03-29T12:50:28.619 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/311 2023-03-29T12:50:28.619 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:50:28.624 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:28.647 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:50:28.647 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/312) 2023-03-29T12:50:29.035 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:50:29.046 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:50:29.047 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:29.374 INFO:teuthology.orchestra.run.smithi111.stdout:generic/312 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:50:29.381 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/312 2023-03-29T12:50:29.381 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/312 2023-03-29T12:50:29.381 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:50:29.387 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:29.406 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:50:29.407 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/313) 2023-03-29T12:50:29.794 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:50:29.805 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:50:29.805 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:36.354 INFO:teuthology.orchestra.run.smithi111.stdout:generic/313 7s 2023-03-29T12:50:36.367 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/313 2023-03-29T12:50:36.367 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:50:36.373 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:36.393 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:50:36.393 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/314) 2023-03-29T12:50:36.806 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:50:36.818 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:50:36.818 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:38.456 INFO:teuthology.orchestra.run.smithi111.stdout:generic/314 2s 2023-03-29T12:50:38.467 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/314 2023-03-29T12:50:38.467 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:50:38.472 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:38.493 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:50:38.493 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/315) 2023-03-29T12:50:38.881 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:50:38.893 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:50:38.894 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:39.873 INFO:teuthology.orchestra.run.smithi111.stdout:generic/315 1s 2023-03-29T12:50:39.885 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/315 2023-03-29T12:50:39.885 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:50:39.890 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:39.931 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:50:39.932 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/316) 2023-03-29T12:50:40.674 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:50:40.687 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:50:40.688 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:42.389 INFO:teuthology.orchestra.run.smithi111.stdout:generic/316 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-03-29T12:50:42.397 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/316 2023-03-29T12:50:42.397 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/316 2023-03-29T12:50:42.397 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:50:42.402 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:42.459 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:50:42.460 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/317) 2023-03-29T12:50:43.179 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:50:43.190 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:50:43.190 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:43.513 INFO:teuthology.orchestra.run.smithi111.stdout:generic/317 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:50:43.521 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/317 2023-03-29T12:50:43.521 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/317 2023-03-29T12:50:43.522 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:50:43.527 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:43.546 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:50:43.547 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/318) 2023-03-29T12:50:43.942 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:50:43.953 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:50:43.953 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:44.244 INFO:teuthology.orchestra.run.smithi111.stdout:generic/318 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:50:44.252 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/318 2023-03-29T12:50:44.252 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/318 2023-03-29T12:50:44.253 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:50:44.258 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:44.278 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:50:44.278 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/319) 2023-03-29T12:50:44.665 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:50:44.676 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:50:44.676 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:45.002 INFO:teuthology.orchestra.run.smithi111.stdout:generic/319 [not run] ACLs not supported by this filesystem type: ceph-fuse 2023-03-29T12:50:45.010 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/319 2023-03-29T12:50:45.010 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/319 2023-03-29T12:50:45.010 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:50:45.015 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:45.033 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:50:45.033 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/320) 2023-03-29T12:50:45.431 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:50:45.442 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:50:45.443 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:45.745 INFO:teuthology.orchestra.run.smithi111.stdout:generic/320 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:50:45.753 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/320 2023-03-29T12:50:45.754 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/320 2023-03-29T12:50:45.754 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:50:45.762 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:45.782 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:50:45.783 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/321) 2023-03-29T12:50:46.165 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:50:46.175 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:50:46.176 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:46.632 INFO:teuthology.orchestra.run.smithi111.stdout:generic/321 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:50:46.640 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/321 2023-03-29T12:50:46.641 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/321 2023-03-29T12:50:46.641 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:50:46.646 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:46.665 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:50:46.666 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/322) 2023-03-29T12:50:47.052 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:50:47.064 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:50:47.064 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:47.379 INFO:teuthology.orchestra.run.smithi111.stdout:generic/322 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:50:47.387 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/322 2023-03-29T12:50:47.387 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/322 2023-03-29T12:50:47.387 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:50:47.393 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:50:47.413 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:50:47.414 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/323) 2023-03-29T12:50:47.804 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:50:47.817 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:50:47.817 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:49.653 INFO:teuthology.orchestra.run.smithi111.stdout:generic/323 122s 2023-03-29T12:52:49.690 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/323 2023-03-29T12:52:49.690 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:52:49.695 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:49.775 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:52:49.775 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/324) 2023-03-29T12:52:50.566 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:52:50.579 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:52:50.579 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:50.861 INFO:teuthology.orchestra.run.smithi111.stdout:generic/324 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:52:50.869 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/324 2023-03-29T12:52:50.869 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/324 2023-03-29T12:52:50.869 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:52:50.874 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:50.895 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:52:50.895 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/325) 2023-03-29T12:52:51.275 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:52:51.287 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:52:51.287 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:51.732 INFO:teuthology.orchestra.run.smithi111.stdout:generic/325 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:52:51.741 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/325 2023-03-29T12:52:51.742 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/325 2023-03-29T12:52:51.742 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:52:51.748 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:51.767 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:52:51.767 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/326) 2023-03-29T12:52:52.147 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:52:52.158 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:52:52.158 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:52.457 INFO:teuthology.orchestra.run.smithi111.stdout:generic/326 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:52:52.465 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/326 2023-03-29T12:52:52.465 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/326 2023-03-29T12:52:52.466 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:52:52.471 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:52.490 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:52:52.490 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/327) 2023-03-29T12:52:52.870 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:52:52.881 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:52:52.881 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:53.243 INFO:teuthology.orchestra.run.smithi111.stdout:generic/327 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:52:53.251 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/327 2023-03-29T12:52:53.251 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/327 2023-03-29T12:52:53.251 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:52:53.256 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:53.277 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:52:53.277 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/328) 2023-03-29T12:52:53.655 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:52:53.665 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:52:53.666 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:54.277 INFO:teuthology.orchestra.run.smithi111.stdout:generic/328 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:52:54.286 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/328 2023-03-29T12:52:54.286 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/328 2023-03-29T12:52:54.286 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:52:54.291 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:54.309 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:52:54.310 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/329) 2023-03-29T12:52:54.686 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:52:54.697 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:52:54.697 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:56.380 INFO:teuthology.orchestra.run.smithi111.stdout:generic/329 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:52:56.389 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/329 2023-03-29T12:52:56.389 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/329 2023-03-29T12:52:56.389 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:52:56.395 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:56.414 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:52:56.414 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/330) 2023-03-29T12:52:56.820 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:52:56.831 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:52:56.831 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:57.122 INFO:teuthology.orchestra.run.smithi111.stdout:generic/330 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:52:57.130 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/330 2023-03-29T12:52:57.131 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/330 2023-03-29T12:52:57.131 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:52:57.136 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:57.156 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:52:57.157 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/331) 2023-03-29T12:52:57.562 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:52:57.575 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:52:57.575 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:57.897 INFO:teuthology.orchestra.run.smithi111.stdout:generic/331 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:52:57.905 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/331 2023-03-29T12:52:57.906 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/331 2023-03-29T12:52:57.906 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:52:57.911 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:57.932 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:52:57.932 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/332) 2023-03-29T12:52:58.350 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:52:58.363 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:52:58.364 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:58.685 INFO:teuthology.orchestra.run.smithi111.stdout:generic/332 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:52:58.695 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/332 2023-03-29T12:52:58.695 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/332 2023-03-29T12:52:58.695 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:52:58.703 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:58.724 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:52:58.724 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/333) 2023-03-29T12:52:59.108 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:52:59.119 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:52:59.119 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:59.393 INFO:teuthology.orchestra.run.smithi111.stdout:generic/333 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:52:59.401 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/333 2023-03-29T12:52:59.401 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/333 2023-03-29T12:52:59.401 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:52:59.406 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:52:59.425 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:52:59.426 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/334) 2023-03-29T12:52:59.808 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:52:59.819 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:52:59.820 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:00.097 INFO:teuthology.orchestra.run.smithi111.stdout:generic/334 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:00.105 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/334 2023-03-29T12:53:00.105 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/334 2023-03-29T12:53:00.106 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:00.111 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:00.130 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:00.131 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/335) 2023-03-29T12:53:00.511 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:00.522 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:00.522 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:00.817 INFO:teuthology.orchestra.run.smithi111.stdout:generic/335 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:00.825 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/335 2023-03-29T12:53:00.826 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/335 2023-03-29T12:53:00.826 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:00.831 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:00.849 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:00.849 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/336) 2023-03-29T12:53:01.231 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:01.242 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:01.242 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:01.538 INFO:teuthology.orchestra.run.smithi111.stdout:generic/336 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:01.546 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/336 2023-03-29T12:53:01.546 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/336 2023-03-29T12:53:01.546 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:01.552 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:01.570 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:01.571 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/337) 2023-03-29T12:53:01.952 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:01.963 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:01.963 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:02.251 INFO:teuthology.orchestra.run.smithi111.stdout:generic/337 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:02.259 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/337 2023-03-29T12:53:02.259 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/337 2023-03-29T12:53:02.259 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:02.265 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:02.284 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:02.284 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/338) 2023-03-29T12:53:02.671 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:02.682 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:02.682 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:02.987 INFO:teuthology.orchestra.run.smithi111.stdout:generic/338 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:02.995 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/338 2023-03-29T12:53:02.995 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/338 2023-03-29T12:53:02.995 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:03.000 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:03.020 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:03.021 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/339) 2023-03-29T12:53:03.407 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:03.418 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:03.419 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:03.693 INFO:teuthology.orchestra.run.smithi111.stdout:generic/339 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:03.701 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/339 2023-03-29T12:53:03.701 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/339 2023-03-29T12:53:03.701 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:03.707 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:03.725 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:03.726 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/340) 2023-03-29T12:53:04.109 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:04.119 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:04.119 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:04.385 INFO:teuthology.orchestra.run.smithi111.stdout:generic/340 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:04.393 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/340 2023-03-29T12:53:04.393 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/340 2023-03-29T12:53:04.393 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:04.398 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:04.417 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:04.417 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/341) 2023-03-29T12:53:04.806 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:04.817 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:04.818 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:05.120 INFO:teuthology.orchestra.run.smithi111.stdout:generic/341 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:05.128 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/341 2023-03-29T12:53:05.128 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/341 2023-03-29T12:53:05.128 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:05.133 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:05.152 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:05.152 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/342) 2023-03-29T12:53:05.538 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:05.549 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:05.549 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:05.863 INFO:teuthology.orchestra.run.smithi111.stdout:generic/342 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:05.871 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/342 2023-03-29T12:53:05.871 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/342 2023-03-29T12:53:05.871 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:05.877 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:05.896 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:05.896 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/343) 2023-03-29T12:53:06.283 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:06.294 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:06.294 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:06.592 INFO:teuthology.orchestra.run.smithi111.stdout:generic/343 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:06.600 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/343 2023-03-29T12:53:06.600 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/343 2023-03-29T12:53:06.600 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:06.605 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:06.624 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:06.624 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/344) 2023-03-29T12:53:07.003 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:07.014 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:07.014 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:07.281 INFO:teuthology.orchestra.run.smithi111.stdout:generic/344 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:07.289 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/344 2023-03-29T12:53:07.289 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/344 2023-03-29T12:53:07.289 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:07.294 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:07.312 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:07.313 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/345) 2023-03-29T12:53:07.697 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:07.709 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:07.709 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:07.983 INFO:teuthology.orchestra.run.smithi111.stdout:generic/345 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:07.991 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/345 2023-03-29T12:53:07.991 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/345 2023-03-29T12:53:07.992 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:07.997 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:08.016 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:08.016 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/346) 2023-03-29T12:53:08.395 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:08.407 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:08.407 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:08.679 INFO:teuthology.orchestra.run.smithi111.stdout:generic/346 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:08.687 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/346 2023-03-29T12:53:08.688 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/346 2023-03-29T12:53:08.688 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:08.693 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:08.712 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:08.713 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/347) 2023-03-29T12:53:09.096 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:09.107 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:09.107 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:09.452 INFO:teuthology.orchestra.run.smithi111.stdout:generic/347 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:09.460 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/347 2023-03-29T12:53:09.460 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/347 2023-03-29T12:53:09.460 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:09.465 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:09.485 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:09.485 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/348) 2023-03-29T12:53:09.865 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:09.876 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:09.877 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:10.172 INFO:teuthology.orchestra.run.smithi111.stdout:generic/348 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:10.181 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/348 2023-03-29T12:53:10.181 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/348 2023-03-29T12:53:10.181 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:10.186 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:10.205 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:10.205 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/349) 2023-03-29T12:53:10.590 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:10.601 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:10.602 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:12.181 INFO:teuthology.orchestra.run.smithi111.stdout:generic/349 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-03-29T12:53:12.189 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/349 2023-03-29T12:53:12.190 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/349 2023-03-29T12:53:12.190 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:12.195 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:12.214 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:12.215 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/350) 2023-03-29T12:53:12.602 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:12.613 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:12.614 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:14.870 INFO:teuthology.orchestra.run.smithi111.stdout:generic/350 2s 2023-03-29T12:53:14.881 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/350 2023-03-29T12:53:14.881 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:14.887 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:14.905 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:14.905 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/351) 2023-03-29T12:53:15.286 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:15.297 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:15.297 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:16.898 INFO:teuthology.orchestra.run.smithi111.stdout:generic/351 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2023-03-29T12:53:16.907 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/351 2023-03-29T12:53:16.907 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/351 2023-03-29T12:53:16.907 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:16.912 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:16.937 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:16.937 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/352) 2023-03-29T12:53:17.674 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:17.685 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:17.686 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:19.543 INFO:teuthology.orchestra.run.smithi111.stdout:generic/352 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:19.551 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/352 2023-03-29T12:53:19.551 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/352 2023-03-29T12:53:19.552 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:19.557 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:19.575 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:19.575 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/353) 2023-03-29T12:53:19.958 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:19.970 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:19.970 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:20.242 INFO:teuthology.orchestra.run.smithi111.stdout:generic/353 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:20.250 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/353 2023-03-29T12:53:20.250 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/353 2023-03-29T12:53:20.250 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:20.256 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:20.276 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:20.276 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/354) 2023-03-29T12:53:20.670 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:20.681 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:20.681 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:20.953 INFO:teuthology.orchestra.run.smithi111.stdout:generic/354 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:20.961 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/354 2023-03-29T12:53:20.961 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/354 2023-03-29T12:53:20.961 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:20.966 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:20.985 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:20.985 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/355) 2023-03-29T12:53:21.365 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:21.377 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:21.377 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:22.498 INFO:teuthology.orchestra.run.smithi111.stdout:generic/355 1s 2023-03-29T12:53:22.510 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/355 2023-03-29T12:53:22.510 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:22.515 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:22.534 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:22.534 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/356) 2023-03-29T12:53:23.253 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:23.264 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:23.264 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:23.537 INFO:teuthology.orchestra.run.smithi111.stdout:generic/356 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:23.545 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/356 2023-03-29T12:53:23.546 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/356 2023-03-29T12:53:23.546 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:23.550 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:23.569 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:23.569 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/357) 2023-03-29T12:53:23.957 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:23.968 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:23.968 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:24.245 INFO:teuthology.orchestra.run.smithi111.stdout:generic/357 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:24.254 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/357 2023-03-29T12:53:24.254 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/357 2023-03-29T12:53:24.254 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:24.260 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:24.279 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:24.279 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/358) 2023-03-29T12:53:24.664 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:24.675 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:24.676 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:24.953 INFO:teuthology.orchestra.run.smithi111.stdout:generic/358 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:24.961 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/358 2023-03-29T12:53:24.962 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/358 2023-03-29T12:53:24.962 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:24.967 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:24.987 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:24.987 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/359) 2023-03-29T12:53:25.372 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:25.384 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:25.384 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:25.659 INFO:teuthology.orchestra.run.smithi111.stdout:generic/359 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:25.668 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/359 2023-03-29T12:53:25.668 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/359 2023-03-29T12:53:25.668 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:25.673 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:25.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:25.693 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/360) 2023-03-29T12:53:26.080 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:26.092 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:26.092 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:26.478 INFO:teuthology.orchestra.run.smithi111.stdout:generic/360 0s 2023-03-29T12:53:26.489 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/360 2023-03-29T12:53:26.489 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:26.494 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:26.514 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:26.514 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/361) 2023-03-29T12:53:26.902 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:26.913 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:26.913 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:27.228 INFO:teuthology.orchestra.run.smithi111.stdout:generic/361 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:27.236 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/361 2023-03-29T12:53:27.237 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/361 2023-03-29T12:53:27.237 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:27.242 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:27.262 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:27.262 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/362) 2023-03-29T12:53:27.645 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:27.656 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:27.657 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:27.924 INFO:teuthology.orchestra.run.smithi111.stdout:generic/362 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:27.932 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/362 2023-03-29T12:53:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/362 2023-03-29T12:53:27.933 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:27.938 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:27.957 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:27.957 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/363) 2023-03-29T12:53:28.342 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:28.353 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:28.353 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:28.629 INFO:teuthology.orchestra.run.smithi111.stdout:generic/363 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:28.637 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/363 2023-03-29T12:53:28.637 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/363 2023-03-29T12:53:28.638 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:28.642 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:28.661 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:28.661 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/364) 2023-03-29T12:53:28.742 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:53:28.746 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:53:28.838 INFO:tasks.ceph.mds.a.smithi111.stderr:2023-03-29T12:53:28.842+0000 7f3a763f9700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 445988) UID: 0 2023-03-29T12:53:28.839 INFO:tasks.ceph.mds.c.smithi111.stderr:2023-03-29T12:53:28.842+0000 7f155ddf9700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 445988) UID: 0 2023-03-29T12:53:29.051 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:29.062 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:29.063 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:29.344 INFO:teuthology.orchestra.run.smithi111.stdout:generic/364 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:29.352 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/364 2023-03-29T12:53:29.353 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/364 2023-03-29T12:53:29.353 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:29.357 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:29.377 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:29.377 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/365) 2023-03-29T12:53:29.793 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:29.804 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:29.804 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:30.079 INFO:teuthology.orchestra.run.smithi111.stdout:generic/365 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:30.087 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/365 2023-03-29T12:53:30.088 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/365 2023-03-29T12:53:30.088 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:30.093 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:30.112 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:30.112 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/366) 2023-03-29T12:53:30.495 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:30.507 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:30.507 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:30.774 INFO:teuthology.orchestra.run.smithi111.stdout:generic/366 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:30.782 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/366 2023-03-29T12:53:30.782 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/366 2023-03-29T12:53:30.782 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:30.788 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:30.808 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:30.808 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/367) 2023-03-29T12:53:31.205 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:31.216 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:31.216 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:31.492 INFO:teuthology.orchestra.run.smithi111.stdout:generic/367 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:31.500 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/367 2023-03-29T12:53:31.500 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/367 2023-03-29T12:53:31.500 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:31.506 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:31.525 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:31.525 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/368) 2023-03-29T12:53:31.914 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:31.925 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:31.926 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:32.198 INFO:teuthology.orchestra.run.smithi111.stdout:generic/368 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:32.205 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/368 2023-03-29T12:53:32.205 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/368 2023-03-29T12:53:32.206 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:32.211 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:32.231 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:32.231 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/369) 2023-03-29T12:53:32.624 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:32.636 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:32.636 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:32.925 INFO:teuthology.orchestra.run.smithi111.stdout:generic/369 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:32.933 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/369 2023-03-29T12:53:32.934 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/369 2023-03-29T12:53:32.934 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:32.940 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:32.961 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:32.961 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/370) 2023-03-29T12:53:33.350 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:33.361 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:33.362 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:33.644 INFO:teuthology.orchestra.run.smithi111.stdout:generic/370 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:33.652 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/370 2023-03-29T12:53:33.652 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/370 2023-03-29T12:53:33.652 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:33.657 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:33.676 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:33.676 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/371) 2023-03-29T12:53:34.060 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:34.071 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:34.071 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:34.352 INFO:teuthology.orchestra.run.smithi111.stdout:generic/371 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:34.360 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/371 2023-03-29T12:53:34.360 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/371 2023-03-29T12:53:34.360 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:34.365 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:34.385 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:34.385 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/372) 2023-03-29T12:53:34.775 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:34.786 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:34.786 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:35.067 INFO:teuthology.orchestra.run.smithi111.stdout:generic/372 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:35.076 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/372 2023-03-29T12:53:35.076 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/372 2023-03-29T12:53:35.076 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:35.082 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:35.101 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:35.102 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/373) 2023-03-29T12:53:35.496 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:35.507 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:35.507 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:35.807 INFO:teuthology.orchestra.run.smithi111.stdout:generic/373 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:35.815 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/373 2023-03-29T12:53:35.815 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/373 2023-03-29T12:53:35.815 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:35.820 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:35.840 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:35.841 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/374) 2023-03-29T12:53:36.224 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:36.235 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:36.235 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:36.535 INFO:teuthology.orchestra.run.smithi111.stdout:generic/374 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:36.543 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/374 2023-03-29T12:53:36.543 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/374 2023-03-29T12:53:36.544 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:36.549 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:36.568 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:36.569 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/375) 2023-03-29T12:53:36.954 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:36.965 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:36.965 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:37.298 INFO:teuthology.orchestra.run.smithi111.stdout:generic/375 [not run] ACLs not supported by this filesystem type: ceph-fuse 2023-03-29T12:53:37.306 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/375 2023-03-29T12:53:37.306 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/375 2023-03-29T12:53:37.306 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:37.311 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:37.330 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:37.330 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/376) 2023-03-29T12:53:37.721 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:37.731 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:37.732 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:38.070 INFO:teuthology.orchestra.run.smithi111.stdout:generic/376 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:38.078 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/376 2023-03-29T12:53:38.078 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/376 2023-03-29T12:53:38.078 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:38.083 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:38.102 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:38.103 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/377) 2023-03-29T12:53:38.491 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:38.496 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:38.497 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:38.772 INFO:teuthology.orchestra.run.smithi111.stdout:generic/377 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:38.781 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/377 2023-03-29T12:53:38.781 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/377 2023-03-29T12:53:38.781 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:38.785 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:38.805 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:38.805 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/378) 2023-03-29T12:53:39.188 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:39.199 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:39.199 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:39.821 INFO:teuthology.orchestra.run.smithi111.stdout:generic/378 0s 2023-03-29T12:53:39.833 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/378 2023-03-29T12:53:39.833 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:39.838 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:39.858 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:39.858 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/379) 2023-03-29T12:53:40.671 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:40.683 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:40.683 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:40.987 INFO:teuthology.orchestra.run.smithi111.stdout:generic/379 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:40.996 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/379 2023-03-29T12:53:40.996 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/379 2023-03-29T12:53:40.996 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:41.001 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:41.022 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:41.022 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/380) 2023-03-29T12:53:41.408 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:41.419 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:41.419 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:41.698 INFO:teuthology.orchestra.run.smithi111.stdout:generic/380 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:41.706 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/380 2023-03-29T12:53:41.706 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/380 2023-03-29T12:53:41.706 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:41.712 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:41.732 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:41.732 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/381) 2023-03-29T12:53:42.118 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:42.129 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:42.129 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:42.402 INFO:teuthology.orchestra.run.smithi111.stdout:generic/381 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:42.410 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/381 2023-03-29T12:53:42.411 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/381 2023-03-29T12:53:42.411 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:42.416 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:42.435 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:42.436 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/382) 2023-03-29T12:53:42.821 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:42.832 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:42.832 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:43.113 INFO:teuthology.orchestra.run.smithi111.stdout:generic/382 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:43.122 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/382 2023-03-29T12:53:43.122 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/382 2023-03-29T12:53:43.122 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:43.127 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:43.147 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:43.147 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/383) 2023-03-29T12:53:43.533 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:43.546 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:43.546 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:43.868 INFO:teuthology.orchestra.run.smithi111.stdout:generic/383 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:43.877 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/383 2023-03-29T12:53:43.877 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/383 2023-03-29T12:53:43.877 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:43.884 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:43.904 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:43.904 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/384) 2023-03-29T12:53:44.292 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:44.304 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:44.304 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:44.588 INFO:teuthology.orchestra.run.smithi111.stdout:generic/384 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2023-03-29T12:53:44.597 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/384 2023-03-29T12:53:44.597 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/384 2023-03-29T12:53:44.597 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:44.606 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:44.625 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:44.625 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/385) 2023-03-29T12:53:45.046 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:45.057 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:45.057 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:45.361 INFO:teuthology.orchestra.run.smithi111.stdout:generic/385 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:45.369 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/385 2023-03-29T12:53:45.369 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/385 2023-03-29T12:53:45.369 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:45.375 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:45.395 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:45.395 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/386) 2023-03-29T12:53:45.775 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:45.786 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:45.786 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:46.126 INFO:teuthology.orchestra.run.smithi111.stdout:generic/386 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2023-03-29T12:53:46.135 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/386 2023-03-29T12:53:46.135 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/386 2023-03-29T12:53:46.135 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:46.140 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:46.160 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:46.160 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/387) 2023-03-29T12:53:46.547 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:46.559 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:46.560 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:46.838 INFO:teuthology.orchestra.run.smithi111.stdout:generic/387 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:46.847 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/387 2023-03-29T12:53:46.848 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/387 2023-03-29T12:53:46.848 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:46.854 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:46.874 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:46.874 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/388) 2023-03-29T12:53:47.255 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:47.266 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:47.266 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:47.558 INFO:teuthology.orchestra.run.smithi111.stdout:generic/388 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:47.566 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/388 2023-03-29T12:53:47.566 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/388 2023-03-29T12:53:47.566 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:47.571 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:47.592 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:47.592 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/389) 2023-03-29T12:53:47.985 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:47.996 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:47.996 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:48.294 INFO:teuthology.orchestra.run.smithi111.stdout:generic/389 [not run] O_TMPFILE is not supported 2023-03-29T12:53:48.303 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/389 2023-03-29T12:53:48.303 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/389 2023-03-29T12:53:48.303 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:48.308 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:48.328 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:48.328 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/390) 2023-03-29T12:53:48.714 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:48.726 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:48.726 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:49.051 INFO:teuthology.orchestra.run.smithi111.stdout:generic/390 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:53:49.060 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/390 2023-03-29T12:53:49.060 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/390 2023-03-29T12:53:49.060 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:53:49.065 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:53:49.085 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:53:49.085 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/391) 2023-03-29T12:53:49.472 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:53:49.483 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:53:49.484 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:35.956 INFO:teuthology.orchestra.run.smithi111.stdout:generic/391 46s 2023-03-29T12:54:36.097 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/391 2023-03-29T12:54:36.097 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:36.102 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:36.400 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:36.401 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/392) 2023-03-29T12:54:39.115 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:39.139 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:39.139 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:39.640 INFO:teuthology.orchestra.run.smithi111.stdout:generic/392 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:54:39.648 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/392 2023-03-29T12:54:39.648 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/392 2023-03-29T12:54:39.648 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:39.654 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:39.674 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:39.674 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/393) 2023-03-29T12:54:40.068 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:40.080 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:40.080 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:40.379 INFO:teuthology.orchestra.run.smithi111.stdout:generic/393 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:54:40.385 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/393 2023-03-29T12:54:40.386 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/393 2023-03-29T12:54:40.386 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:40.391 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:40.412 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:40.412 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/394) 2023-03-29T12:54:40.806 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:40.817 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:40.818 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:41.208 INFO:teuthology.orchestra.run.smithi111.stdout:generic/394 1s 2023-03-29T12:54:41.219 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/394 2023-03-29T12:54:41.220 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:41.225 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:41.251 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:41.251 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/395) 2023-03-29T12:54:42.211 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:42.231 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:42.232 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:42.705 INFO:teuthology.orchestra.run.smithi111.stdout:generic/395 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:54:42.714 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/395 2023-03-29T12:54:42.714 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/395 2023-03-29T12:54:42.714 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:42.719 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:42.738 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:42.739 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/396) 2023-03-29T12:54:43.129 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:43.141 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:43.141 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:43.501 INFO:teuthology.orchestra.run.smithi111.stdout:generic/396 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:54:43.510 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/396 2023-03-29T12:54:43.510 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/396 2023-03-29T12:54:43.510 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:43.516 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:43.537 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:43.538 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/397) 2023-03-29T12:54:43.941 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:43.952 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:43.952 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:44.392 INFO:teuthology.orchestra.run.smithi111.stdout:generic/397 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:54:44.400 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/397 2023-03-29T12:54:44.400 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/397 2023-03-29T12:54:44.400 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:44.405 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:44.426 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:44.427 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/398) 2023-03-29T12:54:44.828 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:44.838 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:44.839 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:45.223 INFO:teuthology.orchestra.run.smithi111.stdout:generic/398 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:54:45.232 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/398 2023-03-29T12:54:45.232 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/398 2023-03-29T12:54:45.232 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:45.237 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:45.259 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:45.260 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/399) 2023-03-29T12:54:45.640 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:45.652 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:45.652 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:46.013 INFO:teuthology.orchestra.run.smithi111.stdout:generic/399 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:54:46.021 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/399 2023-03-29T12:54:46.021 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/399 2023-03-29T12:54:46.022 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:46.026 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:46.046 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:46.047 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/400) 2023-03-29T12:54:46.438 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:46.450 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:46.450 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:46.730 INFO:teuthology.orchestra.run.smithi111.stdout:generic/400 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2023-03-29T12:54:46.738 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/400 2023-03-29T12:54:46.739 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/400 2023-03-29T12:54:46.739 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:46.744 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:46.764 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:46.765 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/401) 2023-03-29T12:54:47.157 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:47.168 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:47.168 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:47.438 INFO:teuthology.orchestra.run.smithi111.stdout:generic/401 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:54:47.446 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/401 2023-03-29T12:54:47.446 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/401 2023-03-29T12:54:47.446 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:47.452 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:47.472 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:47.472 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/402) 2023-03-29T12:54:47.860 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:47.871 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:47.871 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:48.177 INFO:teuthology.orchestra.run.smithi111.stdout:generic/402 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:54:48.185 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/402 2023-03-29T12:54:48.185 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/402 2023-03-29T12:54:48.185 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:48.190 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:48.210 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:48.211 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/403) 2023-03-29T12:54:48.594 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:48.605 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:48.605 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:48.874 INFO:teuthology.orchestra.run.smithi111.stdout:generic/403 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:54:48.883 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/403 2023-03-29T12:54:48.883 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/403 2023-03-29T12:54:48.883 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:48.888 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:48.908 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:48.908 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/404) 2023-03-29T12:54:49.314 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:49.325 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:49.325 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:50.891 INFO:teuthology.orchestra.run.smithi111.stdout:generic/404 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2023-03-29T12:54:50.899 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/404 2023-03-29T12:54:50.899 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/404 2023-03-29T12:54:50.899 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:50.904 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:50.924 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:50.924 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/405) 2023-03-29T12:54:51.682 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:51.693 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:51.693 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:52.381 INFO:teuthology.orchestra.run.smithi111.stdout:generic/405 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:54:52.389 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/405 2023-03-29T12:54:52.389 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/405 2023-03-29T12:54:52.390 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:52.395 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:52.414 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:52.415 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/406) 2023-03-29T12:54:52.803 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:52.814 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:52.815 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:53.101 INFO:teuthology.orchestra.run.smithi111.stdout:generic/406 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:54:53.109 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/406 2023-03-29T12:54:53.110 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/406 2023-03-29T12:54:53.110 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:53.115 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:53.137 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:53.137 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/407) 2023-03-29T12:54:53.526 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:53.537 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:53.537 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:53.929 INFO:teuthology.orchestra.run.smithi111.stdout:generic/407 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:54:53.937 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/407 2023-03-29T12:54:53.938 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/407 2023-03-29T12:54:53.938 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:53.943 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:54.014 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:54.014 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/408) 2023-03-29T12:54:54.759 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:54.770 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:54.771 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:55.166 INFO:teuthology.orchestra.run.smithi111.stdout:generic/408 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2023-03-29T12:54:55.174 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/408 2023-03-29T12:54:55.175 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/408 2023-03-29T12:54:55.175 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:55.180 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:55.198 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:55.199 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/409) 2023-03-29T12:54:55.976 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:55.988 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:55.988 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:56.279 INFO:teuthology.orchestra.run.smithi111.stdout:generic/409 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:54:56.287 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/409 2023-03-29T12:54:56.287 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/409 2023-03-29T12:54:56.287 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:56.293 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:56.313 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:56.313 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/410) 2023-03-29T12:54:56.703 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:56.714 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:56.714 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:57.012 INFO:teuthology.orchestra.run.smithi111.stdout:generic/410 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:54:57.020 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/410 2023-03-29T12:54:57.020 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/410 2023-03-29T12:54:57.020 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:57.025 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:57.047 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:57.047 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/411) 2023-03-29T12:54:57.428 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:57.439 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:57.439 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:57.724 INFO:teuthology.orchestra.run.smithi111.stdout:generic/411 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:54:57.732 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/411 2023-03-29T12:54:57.732 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/411 2023-03-29T12:54:57.732 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:57.737 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:57.757 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:57.758 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/412) 2023-03-29T12:54:58.143 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:58.155 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:58.156 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:58.430 INFO:teuthology.orchestra.run.smithi111.stdout:generic/412 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:54:58.438 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/412 2023-03-29T12:54:58.438 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/412 2023-03-29T12:54:58.439 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:58.444 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:58.464 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:58.464 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/413) 2023-03-29T12:54:58.851 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:58.862 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:58.862 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:59.145 INFO:teuthology.orchestra.run.smithi111.stdout:generic/413 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:54:59.154 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/413 2023-03-29T12:54:59.154 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/413 2023-03-29T12:54:59.154 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:59.159 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:59.180 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:59.180 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/414) 2023-03-29T12:54:59.579 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:54:59.590 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:54:59.590 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:59.880 INFO:teuthology.orchestra.run.smithi111.stdout:generic/414 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:54:59.888 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/414 2023-03-29T12:54:59.889 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/414 2023-03-29T12:54:59.889 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:54:59.894 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:54:59.913 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:54:59.913 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/415) 2023-03-29T12:55:00.297 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:55:00.308 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:55:00.308 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:00.589 INFO:teuthology.orchestra.run.smithi111.stdout:generic/415 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:55:00.597 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/415 2023-03-29T12:55:00.597 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/415 2023-03-29T12:55:00.597 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:55:00.602 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:00.622 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:55:00.622 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/416) 2023-03-29T12:55:01.005 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:55:01.016 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:55:01.016 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:01.291 INFO:teuthology.orchestra.run.smithi111.stdout:generic/416 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:55:01.299 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/416 2023-03-29T12:55:01.299 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/416 2023-03-29T12:55:01.299 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:55:01.305 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:01.326 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:55:01.326 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/417) 2023-03-29T12:55:01.710 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:55:01.722 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:55:01.722 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:01.997 INFO:teuthology.orchestra.run.smithi111.stdout:generic/417 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:55:02.005 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/417 2023-03-29T12:55:02.005 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/417 2023-03-29T12:55:02.005 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:55:02.011 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:02.031 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:55:02.032 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/418) 2023-03-29T12:55:02.411 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:55:02.422 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:55:02.423 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:02.755 INFO:teuthology.orchestra.run.smithi111.stdout:generic/418 [not run] require ceph-fuse to be valid block disk 2023-03-29T12:55:02.763 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/418 2023-03-29T12:55:02.764 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/418 2023-03-29T12:55:02.764 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:55:02.769 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:02.789 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:55:02.789 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/419) 2023-03-29T12:55:03.182 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:55:03.193 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:55:03.193 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:03.542 INFO:teuthology.orchestra.run.smithi111.stdout:generic/419 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:55:03.550 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/419 2023-03-29T12:55:03.551 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/419 2023-03-29T12:55:03.551 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:55:03.557 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:03.579 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:55:03.579 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/420) 2023-03-29T12:55:03.958 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:55:03.969 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:55:03.970 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:06.572 INFO:teuthology.orchestra.run.smithi111.stdout:generic/420 2s 2023-03-29T12:55:06.584 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/420 2023-03-29T12:55:06.584 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:55:06.590 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:06.609 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:55:06.609 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/421) 2023-03-29T12:55:07.345 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:55:07.356 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:55:07.356 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:07.986 INFO:teuthology.orchestra.run.smithi111.stdout:generic/421 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:55:07.994 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/421 2023-03-29T12:55:07.995 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/421 2023-03-29T12:55:07.995 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:55:08.000 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:08.019 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:55:08.020 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/422) 2023-03-29T12:55:08.403 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:55:08.414 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:55:08.415 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:08.694 INFO:teuthology.orchestra.run.smithi111.stdout:generic/422 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:55:08.702 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/422 2023-03-29T12:55:08.702 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/422 2023-03-29T12:55:08.702 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:55:08.707 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:08.727 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:55:08.727 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/423) 2023-03-29T12:55:09.122 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:55:09.134 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:55:09.134 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:09.707 INFO:teuthology.orchestra.run.smithi111.stdout:generic/423 0s 2023-03-29T12:55:09.720 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/423 2023-03-29T12:55:09.720 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:55:09.726 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:09.765 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:55:09.766 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/424) 2023-03-29T12:55:10.513 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:55:10.524 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:55:10.524 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:10.934 INFO:teuthology.orchestra.run.smithi111.stdout:generic/424 [not run] file system doesn't support any of /bin/chattr +a/+c/+d/+i 2023-03-29T12:55:10.942 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/424 2023-03-29T12:55:10.943 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/424 2023-03-29T12:55:10.943 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:55:10.948 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:10.968 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:55:10.969 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/425) 2023-03-29T12:55:11.359 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:55:11.370 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:55:11.371 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:11.676 INFO:teuthology.orchestra.run.smithi111.stdout:generic/425 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:55:11.685 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/425 2023-03-29T12:55:11.685 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/425 2023-03-29T12:55:11.685 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:55:11.690 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:55:11.709 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:55:11.709 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/426) 2023-03-29T12:55:12.088 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:55:12.099 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:55:12.099 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:07.013 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-03-29T12:56:07.017+0000 7f046aff9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 477089) UID: 0 2023-03-29T12:56:07.013 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T12:56:07.017+0000 7f7a699f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 477089) UID: 0 2023-03-29T12:56:07.014 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-03-29T12:56:07.017+0000 7f22b21f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 477089) UID: 0 2023-03-29T12:56:07.014 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-03-29T12:56:07.017+0000 7f93ce7f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 477089) UID: 0 2023-03-29T12:56:14.512 INFO:teuthology.orchestra.run.smithi111.stdout:generic/426 62s 2023-03-29T12:56:14.728 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/426 2023-03-29T12:56:14.729 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:56:14.733 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:15.006 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:56:15.006 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/427) 2023-03-29T12:56:17.933 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:56:17.944 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:56:17.944 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:18.385 INFO:teuthology.orchestra.run.smithi111.stdout:generic/427 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:56:18.392 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/427 2023-03-29T12:56:18.393 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/427 2023-03-29T12:56:18.393 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:56:18.398 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:18.417 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:56:18.417 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/428) 2023-03-29T12:56:18.818 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:56:18.829 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:56:18.829 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:19.233 INFO:teuthology.orchestra.run.smithi111.stdout:generic/428 1s 2023-03-29T12:56:19.244 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/428 2023-03-29T12:56:19.245 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:56:19.249 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:19.269 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:56:19.270 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/429) 2023-03-29T12:56:19.661 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:56:19.670 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:56:19.670 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:20.013 INFO:teuthology.orchestra.run.smithi111.stdout:generic/429 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:56:20.021 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/429 2023-03-29T12:56:20.021 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/429 2023-03-29T12:56:20.021 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:56:20.027 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:20.045 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:56:20.046 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/430) 2023-03-29T12:56:20.433 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:56:20.444 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:56:20.444 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:21.041 INFO:teuthology.orchestra.run.smithi111.stdout:generic/430 1s 2023-03-29T12:56:21.092 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/430 2023-03-29T12:56:21.092 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:56:21.097 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:21.117 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:56:21.117 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/431) 2023-03-29T12:56:21.890 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:56:21.901 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:56:21.902 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:22.455 INFO:teuthology.orchestra.run.smithi111.stdout:generic/431 1s 2023-03-29T12:56:22.466 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/431 2023-03-29T12:56:22.466 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:56:22.472 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:22.499 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:56:22.499 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/432) 2023-03-29T12:56:23.274 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:56:23.285 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:56:23.285 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:23.852 INFO:teuthology.orchestra.run.smithi111.stdout:generic/432 0s 2023-03-29T12:56:23.865 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/432 2023-03-29T12:56:23.865 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:56:23.871 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:23.925 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:56:23.925 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/433) 2023-03-29T12:56:24.671 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:56:24.682 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:56:24.682 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:25.189 INFO:teuthology.orchestra.run.smithi111.stdout:generic/433 1s 2023-03-29T12:56:25.202 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/433 2023-03-29T12:56:25.203 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:56:25.209 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:25.231 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:56:25.231 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/434) 2023-03-29T12:56:26.004 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:56:26.015 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:56:26.016 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:26.535 INFO:teuthology.orchestra.run.smithi111.stdout:generic/434 - output mismatch (see /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/434.out.bad) 2023-03-29T12:56:26.540 INFO:teuthology.orchestra.run.smithi111.stdout: --- tests/generic/434.out 2023-03-29 10:59:21.797746345 +0000 2023-03-29T12:56:26.541 INFO:teuthology.orchestra.run.smithi111.stdout: +++ /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/434.out.bad 2023-03-29 12:56:26.537194173 +0000 2023-03-29T12:56:26.541 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -9,6 +9,6 @@ 2023-03-29T12:56:26.541 INFO:teuthology.orchestra.run.smithi111.stdout: copy_range: Bad file descriptor 2023-03-29T12:56:26.541 INFO:teuthology.orchestra.run.smithi111.stdout: d41d8cd98f00b204e9800998ecf8427e TEST_DIR/test-434/copy 2023-03-29T12:56:26.541 INFO:teuthology.orchestra.run.smithi111.stdout: Try to copy to a device 2023-03-29T12:56:26.541 INFO:teuthology.orchestra.run.smithi111.stdout: -copy_range: Invalid argument 2023-03-29T12:56:26.541 INFO:teuthology.orchestra.run.smithi111.stdout: +/tmp/tmp.6dtVtxiLHFtest/test-434/dev1: Permission denied 2023-03-29T12:56:26.541 INFO:teuthology.orchestra.run.smithi111.stdout: Try to copy to a fifo 2023-03-29T12:56:26.541 INFO:teuthology.orchestra.run.smithi111.stdout: copy_range: Invalid argument 2023-03-29T12:56:26.547 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/434 2023-03-29T12:56:26.547 INFO:teuthology.orchestra.run.smithi111.stdout:Failures: generic/434 2023-03-29T12:56:26.547 INFO:teuthology.orchestra.run.smithi111.stdout:Failed 1 of 1 tests 2023-03-29T12:56:26.553 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:26.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T12:56:26.572 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-03-29T12:56:26.572 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-03-29T12:56:26.573 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/435) 2023-03-29T12:56:27.346 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:56:27.357 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:56:27.357 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:27.697 INFO:teuthology.orchestra.run.smithi111.stdout:generic/435 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:56:27.706 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/435 2023-03-29T12:56:27.706 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/435 2023-03-29T12:56:27.706 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:56:27.711 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:27.730 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:56:27.730 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/436) 2023-03-29T12:56:28.127 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:56:28.138 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:56:28.138 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:29.439 INFO:teuthology.orchestra.run.smithi111.stdout:generic/436 1s 2023-03-29T12:56:29.451 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/436 2023-03-29T12:56:29.451 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:56:29.456 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:29.476 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:56:29.476 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/437) 2023-03-29T12:56:30.211 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:56:30.223 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:56:30.223 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:31.301 INFO:teuthology.orchestra.run.smithi111.stdout:generic/437 1s 2023-03-29T12:56:31.313 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/437 2023-03-29T12:56:31.313 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:56:31.318 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:56:31.337 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:56:31.338 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/438) 2023-03-29T12:56:31.718 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:56:31.729 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:56:31.729 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:23.882 INFO:teuthology.orchestra.run.smithi111.stdout:generic/438 112s 2023-03-29T12:58:23.893 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/438 2023-03-29T12:58:23.894 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:23.899 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:23.920 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:23.921 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/439) 2023-03-29T12:58:24.311 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:24.322 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:24.322 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:24.630 INFO:teuthology.orchestra.run.smithi111.stdout:generic/439 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:58:24.638 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/439 2023-03-29T12:58:24.638 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/439 2023-03-29T12:58:24.638 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:24.644 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:24.663 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:24.663 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/440) 2023-03-29T12:58:25.055 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:25.067 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:25.067 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:25.403 INFO:teuthology.orchestra.run.smithi111.stdout:generic/440 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:58:25.411 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/440 2023-03-29T12:58:25.411 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/440 2023-03-29T12:58:25.412 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:25.417 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:25.436 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:25.437 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/441) 2023-03-29T12:58:25.834 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:25.845 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:25.845 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:26.859 INFO:teuthology.orchestra.run.smithi111.stdout:generic/441 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:58:26.868 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/441 2023-03-29T12:58:26.868 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/441 2023-03-29T12:58:26.868 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:26.873 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:26.894 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:26.894 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/442) 2023-03-29T12:58:27.285 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:27.296 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:27.296 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:27.609 INFO:teuthology.orchestra.run.smithi111.stdout:generic/442 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:58:27.617 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/442 2023-03-29T12:58:27.617 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/442 2023-03-29T12:58:27.617 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:27.622 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:27.643 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:27.644 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/443) 2023-03-29T12:58:28.034 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:28.047 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:28.047 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:28.421 INFO:teuthology.orchestra.run.smithi111.stdout:generic/443 0s 2023-03-29T12:58:28.433 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/443 2023-03-29T12:58:28.433 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:28.438 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:28.460 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:28.461 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/444) 2023-03-29T12:58:28.847 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:28.859 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:28.859 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:29.281 INFO:teuthology.orchestra.run.smithi111.stdout:generic/444 [not run] ACLs not supported by this filesystem type: ceph-fuse 2023-03-29T12:58:29.290 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/444 2023-03-29T12:58:29.290 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/444 2023-03-29T12:58:29.290 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:29.296 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:29.314 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:29.314 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/445) 2023-03-29T12:58:29.703 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:29.715 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:29.715 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:31.048 INFO:teuthology.orchestra.run.smithi111.stdout:generic/445 2s 2023-03-29T12:58:31.060 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/445 2023-03-29T12:58:31.060 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:31.065 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:31.086 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:31.087 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/446) 2023-03-29T12:58:31.810 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:31.821 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:31.822 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:32.162 INFO:teuthology.orchestra.run.smithi111.stdout:generic/446 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:58:32.170 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/446 2023-03-29T12:58:32.170 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/446 2023-03-29T12:58:32.170 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:32.175 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:32.194 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:32.194 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/447) 2023-03-29T12:58:33.682 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:33.693 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:33.694 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:34.001 INFO:teuthology.orchestra.run.smithi111.stdout:generic/447 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:58:34.009 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/447 2023-03-29T12:58:34.009 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/447 2023-03-29T12:58:34.009 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:34.014 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:34.035 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:34.035 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/448) 2023-03-29T12:58:34.419 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:34.430 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:34.430 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:35.704 INFO:teuthology.orchestra.run.smithi111.stdout:generic/448 1s 2023-03-29T12:58:35.716 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/448 2023-03-29T12:58:35.716 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:35.721 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:35.788 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:35.788 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/449) 2023-03-29T12:58:36.530 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:36.542 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:36.542 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:36.825 INFO:teuthology.orchestra.run.smithi111.stdout:generic/449 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:58:36.835 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/449 2023-03-29T12:58:36.835 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/449 2023-03-29T12:58:36.835 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:36.841 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:36.861 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:36.861 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/450) 2023-03-29T12:58:37.247 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:37.258 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:37.258 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:38.770 INFO:teuthology.orchestra.run.smithi111.stdout:generic/450 1s 2023-03-29T12:58:38.781 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/450 2023-03-29T12:58:38.781 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:38.787 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:38.813 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:38.813 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/451) 2023-03-29T12:58:39.569 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:39.581 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:39.581 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:40.275 INFO:teuthology.orchestra.run.smithi111.stdout:generic/451 - output mismatch (see /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/451.out.bad) 2023-03-29T12:58:40.282 INFO:teuthology.orchestra.run.smithi111.stdout: --- tests/generic/451.out 2023-03-29 10:59:21.800746296 +0000 2023-03-29T12:58:40.282 INFO:teuthology.orchestra.run.smithi111.stdout: +++ /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/451.out.bad 2023-03-29 12:58:40.266985607 +0000 2023-03-29T12:58:40.282 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -1,2 +1,5 @@ 2023-03-29T12:58:40.282 INFO:teuthology.orchestra.run.smithi111.stdout: QA output created by 451 2023-03-29T12:58:40.282 INFO:teuthology.orchestra.run.smithi111.stdout: +get stale data from buffer read 2023-03-29T12:58:40.282 INFO:teuthology.orchestra.run.smithi111.stdout: +00000000 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 UUUUUUUUUUUUUUUU 2023-03-29T12:58:40.282 INFO:teuthology.orchestra.run.smithi111.stdout: +000a0000 2023-03-29T12:58:40.282 INFO:teuthology.orchestra.run.smithi111.stdout: Silence is golden 2023-03-29T12:58:40.288 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/451 2023-03-29T12:58:40.288 INFO:teuthology.orchestra.run.smithi111.stdout:Failures: generic/451 2023-03-29T12:58:40.289 INFO:teuthology.orchestra.run.smithi111.stdout:Failed 1 of 1 tests 2023-03-29T12:58:40.294 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:40.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T12:58:40.334 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-03-29T12:58:40.334 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-03-29T12:58:40.334 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/452) 2023-03-29T12:58:41.069 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:41.080 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:41.081 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:41.371 INFO:teuthology.orchestra.run.smithi111.stdout:generic/452 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:58:41.380 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/452 2023-03-29T12:58:41.380 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/452 2023-03-29T12:58:41.380 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:41.385 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:41.404 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:41.405 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/453) 2023-03-29T12:58:41.809 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:41.820 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:41.820 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:42.154 INFO:teuthology.orchestra.run.smithi111.stdout:generic/453 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:58:42.163 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/453 2023-03-29T12:58:42.163 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/453 2023-03-29T12:58:42.163 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:42.169 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:42.191 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:42.192 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/454) 2023-03-29T12:58:42.601 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:42.612 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:42.612 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:42.893 INFO:teuthology.orchestra.run.smithi111.stdout:generic/454 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:58:42.901 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/454 2023-03-29T12:58:42.901 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/454 2023-03-29T12:58:42.901 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:42.907 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:42.928 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:42.928 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/455) 2023-03-29T12:58:43.327 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:43.338 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:43.339 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:43.843 INFO:teuthology.orchestra.run.smithi111.stdout:generic/455 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:58:43.851 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/455 2023-03-29T12:58:43.851 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/455 2023-03-29T12:58:43.851 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:43.857 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:43.876 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:43.876 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/456) 2023-03-29T12:58:44.254 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:44.265 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:44.265 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:44.675 INFO:teuthology.orchestra.run.smithi111.stdout:generic/456 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:58:44.683 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/456 2023-03-29T12:58:44.683 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/456 2023-03-29T12:58:44.683 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:44.689 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:44.709 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:44.709 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/457) 2023-03-29T12:58:45.094 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:45.106 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:45.106 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:45.501 INFO:teuthology.orchestra.run.smithi111.stdout:generic/457 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:58:45.509 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/457 2023-03-29T12:58:45.509 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/457 2023-03-29T12:58:45.509 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:45.514 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:45.535 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:45.535 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/458) 2023-03-29T12:58:45.936 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:45.948 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:45.948 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:46.224 INFO:teuthology.orchestra.run.smithi111.stdout:generic/458 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:58:46.232 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/458 2023-03-29T12:58:46.232 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/458 2023-03-29T12:58:46.233 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:46.237 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:46.258 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:46.258 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/459) 2023-03-29T12:58:46.646 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:46.657 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:46.657 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:47.452 INFO:teuthology.orchestra.run.smithi111.stdout:generic/459 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:58:47.460 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/459 2023-03-29T12:58:47.460 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/459 2023-03-29T12:58:47.460 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:47.465 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:47.485 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:47.485 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/460) 2023-03-29T12:58:47.873 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:47.884 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:47.884 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:48.216 INFO:teuthology.orchestra.run.smithi111.stdout:generic/460 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:58:48.224 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/460 2023-03-29T12:58:48.224 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/460 2023-03-29T12:58:48.225 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:48.230 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:48.251 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:48.252 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/461) 2023-03-29T12:58:48.635 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:48.646 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:48.646 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:48.923 INFO:teuthology.orchestra.run.smithi111.stdout:generic/461 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:58:48.932 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/461 2023-03-29T12:58:48.932 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/461 2023-03-29T12:58:48.932 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:48.937 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:48.956 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:48.956 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/462) 2023-03-29T12:58:49.340 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:49.351 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:49.351 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:49.652 INFO:teuthology.orchestra.run.smithi111.stdout:generic/462 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:58:49.660 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/462 2023-03-29T12:58:49.660 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/462 2023-03-29T12:58:49.660 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:49.665 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:49.685 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:49.686 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/463) 2023-03-29T12:58:50.075 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:50.086 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:50.086 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:50.482 INFO:teuthology.orchestra.run.smithi111.stdout:generic/463 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T12:58:50.490 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/463 2023-03-29T12:58:50.490 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/463 2023-03-29T12:58:50.490 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:50.496 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:50.517 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:50.517 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/464) 2023-03-29T12:58:51.288 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:51.300 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:51.300 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:51.588 INFO:teuthology.orchestra.run.smithi111.stdout:generic/464 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:58:51.597 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/464 2023-03-29T12:58:51.597 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/464 2023-03-29T12:58:51.597 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:58:51.602 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:58:51.622 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:58:51.623 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/465) 2023-03-29T12:58:52.012 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:58:52.023 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:58:52.023 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:08.125 INFO:teuthology.orchestra.run.smithi111.stdout:generic/465 16s 2023-03-29T12:59:08.185 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/465 2023-03-29T12:59:08.185 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:59:08.208 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:08.268 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:59:08.268 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/466) 2023-03-29T12:59:08.826 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:59:08.837 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:59:08.837 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:09.150 INFO:teuthology.orchestra.run.smithi111.stdout:generic/466 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:59:09.158 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/466 2023-03-29T12:59:09.158 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/466 2023-03-29T12:59:09.158 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:59:09.164 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:09.183 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:59:09.183 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/467) 2023-03-29T12:59:09.597 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:59:09.608 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:59:09.608 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:33.735 INFO:teuthology.orchestra.run.smithi111.stdout:generic/467 24s 2023-03-29T12:59:33.830 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/467 2023-03-29T12:59:33.830 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:59:33.835 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:34.089 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:59:34.089 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/468) 2023-03-29T12:59:35.342 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:59:35.352 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:59:35.353 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:35.650 INFO:teuthology.orchestra.run.smithi111.stdout:generic/468 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:59:35.659 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/468 2023-03-29T12:59:35.659 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/468 2023-03-29T12:59:35.659 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:59:35.664 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:35.683 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:59:35.683 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/469) 2023-03-29T12:59:36.062 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:59:36.073 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:59:36.073 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:38.952 INFO:teuthology.orchestra.run.smithi111.stdout:generic/469 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2023-03-29T12:59:38.960 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/469 2023-03-29T12:59:38.960 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/469 2023-03-29T12:59:38.960 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:59:38.966 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:38.986 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:59:38.986 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/470) 2023-03-29T12:59:39.715 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:59:39.726 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:59:39.726 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:40.097 INFO:teuthology.orchestra.run.smithi111.stdout:generic/470 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:59:40.106 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/470 2023-03-29T12:59:40.106 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/470 2023-03-29T12:59:40.106 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:59:40.111 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:40.129 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:59:40.130 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/471) 2023-03-29T12:59:40.515 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:59:40.526 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:59:40.526 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:40.913 INFO:teuthology.orchestra.run.smithi111.stdout:generic/471 [not run] xfs_io pwrite -V 1 -b 4k -N failed (old kernel/wrong fs?) 2023-03-29T12:59:40.921 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/471 2023-03-29T12:59:40.921 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/471 2023-03-29T12:59:40.921 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:59:40.926 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:40.985 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:59:40.985 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/472) 2023-03-29T12:59:41.655 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:59:41.666 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:59:41.666 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:41.959 INFO:teuthology.orchestra.run.smithi111.stdout:generic/472 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:59:41.967 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/472 2023-03-29T12:59:41.967 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/472 2023-03-29T12:59:41.967 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:59:41.972 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:41.991 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:59:41.992 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/473) 2023-03-29T12:59:42.375 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:59:42.387 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:59:42.387 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:42.749 INFO:teuthology.orchestra.run.smithi111.stdout:generic/473 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2023-03-29T12:59:42.758 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/473 2023-03-29T12:59:42.758 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/473 2023-03-29T12:59:42.758 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:59:42.763 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:42.783 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:59:42.783 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/474) 2023-03-29T12:59:43.165 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:59:43.176 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:59:43.176 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:43.468 INFO:teuthology.orchestra.run.smithi111.stdout:generic/474 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:59:43.476 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/474 2023-03-29T12:59:43.476 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/474 2023-03-29T12:59:43.476 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:59:43.482 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:43.502 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:59:43.502 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/475) 2023-03-29T12:59:43.889 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:59:43.900 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:59:43.909 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:44.241 INFO:teuthology.orchestra.run.smithi111.stdout:generic/475 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:59:44.249 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/475 2023-03-29T12:59:44.249 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/475 2023-03-29T12:59:44.249 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:59:44.255 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:44.275 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:59:44.276 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/476) 2023-03-29T12:59:44.659 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:59:44.670 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:59:44.670 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:44.969 INFO:teuthology.orchestra.run.smithi111.stdout:generic/476 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:59:44.977 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/476 2023-03-29T12:59:44.978 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/476 2023-03-29T12:59:44.978 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:59:44.983 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:45.002 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:59:45.002 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/477) 2023-03-29T12:59:45.387 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:59:45.398 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:59:45.398 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:53.716 INFO:teuthology.orchestra.run.smithi111.stdout:generic/477 8s 2023-03-29T12:59:53.819 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/477 2023-03-29T12:59:53.819 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:59:53.824 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:53.885 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:59:53.885 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/478) 2023-03-29T12:59:55.006 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:59:55.017 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:59:55.017 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:55.856 INFO:teuthology.orchestra.run.smithi111.stdout:generic/478 - output mismatch (see /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/478.out.bad) 2023-03-29T12:59:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: --- tests/generic/478.out 2023-03-29 10:59:21.804746231 +0000 2023-03-29T12:59:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: +++ /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/478.out.bad 2023-03-29 12:59:55.857735211 +0000 2023-03-29T12:59:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -10,7 +10,7 @@ 2023-03-29T12:59:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: lock could be placed 2023-03-29T12:59:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: get wrlck 2023-03-29T12:59:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: lock could be placed 2023-03-29T12:59:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: -lock could be placed 2023-03-29T12:59:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: +get wrlck 2023-03-29T12:59:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: lock could be placed 2023-03-29T12:59:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: lock could be placed 2023-03-29T12:59:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: lock could be placed 2023-03-29T12:59:55.862 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -25,7 +25,7 @@ 2023-03-29T12:59:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: lock could be placed 2023-03-29T12:59:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: get wrlck 2023-03-29T12:59:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: lock could be placed 2023-03-29T12:59:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: -lock could be placed 2023-03-29T12:59:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: +get wrlck 2023-03-29T12:59:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: lock could be placed 2023-03-29T12:59:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: lock could be placed 2023-03-29T12:59:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: lock could be placed 2023-03-29T12:59:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -37,7 +37,7 @@ 2023-03-29T12:59:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: get rdlck 2023-03-29T12:59:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: get rdlck 2023-03-29T12:59:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: lock could be placed 2023-03-29T12:59:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: -lock could be placed 2023-03-29T12:59:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: +get rdlck 2023-03-29T12:59:55.863 INFO:teuthology.orchestra.run.smithi111.stdout: get rdlck 2023-03-29T12:59:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: lock could be placed 2023-03-29T12:59:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: get rdlck 2023-03-29T12:59:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -46,7 +46,7 @@ 2023-03-29T12:59:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: get rdlck 2023-03-29T12:59:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: get rdlck 2023-03-29T12:59:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: lock could be placed 2023-03-29T12:59:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: -lock could be placed 2023-03-29T12:59:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: +get rdlck 2023-03-29T12:59:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: lock could be placed 2023-03-29T12:59:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: lock could be placed 2023-03-29T12:59:55.864 INFO:teuthology.orchestra.run.smithi111.stdout: lock could be placed 2023-03-29T12:59:55.867 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/478 2023-03-29T12:59:55.867 INFO:teuthology.orchestra.run.smithi111.stdout:Failures: generic/478 2023-03-29T12:59:55.867 INFO:teuthology.orchestra.run.smithi111.stdout:Failed 1 of 1 tests 2023-03-29T12:59:55.872 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:55.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T12:59:55.891 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-03-29T12:59:55.891 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-03-29T12:59:55.891 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/479) 2023-03-29T12:59:56.278 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:59:56.289 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:59:56.289 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:56.673 INFO:teuthology.orchestra.run.smithi111.stdout:generic/479 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:59:56.681 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/479 2023-03-29T12:59:56.681 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/479 2023-03-29T12:59:56.681 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:59:56.686 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:56.705 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:59:56.705 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/480) 2023-03-29T12:59:57.091 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:59:57.102 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:59:57.102 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:57.404 INFO:teuthology.orchestra.run.smithi111.stdout:generic/480 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:59:57.412 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/480 2023-03-29T12:59:57.412 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/480 2023-03-29T12:59:57.412 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:59:57.417 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:57.436 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:59:57.437 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/481) 2023-03-29T12:59:57.821 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:59:57.832 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:59:57.832 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:58.136 INFO:teuthology.orchestra.run.smithi111.stdout:generic/481 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:59:58.144 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/481 2023-03-29T12:59:58.144 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/481 2023-03-29T12:59:58.144 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:59:58.149 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:58.169 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:59:58.169 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/482) 2023-03-29T12:59:58.554 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:59:58.565 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:59:58.565 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:58.960 INFO:teuthology.orchestra.run.smithi111.stdout:generic/482 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:59:58.968 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/482 2023-03-29T12:59:58.968 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/482 2023-03-29T12:59:58.968 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:59:58.974 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:58.992 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:59:58.993 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/483) 2023-03-29T12:59:59.377 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T12:59:59.388 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T12:59:59.388 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:59.693 INFO:teuthology.orchestra.run.smithi111.stdout:generic/483 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T12:59:59.701 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/483 2023-03-29T12:59:59.701 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/483 2023-03-29T12:59:59.701 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T12:59:59.706 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T12:59:59.725 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T12:59:59.725 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/484) 2023-03-29T12:59:59.877 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:59:59.880 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T12:59:59.965 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-03-29T12:59:59.968+0000 7fabb15f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123883) UID: 0 2023-03-29T12:59:59.966 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-03-29T12:59:59.968+0000 7f0c30df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123883) UID: 0 2023-03-29T12:59:59.966 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-03-29T12:59:59.968+0000 7fd2277f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123883) UID: 0 2023-03-29T12:59:59.966 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-03-29T12:59:59.968+0000 7f9655df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123883) UID: 0 2023-03-29T13:00:00.111 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:00.122 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:00.122 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:00.430 INFO:teuthology.orchestra.run.smithi111.stdout:generic/484 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:00.438 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/484 2023-03-29T13:00:00.438 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/484 2023-03-29T13:00:00.438 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:00.443 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:00.462 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:00.462 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/485) 2023-03-29T13:00:00.884 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:00.895 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:00.895 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:02.511 INFO:teuthology.orchestra.run.smithi111.stdout:generic/485 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2023-03-29T13:00:02.519 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/485 2023-03-29T13:00:02.520 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/485 2023-03-29T13:00:02.520 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:02.525 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:02.545 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:02.545 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/486) 2023-03-29T13:00:03.249 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:03.260 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:03.260 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:03.583 INFO:teuthology.orchestra.run.smithi111.stdout:generic/486 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:03.591 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/486 2023-03-29T13:00:03.591 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/486 2023-03-29T13:00:03.591 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:03.597 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:03.615 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:03.616 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/487) 2023-03-29T13:00:03.999 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:04.010 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:04.010 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:04.310 INFO:teuthology.orchestra.run.smithi111.stdout:generic/487 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:04.318 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/487 2023-03-29T13:00:04.319 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/487 2023-03-29T13:00:04.319 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:04.324 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:04.343 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:04.343 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/488) 2023-03-29T13:00:04.733 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:04.744 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:04.744 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:05.012 INFO:teuthology.orchestra.run.smithi111.stdout:generic/488 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:05.020 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/488 2023-03-29T13:00:05.020 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/488 2023-03-29T13:00:05.021 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:05.026 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:05.045 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:05.046 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/489) 2023-03-29T13:00:05.430 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:05.441 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:05.442 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:05.740 INFO:teuthology.orchestra.run.smithi111.stdout:generic/489 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:05.747 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/489 2023-03-29T13:00:05.748 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/489 2023-03-29T13:00:05.748 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:05.753 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:05.774 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:05.774 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/490) 2023-03-29T13:00:06.151 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:06.162 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:06.162 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:07.460 INFO:teuthology.orchestra.run.smithi111.stdout:generic/490 1s 2023-03-29T13:00:07.472 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/490 2023-03-29T13:00:07.472 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:07.477 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:07.498 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:07.498 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/491) 2023-03-29T13:00:08.219 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:08.230 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:08.231 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:08.524 INFO:teuthology.orchestra.run.smithi111.stdout:generic/491 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:08.532 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/491 2023-03-29T13:00:08.532 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/491 2023-03-29T13:00:08.533 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:08.538 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:08.557 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:08.557 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/492) 2023-03-29T13:00:08.939 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:08.951 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:08.951 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:09.230 INFO:teuthology.orchestra.run.smithi111.stdout:generic/492 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:09.238 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/492 2023-03-29T13:00:09.239 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/492 2023-03-29T13:00:09.239 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:09.244 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:09.263 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:09.264 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/493) 2023-03-29T13:00:09.645 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:09.656 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:09.656 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:09.935 INFO:teuthology.orchestra.run.smithi111.stdout:generic/493 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:09.943 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/493 2023-03-29T13:00:09.943 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/493 2023-03-29T13:00:09.943 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:09.949 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:09.968 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:09.968 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/494) 2023-03-29T13:00:10.351 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:10.362 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:10.362 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:10.630 INFO:teuthology.orchestra.run.smithi111.stdout:generic/494 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:10.638 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/494 2023-03-29T13:00:10.638 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/494 2023-03-29T13:00:10.638 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:10.644 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:10.663 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:10.663 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/495) 2023-03-29T13:00:11.049 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:11.061 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:11.061 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:11.337 INFO:teuthology.orchestra.run.smithi111.stdout:generic/495 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:11.345 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/495 2023-03-29T13:00:11.346 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/495 2023-03-29T13:00:11.346 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:11.351 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:11.370 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:11.370 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/496) 2023-03-29T13:00:11.753 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:11.764 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:11.764 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:12.055 INFO:teuthology.orchestra.run.smithi111.stdout:generic/496 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:12.063 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/496 2023-03-29T13:00:12.063 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/496 2023-03-29T13:00:12.063 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:12.068 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:12.087 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:12.087 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/497) 2023-03-29T13:00:12.471 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:12.482 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:12.482 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:12.753 INFO:teuthology.orchestra.run.smithi111.stdout:generic/497 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:12.761 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/497 2023-03-29T13:00:12.762 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/497 2023-03-29T13:00:12.762 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:12.767 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:12.786 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:12.786 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/498) 2023-03-29T13:00:13.168 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:13.179 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:13.179 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:13.479 INFO:teuthology.orchestra.run.smithi111.stdout:generic/498 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:13.488 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/498 2023-03-29T13:00:13.488 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/498 2023-03-29T13:00:13.488 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:13.493 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:13.511 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:13.512 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/499) 2023-03-29T13:00:13.896 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:13.907 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:13.907 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:14.176 INFO:teuthology.orchestra.run.smithi111.stdout:generic/499 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:14.184 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/499 2023-03-29T13:00:14.185 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/499 2023-03-29T13:00:14.185 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:14.190 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:14.210 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:14.210 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/500) 2023-03-29T13:00:14.592 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:14.604 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:14.604 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:14.963 INFO:teuthology.orchestra.run.smithi111.stdout:generic/500 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:14.971 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/500 2023-03-29T13:00:14.971 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/500 2023-03-29T13:00:14.971 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:14.977 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:14.996 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:14.996 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/501) 2023-03-29T13:00:15.376 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:15.387 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:15.388 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:15.689 INFO:teuthology.orchestra.run.smithi111.stdout:generic/501 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:15.697 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/501 2023-03-29T13:00:15.697 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/501 2023-03-29T13:00:15.697 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:15.702 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:15.721 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:15.721 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/502) 2023-03-29T13:00:16.102 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:16.113 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:16.113 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:16.416 INFO:teuthology.orchestra.run.smithi111.stdout:generic/502 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:16.424 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/502 2023-03-29T13:00:16.424 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/502 2023-03-29T13:00:16.424 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:16.430 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:16.449 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:16.450 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/503) 2023-03-29T13:00:16.836 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:16.847 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:16.847 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:17.130 INFO:teuthology.orchestra.run.smithi111.stdout:generic/503 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:17.138 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/503 2023-03-29T13:00:17.138 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/503 2023-03-29T13:00:17.138 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:17.143 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:17.162 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:17.163 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/504) 2023-03-29T13:00:17.545 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:17.556 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:17.556 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:17.906 INFO:teuthology.orchestra.run.smithi111.stdout:generic/504 - output mismatch (see /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/504.out.bad) 2023-03-29T13:00:17.910 INFO:teuthology.orchestra.run.smithi111.stdout: --- tests/generic/504.out 2023-03-29 10:59:21.807746182 +0000 2023-03-29T13:00:17.911 INFO:teuthology.orchestra.run.smithi111.stdout: +++ /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/504.out.bad 2023-03-29 13:00:17.907370055 +0000 2023-03-29T13:00:17.911 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -1,2 +1,3 @@ 2023-03-29T13:00:17.911 INFO:teuthology.orchestra.run.smithi111.stdout: QA output created by 504 2023-03-29T13:00:17.911 INFO:teuthology.orchestra.run.smithi111.stdout: +lock info not found 2023-03-29T13:00:17.911 INFO:teuthology.orchestra.run.smithi111.stdout: Silence is golden 2023-03-29T13:00:17.916 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/504 2023-03-29T13:00:17.916 INFO:teuthology.orchestra.run.smithi111.stdout:Failures: generic/504 2023-03-29T13:00:17.916 INFO:teuthology.orchestra.run.smithi111.stdout:Failed 1 of 1 tests 2023-03-29T13:00:17.921 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:17.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T13:00:17.940 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-03-29T13:00:17.941 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-03-29T13:00:17.941 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/505) 2023-03-29T13:00:18.321 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:18.332 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:18.332 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:18.609 INFO:teuthology.orchestra.run.smithi111.stdout:generic/505 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:18.618 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/505 2023-03-29T13:00:18.618 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/505 2023-03-29T13:00:18.618 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:18.623 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:18.643 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:18.643 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/506) 2023-03-29T13:00:19.027 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:19.038 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:19.039 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:19.316 INFO:teuthology.orchestra.run.smithi111.stdout:generic/506 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:19.324 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/506 2023-03-29T13:00:19.324 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/506 2023-03-29T13:00:19.324 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:19.330 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:19.349 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:19.349 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/507) 2023-03-29T13:00:19.730 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:19.742 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:19.742 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:20.087 INFO:teuthology.orchestra.run.smithi111.stdout:generic/507 [not run] file system doesn't support chattr +ASai 2023-03-29T13:00:20.095 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/507 2023-03-29T13:00:20.095 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/507 2023-03-29T13:00:20.095 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:20.100 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:20.120 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:20.120 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/508) 2023-03-29T13:00:20.512 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:20.524 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:20.524 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:20.814 INFO:teuthology.orchestra.run.smithi111.stdout:generic/508 [not run] lsattr not supported by test filesystem type: ceph-fuse 2023-03-29T13:00:20.823 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/508 2023-03-29T13:00:20.823 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/508 2023-03-29T13:00:20.823 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:20.828 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:20.847 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:20.848 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/509) 2023-03-29T13:00:21.233 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:21.244 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:21.244 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:21.538 INFO:teuthology.orchestra.run.smithi111.stdout:generic/509 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:21.546 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/509 2023-03-29T13:00:21.546 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/509 2023-03-29T13:00:21.546 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:21.551 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:21.572 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:21.572 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/510) 2023-03-29T13:00:21.956 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:21.967 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:21.968 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:22.272 INFO:teuthology.orchestra.run.smithi111.stdout:generic/510 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:22.280 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/510 2023-03-29T13:00:22.280 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/510 2023-03-29T13:00:22.280 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:22.285 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:22.304 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:22.304 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/511) 2023-03-29T13:00:22.687 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:22.698 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:22.698 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:22.970 INFO:teuthology.orchestra.run.smithi111.stdout:generic/511 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:22.978 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/511 2023-03-29T13:00:22.978 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/511 2023-03-29T13:00:22.978 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:22.984 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:23.004 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:23.005 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/512) 2023-03-29T13:00:23.387 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:23.398 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:23.398 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:23.697 INFO:teuthology.orchestra.run.smithi111.stdout:generic/512 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:23.705 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/512 2023-03-29T13:00:23.705 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/512 2023-03-29T13:00:23.706 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:23.711 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:23.730 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:23.731 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/513) 2023-03-29T13:00:24.114 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:24.125 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:24.125 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:24.400 INFO:teuthology.orchestra.run.smithi111.stdout:generic/513 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:24.408 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/513 2023-03-29T13:00:24.408 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/513 2023-03-29T13:00:24.408 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:24.413 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:24.433 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:24.433 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/514) 2023-03-29T13:00:24.825 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:24.836 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:24.836 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:25.114 INFO:teuthology.orchestra.run.smithi111.stdout:generic/514 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:25.122 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/514 2023-03-29T13:00:25.122 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/514 2023-03-29T13:00:25.122 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:25.128 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:25.147 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:25.147 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/515) 2023-03-29T13:00:25.527 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:25.575 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:25.575 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:25.830 INFO:teuthology.orchestra.run.smithi111.stdout:generic/515 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:25.838 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/515 2023-03-29T13:00:25.838 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/515 2023-03-29T13:00:25.838 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:25.844 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:25.863 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:25.864 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/516) 2023-03-29T13:00:26.252 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:26.263 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:26.264 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:26.657 INFO:teuthology.orchestra.run.smithi111.stdout:generic/516 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2023-03-29T13:00:26.666 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/516 2023-03-29T13:00:26.666 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/516 2023-03-29T13:00:26.666 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:26.671 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:26.689 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:26.690 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/517) 2023-03-29T13:00:27.402 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:27.413 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:27.414 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:27.711 INFO:teuthology.orchestra.run.smithi111.stdout:generic/517 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:27.719 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/517 2023-03-29T13:00:27.719 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/517 2023-03-29T13:00:27.719 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:27.724 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:27.744 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:27.744 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/518) 2023-03-29T13:00:28.125 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:28.136 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:28.136 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:28.410 INFO:teuthology.orchestra.run.smithi111.stdout:generic/518 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:28.418 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/518 2023-03-29T13:00:28.418 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/518 2023-03-29T13:00:28.418 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:28.423 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:28.443 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:28.443 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/519) 2023-03-29T13:00:28.830 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:28.841 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:28.841 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:29.114 INFO:teuthology.orchestra.run.smithi111.stdout:generic/519 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:29.122 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/519 2023-03-29T13:00:29.123 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/519 2023-03-29T13:00:29.123 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:29.128 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:29.147 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:29.147 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/520) 2023-03-29T13:00:29.532 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:29.543 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:29.543 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:29.850 INFO:teuthology.orchestra.run.smithi111.stdout:generic/520 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T13:00:29.858 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/520 2023-03-29T13:00:29.858 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/520 2023-03-29T13:00:29.858 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:00:29.863 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:00:29.883 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:00:29.883 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/521) 2023-03-29T13:00:30.262 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:00:30.273 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:00:30.273 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:03:00.940 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:03:00.946 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:03:01.030 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-03-29T13:03:01.034+0000 7fd2277f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123949) UID: 0 2023-03-29T13:03:01.031 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-03-29T13:03:01.035+0000 7f0c30df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123949) UID: 0 2023-03-29T13:03:01.032 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-03-29T13:03:01.035+0000 7fabb15f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123949) UID: 0 2023-03-29T13:03:01.032 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-03-29T13:03:01.035+0000 7f9655df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 123949) UID: 0 2023-03-29T13:05:55.966 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:05:55.971 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:06:26.143 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:06:26.148 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:06:56.242 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:06:56.246 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:07:26.339 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:07:26.344 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:07:56.496 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:07:56.500 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:08:26.606 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:08:26.610 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:08:56.716 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:08:56.720 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:09:26.838 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:09:26.842 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:09:56.956 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:09:56.961 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:10:27.069 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:10:27.073 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:10:57.175 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:10:57.179 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:11:27.362 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:11:27.367 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:11:57.478 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:11:57.482 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:12:27.583 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:12:27.587 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:12:59.611 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:12:59.615 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:13:30.387 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:13:30.391 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:14:00.497 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:14:00.501 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:14:30.611 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:14:30.615 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:15:00.714 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:15:00.719 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:15:30.812 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:15:30.815 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:15:30.901 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-03-29T13:15:30.905+0000 7fd2277f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 124552) UID: 0 2023-03-29T13:15:30.902 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-03-29T13:15:30.905+0000 7fabb15f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 124552) UID: 0 2023-03-29T13:15:30.902 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-03-29T13:15:30.905+0000 7f0c30df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 124552) UID: 0 2023-03-29T13:15:30.902 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-03-29T13:15:30.905+0000 7f9655df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 124552) UID: 0 2023-03-29T13:18:28.840 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:18:28.845 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:18:59.017 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:18:59.021 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:19:29.132 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:19:29.136 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:19:59.237 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:19:59.241 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:20:29.401 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:20:29.405 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:20:59.505 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:20:59.509 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:21:29.619 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:21:29.623 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:21:59.732 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:21:59.736 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:22:29.862 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:22:29.866 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:23:00.149 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:23:00.153 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:23:30.363 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:23:30.367 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:24:00.620 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:24:00.625 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:24:30.750 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:24:30.755 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:25:01.797 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:25:01.802 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:25:31.911 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:25:31.918 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:26:02.049 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:26:02.054 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:26:32.255 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:26:32.259 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:27:02.364 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:27:02.369 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:27:32.469 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:27:32.473 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:28:02.582 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:28:02.587 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:28:32.758 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:28:32.762 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:29:02.869 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:29:02.873 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:29:32.969 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:29:32.974 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:29:33.056 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-03-29T13:29:33.061+0000 7fd2277f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 125238) UID: 0 2023-03-29T13:29:33.057 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-03-29T13:29:33.061+0000 7fabb15f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 125238) UID: 0 2023-03-29T13:29:33.057 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-03-29T13:29:33.061+0000 7f0c30df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 125238) UID: 0 2023-03-29T13:29:33.057 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-03-29T13:29:33.062+0000 7f9655df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 125238) UID: 0 2023-03-29T13:32:28.347 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:32:28.355 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:32:28.441 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-03-29T13:32:28.446+0000 7f046aff9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 537901) UID: 0 2023-03-29T13:32:28.441 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T13:32:28.446+0000 7f7a699f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 537901) UID: 0 2023-03-29T13:32:28.441 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-03-29T13:32:28.446+0000 7f22b21f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 537901) UID: 0 2023-03-29T13:32:28.442 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-03-29T13:32:28.446+0000 7f93ce7f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 537901) UID: 0 2023-03-29T13:35:19.697 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:35:19.701 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:35:49.808 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:35:49.813 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:36:19.910 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:36:19.914 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:36:50.249 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:36:50.253 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:37:20.347 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:37:20.352 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:37:50.466 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:37:50.471 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:38:20.565 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:38:20.570 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:38:51.729 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:38:51.733 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:39:22.051 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:39:22.056 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:39:52.157 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:39:52.162 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:40:22.263 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:40:22.267 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:40:52.443 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:40:52.448 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:41:22.647 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:41:22.651 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:41:52.821 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:41:52.825 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:42:22.943 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:42:22.947 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:42:53.128 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:42:53.132 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:43:23.245 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:43:23.250 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:43:53.342 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:43:53.347 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:44:23.455 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:44:23.460 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:44:53.561 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:44:53.565 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:45:23.724 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:45:23.728 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:45:53.855 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:45:53.859 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:46:23.951 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:46:23.954 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:46:54.057 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:46:54.061 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:47:24.203 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:47:24.206 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:47:54.334 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:47:54.339 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:48:24.439 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:48:24.444 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:48:54.553 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:48:54.557 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:49:24.667 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:49:24.671 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:49:55.348 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:49:55.352 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:50:25.465 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:50:25.469 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:50:55.574 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:50:55.578 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:51:25.669 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:51:25.673 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:51:55.785 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:51:55.789 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:52:25.932 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:52:25.937 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:52:56.154 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:52:56.159 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:53:26.259 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:53:26.264 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:53:56.357 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:53:56.362 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:54:26.472 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:54:26.476 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:54:56.576 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:54:56.580 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:55:26.698 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:55:26.703 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:55:56.983 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:55:56.987 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:56:27.105 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:56:27.109 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:56:27.212 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-03-29T13:56:27.212+0000 7fd2277f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 126550) UID: 0 2023-03-29T13:56:27.212 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-03-29T13:56:27.213+0000 7fabb15f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 126550) UID: 0 2023-03-29T13:56:27.213 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-03-29T13:56:27.213+0000 7f0c30df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 126550) UID: 0 2023-03-29T13:56:27.213 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-03-29T13:56:27.213+0000 7f9655df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 126550) UID: 0 2023-03-29T13:56:51.180 INFO:teuthology.orchestra.run.smithi111.stdout:generic/521 3381s 2023-03-29T13:56:51.191 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/521 2023-03-29T13:56:51.192 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T13:56:51.197 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:56:51.243 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T13:56:51.243 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/522) 2023-03-29T13:56:51.930 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T13:56:51.942 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T13:56:51.942 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T13:59:17.886 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:59:17.890 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:59:47.990 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T13:59:47.994 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:00:18.096 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:00:18.100 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:00:18.186 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-03-29T14:00:18.186+0000 7fd2277f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 126650) UID: 0 2023-03-29T14:00:18.187 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-03-29T14:00:18.186+0000 7f0c30df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 126650) UID: 0 2023-03-29T14:00:18.187 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-03-29T14:00:18.186+0000 7fabb15f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 126650) UID: 0 2023-03-29T14:00:18.187 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-03-29T14:00:18.186+0000 7f9655df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 126650) UID: 0 2023-03-29T14:02:59.974 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:02:59.979 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:03:30.182 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:03:30.212 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:04:01.596 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:04:01.600 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:04:31.762 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:04:31.766 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:05:01.910 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:05:01.915 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:05:32.357 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:05:32.362 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:06:02.496 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:06:02.501 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:06:32.644 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:06:32.648 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:07:02.758 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:07:02.763 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:07:32.861 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:07:32.865 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:08:02.964 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:08:02.968 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:08:03.098 INFO:tasks.ceph.mds.a.smithi111.stderr:2023-03-29T14:08:03.097+0000 7f3a763f9700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 540087) UID: 0 2023-03-29T14:08:03.098 INFO:tasks.ceph.mds.c.smithi111.stderr:2023-03-29T14:08:03.097+0000 7f155ddf9700 -1 received signal: Hangup from killall ceph-mds -1 (PID: 540087) UID: 0 2023-03-29T14:10:50.722 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:10:50.727 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:11:20.837 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:11:20.842 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:11:50.941 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:11:50.945 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:12:21.073 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:12:21.079 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:12:52.188 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:12:52.193 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:13:22.294 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:13:22.304 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:13:52.975 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:13:52.980 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:14:23.098 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:14:23.102 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:14:53.197 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:14:53.201 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:15:23.375 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:15:23.379 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:15:53.507 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:15:53.512 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:16:23.601 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:16:23.605 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:16:53.765 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:16:53.770 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:17:23.879 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:17:23.883 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:17:54.000 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:17:54.005 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:18:24.348 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:18:24.352 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:18:54.455 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:18:54.459 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:19:24.593 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:19:24.598 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:19:54.715 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:19:54.720 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:20:24.821 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:20:24.826 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:20:24.907 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-03-29T14:20:24.906+0000 7f046aff9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 540687) UID: 0 2023-03-29T14:20:24.907 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T14:20:24.906+0000 7f7a699f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 540687) UID: 0 2023-03-29T14:20:24.908 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-03-29T14:20:24.906+0000 7f22b21f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 540687) UID: 0 2023-03-29T14:20:24.908 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-03-29T14:20:24.906+0000 7f93ce7f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 540687) UID: 0 2023-03-29T14:23:23.239 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:23:23.244 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:23:54.002 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:23:54.007 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:24:24.282 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:24:24.286 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:24:54.388 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:24:54.392 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:25:24.511 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:25:24.515 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:25:55.184 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:25:55.188 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:26:25.290 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:26:25.295 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:26:55.428 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:26:55.433 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:27:25.530 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:27:25.534 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:27:55.626 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:27:55.631 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:28:25.740 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:28:25.767 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:28:55.880 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:28:55.885 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:29:25.978 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:29:25.983 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:29:56.105 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:29:56.110 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:30:28.024 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:30:28.029 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:30:58.224 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:30:58.228 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:31:28.336 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:31:28.340 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:31:58.492 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:31:58.496 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:32:28.599 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:32:28.604 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:32:58.705 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:32:58.709 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:33:28.870 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:33:28.875 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:33:58.984 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:33:58.989 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:34:09.266 INFO:teuthology.orchestra.run.smithi111.stdout:generic/522 2238s 2023-03-29T14:34:09.303 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/522 2023-03-29T14:34:09.303 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T14:34:09.308 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:09.330 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T14:34:09.331 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/523) 2023-03-29T14:34:09.869 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T14:34:09.880 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T14:34:09.881 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:10.179 INFO:teuthology.orchestra.run.smithi111.stdout:generic/523 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T14:34:10.187 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/523 2023-03-29T14:34:10.188 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/523 2023-03-29T14:34:10.188 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T14:34:10.193 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:10.213 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T14:34:10.214 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/524) 2023-03-29T14:34:10.598 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T14:34:10.609 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T14:34:10.609 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:10.886 INFO:teuthology.orchestra.run.smithi111.stdout:generic/524 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T14:34:10.894 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/524 2023-03-29T14:34:10.894 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/524 2023-03-29T14:34:10.894 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T14:34:10.899 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:10.919 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T14:34:10.919 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/525) 2023-03-29T14:34:11.301 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T14:34:11.312 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T14:34:11.312 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:11.587 INFO:teuthology.orchestra.run.smithi111.stdout:generic/525 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T14:34:11.595 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/525 2023-03-29T14:34:11.596 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/525 2023-03-29T14:34:11.596 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T14:34:11.601 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:11.621 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T14:34:11.622 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/526) 2023-03-29T14:34:12.004 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T14:34:12.015 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T14:34:12.016 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:12.402 INFO:teuthology.orchestra.run.smithi111.stdout:generic/526 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T14:34:12.410 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/526 2023-03-29T14:34:12.410 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/526 2023-03-29T14:34:12.410 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T14:34:12.415 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:12.436 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T14:34:12.436 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/527) 2023-03-29T14:34:12.821 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T14:34:12.833 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T14:34:12.833 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:13.136 INFO:teuthology.orchestra.run.smithi111.stdout:generic/527 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T14:34:13.144 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/527 2023-03-29T14:34:13.144 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/527 2023-03-29T14:34:13.144 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T14:34:13.149 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:13.170 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T14:34:13.170 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/528) 2023-03-29T14:34:13.551 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T14:34:13.562 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T14:34:13.562 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:13.889 INFO:teuthology.orchestra.run.smithi111.stdout:generic/528 [not run] inode creation time not supported by this filesystem 2023-03-29T14:34:13.897 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/528 2023-03-29T14:34:13.898 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/528 2023-03-29T14:34:13.898 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T14:34:13.903 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:13.924 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T14:34:13.924 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/529) 2023-03-29T14:34:14.304 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T14:34:14.315 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T14:34:14.316 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:14.637 INFO:teuthology.orchestra.run.smithi111.stdout:generic/529 [not run] ACLs not supported by this filesystem type: ceph-fuse 2023-03-29T14:34:14.646 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/529 2023-03-29T14:34:14.646 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/529 2023-03-29T14:34:14.646 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T14:34:14.651 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:14.669 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T14:34:14.670 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/530) 2023-03-29T14:34:15.050 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T14:34:15.061 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T14:34:15.061 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:15.337 INFO:teuthology.orchestra.run.smithi111.stdout:generic/530 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T14:34:15.345 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/530 2023-03-29T14:34:15.345 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/530 2023-03-29T14:34:15.345 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T14:34:15.351 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:15.370 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T14:34:15.371 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/531) 2023-03-29T14:34:15.758 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T14:34:15.770 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T14:34:15.770 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:16.152 INFO:teuthology.orchestra.run.smithi111.stdout:generic/531 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T14:34:16.161 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/531 2023-03-29T14:34:16.161 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/531 2023-03-29T14:34:16.161 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T14:34:16.166 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:16.185 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T14:34:16.186 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/532) 2023-03-29T14:34:16.570 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T14:34:16.582 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T14:34:16.582 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:16.948 INFO:teuthology.orchestra.run.smithi111.stdout:generic/532 0s 2023-03-29T14:34:16.959 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/532 2023-03-29T14:34:16.959 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T14:34:16.964 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:16.985 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T14:34:16.985 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/533) 2023-03-29T14:34:17.372 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T14:34:17.383 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T14:34:17.383 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:18.046 INFO:teuthology.orchestra.run.smithi111.stdout:generic/533 1s 2023-03-29T14:34:18.057 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/533 2023-03-29T14:34:18.058 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T14:34:18.063 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:18.081 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T14:34:18.081 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/534) 2023-03-29T14:34:18.463 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T14:34:18.474 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T14:34:18.474 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:18.774 INFO:teuthology.orchestra.run.smithi111.stdout:generic/534 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T14:34:18.782 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/534 2023-03-29T14:34:18.782 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/534 2023-03-29T14:34:18.782 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T14:34:18.787 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:18.807 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T14:34:18.807 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/535) 2023-03-29T14:34:19.200 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T14:34:19.212 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T14:34:19.212 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:19.516 INFO:teuthology.orchestra.run.smithi111.stdout:generic/535 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T14:34:19.523 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/535 2023-03-29T14:34:19.523 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/535 2023-03-29T14:34:19.523 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T14:34:19.528 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:19.550 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T14:34:19.550 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/536) 2023-03-29T14:34:19.927 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T14:34:19.939 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T14:34:19.939 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:20.207 INFO:teuthology.orchestra.run.smithi111.stdout:generic/536 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T14:34:20.215 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/536 2023-03-29T14:34:20.216 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/536 2023-03-29T14:34:20.216 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T14:34:20.221 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:20.242 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T14:34:20.242 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/537) 2023-03-29T14:34:20.625 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T14:34:20.636 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T14:34:20.636 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:20.910 INFO:teuthology.orchestra.run.smithi111.stdout:generic/537 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T14:34:20.918 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/537 2023-03-29T14:34:20.918 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/537 2023-03-29T14:34:20.918 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T14:34:20.924 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:20.944 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T14:34:20.944 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/538) 2023-03-29T14:34:21.325 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T14:34:21.336 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T14:34:21.337 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T14:34:29.085 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:34:29.090 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:34:59.251 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:34:59.256 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:35:29.365 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:35:29.369 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:35:59.523 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:35:59.527 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:36:29.622 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:36:29.626 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:36:59.726 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:36:59.731 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:37:29.900 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:37:29.905 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:38:00.007 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:38:00.012 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:38:30.302 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:38:30.307 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:39:00.403 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:39:00.407 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:39:30.537 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:39:30.541 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:40:00.842 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:40:00.847 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:40:30.955 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:40:30.960 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:41:01.129 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:41:01.134 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:41:31.251 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:41:31.256 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:42:01.515 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:42:01.519 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:42:31.623 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:42:31.628 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:43:01.742 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:43:01.747 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:43:31.869 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:43:31.873 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:44:02.108 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:44:02.113 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:44:32.690 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:44:32.694 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:45:02.837 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:45:02.842 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:45:33.776 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:45:33.782 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:46:03.885 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:46:03.889 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:46:33.999 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:46:34.005 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:47:04.125 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:47:04.130 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:47:34.981 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:47:34.985 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:48:05.093 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:48:05.099 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:48:35.356 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:48:35.361 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:49:05.470 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:49:05.475 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:49:35.585 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:49:35.592 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:50:05.723 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:50:05.727 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:50:35.827 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:50:35.831 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:51:06.117 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:51:06.121 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:51:36.229 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:51:36.234 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:52:06.369 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:52:06.373 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:52:36.523 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:52:36.528 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:53:06.662 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:53:06.666 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:53:36.801 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:53:36.805 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:54:06.914 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:54:06.931 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:54:37.027 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:54:37.032 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:55:08.029 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:55:08.034 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:55:38.138 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:55:38.142 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:56:08.284 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:56:08.289 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:56:38.547 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:56:38.551 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:57:08.719 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:57:08.724 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:57:38.826 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:57:38.831 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:58:08.966 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:58:08.972 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:58:39.073 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:58:39.077 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:59:09.171 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:59:09.176 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:59:39.269 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T14:59:39.272 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:00:09.366 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:00:09.371 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:00:39.472 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:00:39.477 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:01:09.586 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:01:09.591 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:01:39.719 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:01:39.723 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:02:10.031 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:02:10.035 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:02:40.152 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:02:40.157 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:03:10.803 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:03:10.808 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:03:40.918 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:03:40.922 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:04:11.008 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:04:11.013 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:04:41.106 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:04:41.110 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:05:11.220 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:05:11.225 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:05:41.342 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:05:41.347 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:06:11.455 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:06:11.459 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:06:41.611 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:06:41.616 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:07:11.733 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:07:11.738 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:07:41.834 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:07:41.838 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:08:11.940 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:08:11.944 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:08:42.064 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:08:42.068 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:09:12.337 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:09:12.341 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:09:42.459 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:09:42.463 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:10:12.766 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:10:12.770 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:10:43.806 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:10:43.811 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:11:13.903 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:11:13.907 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:11:44.068 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:11:44.072 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:12:14.176 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:12:14.180 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:12:44.497 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:12:44.501 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:13:14.603 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:13:14.607 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:13:45.742 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:13:45.747 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:14:15.848 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:14:15.853 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:14:45.977 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:14:45.981 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:15:16.082 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:15:16.086 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:15:46.198 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:15:46.203 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:16:16.299 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:16:16.304 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:16:46.447 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:16:46.452 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:17:16.553 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:17:16.557 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:17:46.690 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:17:46.695 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:18:16.838 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:18:16.842 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:18:46.976 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:18:46.980 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:19:18.303 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:19:18.308 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:19:48.404 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:19:48.408 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:20:18.506 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:20:18.511 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:20:48.630 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:20:48.634 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:20:58.035 INFO:teuthology.orchestra.run.smithi111.stdout:generic/538 2797s 2023-03-29T15:20:58.066 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/538 2023-03-29T15:20:58.066 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:20:58.066 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:20:58.076 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:20:58.077 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/539) 2023-03-29T15:20:58.799 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:20:58.810 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:20:58.811 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:01.485 INFO:teuthology.orchestra.run.smithi111.stdout:generic/539 3s 2023-03-29T15:21:01.497 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/539 2023-03-29T15:21:01.497 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:01.502 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:01.521 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:01.521 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/540) 2023-03-29T15:21:02.234 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:02.245 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:02.245 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:02.520 INFO:teuthology.orchestra.run.smithi111.stdout:generic/540 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:02.528 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/540 2023-03-29T15:21:02.529 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/540 2023-03-29T15:21:02.529 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:02.534 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:02.553 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:02.553 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/541) 2023-03-29T15:21:02.934 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:02.945 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:02.945 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:03.220 INFO:teuthology.orchestra.run.smithi111.stdout:generic/541 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:03.227 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/541 2023-03-29T15:21:03.227 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/541 2023-03-29T15:21:03.228 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:03.233 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:03.251 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:03.251 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/542) 2023-03-29T15:21:03.636 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:03.647 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:03.648 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:03.919 INFO:teuthology.orchestra.run.smithi111.stdout:generic/542 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:03.927 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/542 2023-03-29T15:21:03.928 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/542 2023-03-29T15:21:03.928 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:03.933 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:03.952 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:03.952 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/543) 2023-03-29T15:21:04.335 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:04.346 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:04.347 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:04.624 INFO:teuthology.orchestra.run.smithi111.stdout:generic/543 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:04.632 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/543 2023-03-29T15:21:04.632 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/543 2023-03-29T15:21:04.632 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:04.637 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:04.656 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:04.656 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/544) 2023-03-29T15:21:05.038 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:05.049 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:05.049 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:05.332 INFO:teuthology.orchestra.run.smithi111.stdout:generic/544 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:05.340 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/544 2023-03-29T15:21:05.341 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/544 2023-03-29T15:21:05.341 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:05.346 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:05.364 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:05.364 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/545) 2023-03-29T15:21:05.750 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:05.762 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:05.762 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:06.124 INFO:teuthology.orchestra.run.smithi111.stdout:generic/545 [not run] file system doesn't support chattr +i 2023-03-29T15:21:06.132 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/545 2023-03-29T15:21:06.132 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/545 2023-03-29T15:21:06.132 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:06.137 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:06.156 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:06.157 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/546) 2023-03-29T15:21:06.537 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:06.549 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:06.549 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:06.972 INFO:teuthology.orchestra.run.smithi111.stdout:generic/546 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:06.980 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/546 2023-03-29T15:21:06.980 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/546 2023-03-29T15:21:06.980 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:06.985 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:07.004 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:07.004 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/547) 2023-03-29T15:21:07.396 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:07.407 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:07.408 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:07.718 INFO:teuthology.orchestra.run.smithi111.stdout:generic/547 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:07.726 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/547 2023-03-29T15:21:07.726 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/547 2023-03-29T15:21:07.726 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:07.731 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:07.750 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:07.750 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/548) 2023-03-29T15:21:08.132 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:08.143 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:08.143 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:08.483 INFO:teuthology.orchestra.run.smithi111.stdout:generic/548 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:08.491 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/548 2023-03-29T15:21:08.491 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/548 2023-03-29T15:21:08.491 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:08.496 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:08.516 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:08.517 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/549) 2023-03-29T15:21:08.906 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:08.917 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:08.917 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:09.258 INFO:teuthology.orchestra.run.smithi111.stdout:generic/549 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:09.267 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/549 2023-03-29T15:21:09.267 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/549 2023-03-29T15:21:09.267 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:09.272 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:09.291 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:09.291 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/550) 2023-03-29T15:21:09.674 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:09.685 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:09.685 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:10.025 INFO:teuthology.orchestra.run.smithi111.stdout:generic/550 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:10.033 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/550 2023-03-29T15:21:10.034 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/550 2023-03-29T15:21:10.034 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:10.039 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:10.057 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:10.057 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/551) 2023-03-29T15:21:10.440 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:10.451 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:10.451 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:10.724 INFO:teuthology.orchestra.run.smithi111.stdout:generic/551 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:10.731 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/551 2023-03-29T15:21:10.732 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/551 2023-03-29T15:21:10.732 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:10.737 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:10.756 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:10.756 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/552) 2023-03-29T15:21:11.140 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:11.151 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:11.152 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:11.455 INFO:teuthology.orchestra.run.smithi111.stdout:generic/552 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:11.463 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/552 2023-03-29T15:21:11.464 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/552 2023-03-29T15:21:11.464 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:11.469 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:11.488 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:11.488 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/553) 2023-03-29T15:21:11.871 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:11.882 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:11.882 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:12.303 INFO:teuthology.orchestra.run.smithi111.stdout:generic/553 [not run] xfs_io chattr +i failed (old kernel/wrong fs/bad args?) 2023-03-29T15:21:12.311 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/553 2023-03-29T15:21:12.311 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/553 2023-03-29T15:21:12.311 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:12.317 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:12.371 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:12.371 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/554) 2023-03-29T15:21:13.039 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:13.050 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:13.050 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:13.325 INFO:teuthology.orchestra.run.smithi111.stdout:generic/554 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:13.333 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/554 2023-03-29T15:21:13.333 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/554 2023-03-29T15:21:13.333 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:13.338 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:13.357 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:13.359 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/555) 2023-03-29T15:21:13.738 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:13.749 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:13.749 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:14.112 INFO:teuthology.orchestra.run.smithi111.stdout:generic/555 [not run] xfs_io chattr +ia failed (old kernel/wrong fs/bad args?) 2023-03-29T15:21:14.120 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/555 2023-03-29T15:21:14.120 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/555 2023-03-29T15:21:14.121 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:14.126 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:14.179 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:14.179 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/556) 2023-03-29T15:21:14.851 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:14.862 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:14.862 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:15.137 INFO:teuthology.orchestra.run.smithi111.stdout:generic/556 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:15.145 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/556 2023-03-29T15:21:15.145 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/556 2023-03-29T15:21:15.145 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:15.150 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:15.168 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:15.169 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/557) 2023-03-29T15:21:15.548 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:15.560 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:15.560 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:15.864 INFO:teuthology.orchestra.run.smithi111.stdout:generic/557 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:15.872 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/557 2023-03-29T15:21:15.873 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/557 2023-03-29T15:21:15.873 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:15.878 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:15.896 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:15.897 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/558) 2023-03-29T15:21:16.283 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:16.295 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:16.295 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:16.596 INFO:teuthology.orchestra.run.smithi111.stdout:generic/558 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:16.604 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/558 2023-03-29T15:21:16.605 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/558 2023-03-29T15:21:16.605 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:16.610 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:16.628 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:16.628 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/559) 2023-03-29T15:21:17.010 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:17.022 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:17.022 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:17.300 INFO:teuthology.orchestra.run.smithi111.stdout:generic/559 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:17.308 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/559 2023-03-29T15:21:17.308 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/559 2023-03-29T15:21:17.308 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:17.313 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:17.334 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:17.334 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/560) 2023-03-29T15:21:17.717 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:17.728 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:17.728 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:18.012 INFO:teuthology.orchestra.run.smithi111.stdout:generic/560 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:18.020 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/560 2023-03-29T15:21:18.020 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/560 2023-03-29T15:21:18.020 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:18.025 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:18.043 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:18.044 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/561) 2023-03-29T15:21:18.428 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:18.440 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:18.440 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:18.714 INFO:teuthology.orchestra.run.smithi111.stdout:generic/561 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:18.722 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/561 2023-03-29T15:21:18.723 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/561 2023-03-29T15:21:18.723 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:18.728 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:18.746 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:18.747 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/562) 2023-03-29T15:21:18.769 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:21:18.795 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:21:18.891 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-03-29T15:21:18.891+0000 7fd2277f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 130931) UID: 0 2023-03-29T15:21:18.891 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-03-29T15:21:18.891+0000 7fabb15f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 130931) UID: 0 2023-03-29T15:21:18.891 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-03-29T15:21:18.891+0000 7f0c30df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 130931) UID: 0 2023-03-29T15:21:18.892 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-03-29T15:21:18.891+0000 7f9655df9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 130931) UID: 0 2023-03-29T15:21:19.130 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:19.141 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:19.141 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:19.418 INFO:teuthology.orchestra.run.smithi111.stdout:generic/562 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:19.426 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/562 2023-03-29T15:21:19.426 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/562 2023-03-29T15:21:19.426 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:19.432 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:19.450 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:19.450 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/563) 2023-03-29T15:21:19.836 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:19.847 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:19.847 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:20.164 INFO:teuthology.orchestra.run.smithi111.stdout:generic/563 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:20.172 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/563 2023-03-29T15:21:20.172 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/563 2023-03-29T15:21:20.172 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:20.177 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:20.196 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:20.197 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/564) 2023-03-29T15:21:20.578 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:20.589 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:20.589 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:21.473 INFO:teuthology.orchestra.run.smithi111.stdout:generic/564 [not run] xfs_io copy_range doesn't support -f 2023-03-29T15:21:21.492 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/564 2023-03-29T15:21:21.492 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/564 2023-03-29T15:21:21.492 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:21.492 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:21.505 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:21.506 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/565) 2023-03-29T15:21:22.213 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:22.224 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:22.224 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:22.508 INFO:teuthology.orchestra.run.smithi111.stdout:generic/565 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:22.516 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/565 2023-03-29T15:21:22.516 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/565 2023-03-29T15:21:22.516 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:22.522 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:22.541 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:22.542 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/566) 2023-03-29T15:21:22.928 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:22.940 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:22.940 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:23.245 INFO:teuthology.orchestra.run.smithi111.stdout:generic/566 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:23.254 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/566 2023-03-29T15:21:23.254 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/566 2023-03-29T15:21:23.254 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:23.259 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:23.278 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:23.278 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/567) 2023-03-29T15:21:23.660 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:23.671 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:23.671 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:23.947 INFO:teuthology.orchestra.run.smithi111.stdout:generic/567 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:23.955 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/567 2023-03-29T15:21:23.955 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/567 2023-03-29T15:21:23.955 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:23.960 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:23.979 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:23.980 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/568) 2023-03-29T15:21:24.363 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:24.374 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:24.374 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:24.827 INFO:teuthology.orchestra.run.smithi111.stdout:generic/568 0s 2023-03-29T15:21:24.879 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/568 2023-03-29T15:21:24.883 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:24.910 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:24.999 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:24.999 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/569) 2023-03-29T15:21:25.512 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:25.523 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:25.523 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:25.936 INFO:teuthology.orchestra.run.smithi111.stdout:generic/569 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:25.945 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/569 2023-03-29T15:21:25.945 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/569 2023-03-29T15:21:25.945 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:25.950 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:25.969 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:25.970 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/570) 2023-03-29T15:21:26.355 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:26.366 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:26.367 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:26.646 INFO:teuthology.orchestra.run.smithi111.stdout:generic/570 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:26.654 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/570 2023-03-29T15:21:26.654 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/570 2023-03-29T15:21:26.655 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:26.660 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:26.678 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:26.679 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/571) 2023-03-29T15:21:27.063 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:27.074 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:27.075 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:33.567 INFO:teuthology.orchestra.run.smithi111.stdout:generic/571 6s 2023-03-29T15:21:33.580 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/571 2023-03-29T15:21:33.580 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:33.585 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:33.605 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:33.606 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/572) 2023-03-29T15:21:34.312 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:34.323 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:34.323 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:34.599 INFO:teuthology.orchestra.run.smithi111.stdout:generic/572 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:34.607 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/572 2023-03-29T15:21:34.607 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/572 2023-03-29T15:21:34.607 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:34.612 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:34.631 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:34.631 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/573) 2023-03-29T15:21:35.034 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:35.046 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:35.046 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:35.316 INFO:teuthology.orchestra.run.smithi111.stdout:generic/573 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:35.324 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/573 2023-03-29T15:21:35.324 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/573 2023-03-29T15:21:35.324 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:35.330 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:35.349 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:35.349 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/574) 2023-03-29T15:21:35.733 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:35.744 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:35.744 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:36.019 INFO:teuthology.orchestra.run.smithi111.stdout:generic/574 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:36.027 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/574 2023-03-29T15:21:36.027 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/574 2023-03-29T15:21:36.027 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:36.032 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:36.051 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:36.051 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/575) 2023-03-29T15:21:36.438 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:36.449 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:36.449 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:36.733 INFO:teuthology.orchestra.run.smithi111.stdout:generic/575 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:36.742 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/575 2023-03-29T15:21:36.742 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/575 2023-03-29T15:21:36.742 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:36.747 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:36.766 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:36.766 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/576) 2023-03-29T15:21:37.153 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:37.164 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:37.164 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:37.511 INFO:teuthology.orchestra.run.smithi111.stdout:generic/576 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:37.519 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/576 2023-03-29T15:21:37.519 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/576 2023-03-29T15:21:37.519 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:37.524 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:37.543 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:37.544 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/577) 2023-03-29T15:21:37.933 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:37.944 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:37.944 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:38.221 INFO:teuthology.orchestra.run.smithi111.stdout:generic/577 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:38.230 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/577 2023-03-29T15:21:38.230 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/577 2023-03-29T15:21:38.230 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:38.235 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:38.254 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:38.254 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/578) 2023-03-29T15:21:38.657 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:38.668 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:38.669 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:39.043 INFO:teuthology.orchestra.run.smithi111.stdout:generic/578 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T15:21:39.088 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/578 2023-03-29T15:21:39.091 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/578 2023-03-29T15:21:39.091 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:39.091 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:39.096 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:39.096 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/579) 2023-03-29T15:21:39.797 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:39.808 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:39.808 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:40.128 INFO:teuthology.orchestra.run.smithi111.stdout:generic/579 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:40.136 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/579 2023-03-29T15:21:40.136 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/579 2023-03-29T15:21:40.137 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:40.142 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:40.160 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:40.160 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/580) 2023-03-29T15:21:40.555 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:40.566 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:40.566 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:40.906 INFO:teuthology.orchestra.run.smithi111.stdout:generic/580 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:40.915 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/580 2023-03-29T15:21:40.915 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/580 2023-03-29T15:21:40.915 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:40.920 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:40.938 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:40.939 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/581) 2023-03-29T15:21:41.323 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:41.334 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:41.334 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:42.143 INFO:teuthology.orchestra.run.smithi111.stdout:generic/581 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:42.151 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/581 2023-03-29T15:21:42.152 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/581 2023-03-29T15:21:42.152 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:42.157 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:42.176 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:42.176 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/582) 2023-03-29T15:21:42.566 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:42.577 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:42.577 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:42.915 INFO:teuthology.orchestra.run.smithi111.stdout:generic/582 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:42.923 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/582 2023-03-29T15:21:42.923 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/582 2023-03-29T15:21:42.923 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:42.929 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:42.948 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:42.949 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/583) 2023-03-29T15:21:43.330 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:43.341 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:43.341 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:43.718 INFO:teuthology.orchestra.run.smithi111.stdout:generic/583 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:43.726 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/583 2023-03-29T15:21:43.726 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/583 2023-03-29T15:21:43.726 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:43.731 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:43.750 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:43.750 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/584) 2023-03-29T15:21:44.131 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:44.142 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:44.142 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:44.510 INFO:teuthology.orchestra.run.smithi111.stdout:generic/584 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:44.518 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/584 2023-03-29T15:21:44.518 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/584 2023-03-29T15:21:44.518 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:44.524 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:44.542 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:44.543 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/585) 2023-03-29T15:21:44.925 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:44.937 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:44.937 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:45.207 INFO:teuthology.orchestra.run.smithi111.stdout:generic/585 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:21:45.215 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/585 2023-03-29T15:21:45.216 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/585 2023-03-29T15:21:45.216 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:21:45.221 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:21:45.240 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:21:45.240 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/586) 2023-03-29T15:21:45.630 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:21:45.641 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:21:45.641 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:10.668 INFO:teuthology.orchestra.run.smithi111.stdout:generic/586 25s 2023-03-29T15:22:10.680 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/586 2023-03-29T15:22:10.680 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:10.707 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:10.740 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:10.740 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/587) 2023-03-29T15:22:11.433 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:11.444 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:11.444 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:12.019 INFO:teuthology.orchestra.run.smithi111.stdout:generic/587 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2023-03-29T15:22:12.027 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/587 2023-03-29T15:22:12.027 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/587 2023-03-29T15:22:12.027 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:12.032 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:12.051 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:12.051 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/588) 2023-03-29T15:22:12.437 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:12.448 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:12.448 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:12.825 INFO:teuthology.orchestra.run.smithi111.stdout:generic/588 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:12.833 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/588 2023-03-29T15:22:12.833 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/588 2023-03-29T15:22:12.833 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:12.838 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:12.857 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:12.857 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/589) 2023-03-29T15:22:13.239 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:13.250 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:13.250 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:13.534 INFO:teuthology.orchestra.run.smithi111.stdout:generic/589 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:13.542 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/589 2023-03-29T15:22:13.542 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/589 2023-03-29T15:22:13.543 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:13.547 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:13.566 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:13.567 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/590) 2023-03-29T15:22:13.947 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:13.958 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:13.959 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:14.233 INFO:teuthology.orchestra.run.smithi111.stdout:generic/590 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:14.241 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/590 2023-03-29T15:22:14.242 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/590 2023-03-29T15:22:14.242 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:14.247 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:14.266 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:14.266 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/591) 2023-03-29T15:22:14.651 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:14.663 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:14.663 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:15.153 INFO:teuthology.orchestra.run.smithi111.stdout:generic/591 1s 2023-03-29T15:22:15.164 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/591 2023-03-29T15:22:15.164 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:15.169 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:15.188 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:15.189 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/592) 2023-03-29T15:22:15.900 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:15.911 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:15.911 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:16.251 INFO:teuthology.orchestra.run.smithi111.stdout:generic/592 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:16.259 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/592 2023-03-29T15:22:16.259 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/592 2023-03-29T15:22:16.259 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:16.264 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:16.283 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:16.283 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/593) 2023-03-29T15:22:16.672 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:16.683 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:16.683 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:17.038 INFO:teuthology.orchestra.run.smithi111.stdout:generic/593 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:17.047 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/593 2023-03-29T15:22:17.047 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/593 2023-03-29T15:22:17.047 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:17.052 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:17.071 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:17.071 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/594) 2023-03-29T15:22:17.456 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:17.468 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:17.468 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:17.749 INFO:teuthology.orchestra.run.smithi111.stdout:generic/594 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:17.757 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/594 2023-03-29T15:22:17.761 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/594 2023-03-29T15:22:17.761 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:17.762 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:17.781 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:17.781 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/595) 2023-03-29T15:22:18.168 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:18.179 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:18.180 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:18.522 INFO:teuthology.orchestra.run.smithi111.stdout:generic/595 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:18.530 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/595 2023-03-29T15:22:18.530 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/595 2023-03-29T15:22:18.530 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:18.536 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:18.556 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:18.556 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/596) 2023-03-29T15:22:18.940 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:18.951 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:18.951 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:19.320 INFO:teuthology.orchestra.run.smithi111.stdout:generic/596 [not run] file system doesn't support chattr +S 2023-03-29T15:22:19.328 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/596 2023-03-29T15:22:19.328 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/596 2023-03-29T15:22:19.328 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:19.334 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:19.353 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:19.353 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/597) 2023-03-29T15:22:19.747 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:19.759 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:19.759 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:20.712 INFO:teuthology.orchestra.run.smithi111.stdout:generic/597 - output mismatch (see /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/597.out.bad) 2023-03-29T15:22:20.717 INFO:teuthology.orchestra.run.smithi111.stdout: --- tests/generic/597.out 2023-03-29 10:59:21.821745953 +0000 2023-03-29T15:22:20.717 INFO:teuthology.orchestra.run.smithi111.stdout: +++ /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/597.out.bad 2023-03-29 15:22:20.709615325 +0000 2023-03-29T15:22:20.718 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -4,11 +4,11 @@ 2023-03-29T15:22:20.718 INFO:teuthology.orchestra.run.smithi111.stdout: fs.protected_symlinks = 0 2023-03-29T15:22:20.718 INFO:teuthology.orchestra.run.smithi111.stdout: successfully followed symlink 2023-03-29T15:22:20.718 INFO:teuthology.orchestra.run.smithi111.stdout: fs.protected_symlinks = 1 2023-03-29T15:22:20.718 INFO:teuthology.orchestra.run.smithi111.stdout: -Permission denied 2023-03-29T15:22:20.718 INFO:teuthology.orchestra.run.smithi111.stdout: +successfully followed symlink 2023-03-29T15:22:20.718 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:20.718 INFO:teuthology.orchestra.run.smithi111.stdout: == Test hardlink create protection when 2023-03-29T15:22:20.718 INFO:teuthology.orchestra.run.smithi111.stdout: == process != target owner and process cannot read target 2023-03-29T15:22:20.718 INFO:teuthology.orchestra.run.smithi111.stdout: fs.protected_hardlinks = 0 2023-03-29T15:22:20.718 INFO:teuthology.orchestra.run.smithi111.stdout: -successfully created hardlink 2023-03-29T15:22:20.718 INFO:teuthology.orchestra.run.smithi111.stdout: +Permission denied 2023-03-29T15:22:20.718 INFO:teuthology.orchestra.run.smithi111.stdout: fs.protected_hardlinks = 1 2023-03-29T15:22:20.718 INFO:teuthology.orchestra.run.smithi111.stdout: -ln: failed to create hard link 'TEST_DIR/597/sticky_dir/hardlink' => 'TEST_DIR/597/target': Operation not permitted 2023-03-29T15:22:20.718 INFO:teuthology.orchestra.run.smithi111.stdout: +Permission denied 2023-03-29T15:22:20.723 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/597 2023-03-29T15:22:20.723 INFO:teuthology.orchestra.run.smithi111.stdout:Failures: generic/597 2023-03-29T15:22:20.723 INFO:teuthology.orchestra.run.smithi111.stdout:Failed 1 of 1 tests 2023-03-29T15:22:20.728 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:20.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T15:22:20.747 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-03-29T15:22:20.747 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-03-29T15:22:20.748 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/598) 2023-03-29T15:22:21.459 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:21.471 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:21.471 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:22.632 INFO:teuthology.orchestra.run.smithi111.stdout:generic/598 - output mismatch (see /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/598.out.bad) 2023-03-29T15:22:22.637 INFO:teuthology.orchestra.run.smithi111.stdout: --- tests/generic/598.out 2023-03-29 10:59:21.822745936 +0000 2023-03-29T15:22:22.637 INFO:teuthology.orchestra.run.smithi111.stdout: +++ /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/598.out.bad 2023-03-29 15:22:22.629583577 +0000 2023-03-29T15:22:22.637 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -5,13 +5,10 @@ 2023-03-29T15:22:22.637 INFO:teuthology.orchestra.run.smithi111.stdout: = only group writable dir 2023-03-29T15:22:22.637 INFO:teuthology.orchestra.run.smithi111.stdout: fs.protected_regular = 1 2023-03-29T15:22:22.637 INFO:teuthology.orchestra.run.smithi111.stdout: = group & world writable dir 2023-03-29T15:22:22.637 INFO:teuthology.orchestra.run.smithi111.stdout: -Permission denied 2023-03-29T15:22:22.637 INFO:teuthology.orchestra.run.smithi111.stdout: = only group writable dir 2023-03-29T15:22:22.638 INFO:teuthology.orchestra.run.smithi111.stdout: fs.protected_regular = 2 2023-03-29T15:22:22.638 INFO:teuthology.orchestra.run.smithi111.stdout: = group & world writable dir 2023-03-29T15:22:22.638 INFO:teuthology.orchestra.run.smithi111.stdout: -Permission denied 2023-03-29T15:22:22.638 INFO:teuthology.orchestra.run.smithi111.stdout: = only group writable dir 2023-03-29T15:22:22.638 INFO:teuthology.orchestra.run.smithi111.stdout: -Permission denied 2023-03-29T15:22:22.638 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:22.638 INFO:teuthology.orchestra.run.smithi111.stdout: == Test fifo open when owned by another and fifo owner != dir owner 2023-03-29T15:22:22.638 INFO:teuthology.orchestra.run.smithi111.stdout: fs.protected_fifos = 0 2023-03-29T15:22:22.638 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -19,10 +16,7 @@ 2023-03-29T15:22:22.638 INFO:teuthology.orchestra.run.smithi111.stdout: = only group writable dir 2023-03-29T15:22:22.638 INFO:teuthology.orchestra.run.smithi111.stdout: fs.protected_fifos = 1 2023-03-29T15:22:22.638 INFO:teuthology.orchestra.run.smithi111.stdout: = group & world writable dir 2023-03-29T15:22:22.638 INFO:teuthology.orchestra.run.smithi111.stdout: -Permission denied 2023-03-29T15:22:22.638 INFO:teuthology.orchestra.run.smithi111.stdout: = only group writable dir 2023-03-29T15:22:22.638 INFO:teuthology.orchestra.run.smithi111.stdout: fs.protected_fifos = 2 2023-03-29T15:22:22.638 INFO:teuthology.orchestra.run.smithi111.stdout: = group & world writable dir 2023-03-29T15:22:22.638 INFO:teuthology.orchestra.run.smithi111.stdout: -Permission denied 2023-03-29T15:22:22.638 INFO:teuthology.orchestra.run.smithi111.stdout: = only group writable dir 2023-03-29T15:22:22.639 INFO:teuthology.orchestra.run.smithi111.stdout: -Permission denied 2023-03-29T15:22:22.642 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/598 2023-03-29T15:22:22.642 INFO:teuthology.orchestra.run.smithi111.stdout:Failures: generic/598 2023-03-29T15:22:22.642 INFO:teuthology.orchestra.run.smithi111.stdout:Failed 1 of 1 tests 2023-03-29T15:22:22.648 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:22.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T15:22:22.667 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-03-29T15:22:22.667 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-03-29T15:22:22.667 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/599) 2023-03-29T15:22:23.395 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:23.406 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:23.406 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:23.693 INFO:teuthology.orchestra.run.smithi111.stdout:generic/599 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:23.701 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/599 2023-03-29T15:22:23.701 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/599 2023-03-29T15:22:23.701 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:23.706 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:23.726 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:23.726 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/600) 2023-03-29T15:22:24.107 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:24.118 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:24.118 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:24.397 INFO:teuthology.orchestra.run.smithi111.stdout:generic/600 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:24.405 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/600 2023-03-29T15:22:24.405 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/600 2023-03-29T15:22:24.405 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:24.410 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:24.429 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:24.429 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/601) 2023-03-29T15:22:24.818 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:24.829 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:24.829 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:25.104 INFO:teuthology.orchestra.run.smithi111.stdout:generic/601 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:25.113 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/601 2023-03-29T15:22:25.113 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/601 2023-03-29T15:22:25.113 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:25.118 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:25.137 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:25.138 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/602) 2023-03-29T15:22:25.526 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:25.538 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:25.538 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:25.887 INFO:teuthology.orchestra.run.smithi111.stdout:generic/602 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:25.895 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/602 2023-03-29T15:22:25.895 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/602 2023-03-29T15:22:25.895 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:25.900 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:25.919 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:25.920 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/603) 2023-03-29T15:22:26.304 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:26.315 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:26.315 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:26.608 INFO:teuthology.orchestra.run.smithi111.stdout:generic/603 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:26.616 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/603 2023-03-29T15:22:26.616 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/603 2023-03-29T15:22:26.616 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:26.621 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:26.641 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:26.641 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/604) 2023-03-29T15:22:27.023 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:27.034 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:27.034 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:27.318 INFO:teuthology.orchestra.run.smithi111.stdout:generic/604 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:27.326 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/604 2023-03-29T15:22:27.326 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/604 2023-03-29T15:22:27.326 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:27.332 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:27.351 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:27.351 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/605) 2023-03-29T15:22:27.736 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:27.748 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:27.748 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:28.023 INFO:teuthology.orchestra.run.smithi111.stdout:generic/605 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:28.031 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/605 2023-03-29T15:22:28.031 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/605 2023-03-29T15:22:28.031 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:28.036 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:28.055 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:28.055 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/606) 2023-03-29T15:22:28.440 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:28.451 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:28.451 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:28.727 INFO:teuthology.orchestra.run.smithi111.stdout:generic/606 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:28.735 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/606 2023-03-29T15:22:28.735 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/606 2023-03-29T15:22:28.735 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:28.741 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:28.761 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:28.761 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/607) 2023-03-29T15:22:29.153 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:29.164 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:29.164 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:29.435 INFO:teuthology.orchestra.run.smithi111.stdout:generic/607 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:29.443 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/607 2023-03-29T15:22:29.443 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/607 2023-03-29T15:22:29.443 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:29.449 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:29.468 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:29.468 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/608) 2023-03-29T15:22:29.855 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:29.866 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:29.866 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:30.141 INFO:teuthology.orchestra.run.smithi111.stdout:generic/608 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:30.149 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/608 2023-03-29T15:22:30.150 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/608 2023-03-29T15:22:30.150 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:30.155 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:30.174 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:30.174 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/609) 2023-03-29T15:22:30.555 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:30.567 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:30.567 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:31.204 INFO:teuthology.orchestra.run.smithi111.stdout:generic/609 1s 2023-03-29T15:22:31.215 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/609 2023-03-29T15:22:31.215 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:31.220 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:31.239 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:31.239 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/610) 2023-03-29T15:22:31.957 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:31.968 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:31.968 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:32.244 INFO:teuthology.orchestra.run.smithi111.stdout:generic/610 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:32.252 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/610 2023-03-29T15:22:32.252 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/610 2023-03-29T15:22:32.252 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:32.257 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:32.276 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:32.276 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/611) 2023-03-29T15:22:32.667 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:32.678 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:32.678 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:32.964 INFO:teuthology.orchestra.run.smithi111.stdout:generic/611 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:32.972 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/611 2023-03-29T15:22:32.973 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/611 2023-03-29T15:22:32.973 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:32.978 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:32.996 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:32.996 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/612) 2023-03-29T15:22:33.382 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:33.393 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:33.393 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:33.754 INFO:teuthology.orchestra.run.smithi111.stdout:generic/612 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T15:22:33.762 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/612 2023-03-29T15:22:33.762 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/612 2023-03-29T15:22:33.762 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:33.768 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:33.815 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:33.816 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/613) 2023-03-29T15:22:34.494 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:34.506 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:34.506 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:34.845 INFO:teuthology.orchestra.run.smithi111.stdout:generic/613 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:34.853 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/613 2023-03-29T15:22:34.853 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/613 2023-03-29T15:22:34.853 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:34.858 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:34.877 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:34.877 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/614) 2023-03-29T15:22:35.265 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:35.276 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:35.276 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:35.545 INFO:teuthology.orchestra.run.smithi111.stdout:generic/614 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:35.553 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/614 2023-03-29T15:22:35.554 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/614 2023-03-29T15:22:35.554 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:35.559 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:35.578 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:35.578 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/615) 2023-03-29T15:22:35.963 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:35.974 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:35.974 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:36.300 INFO:teuthology.orchestra.run.smithi111.stdout:generic/615 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:36.308 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/615 2023-03-29T15:22:36.308 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/615 2023-03-29T15:22:36.308 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:36.313 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:36.332 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:36.332 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/616) 2023-03-29T15:22:36.709 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:36.720 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:36.720 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:37.020 INFO:teuthology.orchestra.run.smithi111.stdout:generic/616 [not run] kernel does not support IO_URING 2023-03-29T15:22:37.028 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/616 2023-03-29T15:22:37.028 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/616 2023-03-29T15:22:37.028 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:37.034 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:37.052 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:37.052 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/617) 2023-03-29T15:22:37.438 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:37.449 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:37.449 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:37.793 INFO:teuthology.orchestra.run.smithi111.stdout:generic/617 [not run] kernel does not support IO_URING 2023-03-29T15:22:37.801 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/617 2023-03-29T15:22:37.802 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/617 2023-03-29T15:22:37.802 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:37.807 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:37.825 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:37.826 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/618) 2023-03-29T15:22:38.208 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:38.219 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:38.219 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:38.501 INFO:teuthology.orchestra.run.smithi111.stdout:generic/618 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:38.509 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/618 2023-03-29T15:22:38.509 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/618 2023-03-29T15:22:38.509 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:38.514 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:38.532 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:38.533 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/619) 2023-03-29T15:22:38.911 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:38.922 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:38.923 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:39.195 INFO:teuthology.orchestra.run.smithi111.stdout:generic/619 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:39.203 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/619 2023-03-29T15:22:39.203 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/619 2023-03-29T15:22:39.203 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:39.208 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:39.227 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:39.227 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/620) 2023-03-29T15:22:39.626 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:39.637 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:39.637 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:40.021 INFO:teuthology.orchestra.run.smithi111.stdout:generic/620 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:40.029 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/620 2023-03-29T15:22:40.029 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/620 2023-03-29T15:22:40.029 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:40.034 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:40.053 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:40.053 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/621) 2023-03-29T15:22:40.443 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:40.454 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:40.455 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:40.805 INFO:teuthology.orchestra.run.smithi111.stdout:generic/621 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:40.813 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/621 2023-03-29T15:22:40.813 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/621 2023-03-29T15:22:40.813 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:40.818 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:40.837 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:40.838 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/622) 2023-03-29T15:22:41.219 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:41.230 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:41.230 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:41.513 INFO:teuthology.orchestra.run.smithi111.stdout:generic/622 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:41.521 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/622 2023-03-29T15:22:41.522 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/622 2023-03-29T15:22:41.522 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:41.527 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:41.545 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:41.546 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/623) 2023-03-29T15:22:41.932 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:41.943 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:41.943 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:42.224 INFO:teuthology.orchestra.run.smithi111.stdout:generic/623 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:42.232 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/623 2023-03-29T15:22:42.233 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/623 2023-03-29T15:22:42.233 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:42.238 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:42.257 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:42.257 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/624) 2023-03-29T15:22:42.649 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:42.660 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:42.661 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:42.935 INFO:teuthology.orchestra.run.smithi111.stdout:generic/624 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:42.943 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/624 2023-03-29T15:22:42.943 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/624 2023-03-29T15:22:42.943 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:42.948 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:42.967 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:42.967 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/625) 2023-03-29T15:22:43.351 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:43.362 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:43.362 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:43.636 INFO:teuthology.orchestra.run.smithi111.stdout:generic/625 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:43.644 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/625 2023-03-29T15:22:43.645 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/625 2023-03-29T15:22:43.645 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:43.650 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:43.669 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:43.669 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/626) 2023-03-29T15:22:44.048 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:44.059 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:44.059 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:44.337 INFO:teuthology.orchestra.run.smithi111.stdout:generic/626 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:44.345 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/626 2023-03-29T15:22:44.345 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/626 2023-03-29T15:22:44.345 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:44.350 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:44.369 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:44.369 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/627) 2023-03-29T15:22:44.750 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:44.761 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:44.761 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:45.044 INFO:teuthology.orchestra.run.smithi111.stdout:generic/627 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:45.052 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/627 2023-03-29T15:22:45.052 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/627 2023-03-29T15:22:45.052 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:45.057 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:45.076 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:45.076 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/628) 2023-03-29T15:22:45.462 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:45.473 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:45.473 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:45.786 INFO:teuthology.orchestra.run.smithi111.stdout:generic/628 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:45.794 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/628 2023-03-29T15:22:45.794 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/628 2023-03-29T15:22:45.794 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:45.799 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:45.819 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:45.819 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/629) 2023-03-29T15:22:46.199 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:46.210 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:46.210 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:46.515 INFO:teuthology.orchestra.run.smithi111.stdout:generic/629 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:46.523 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/629 2023-03-29T15:22:46.523 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/629 2023-03-29T15:22:46.523 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:46.528 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:46.546 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:46.546 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/630) 2023-03-29T15:22:46.928 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:46.939 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:46.939 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:47.212 INFO:teuthology.orchestra.run.smithi111.stdout:generic/630 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:47.220 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/630 2023-03-29T15:22:47.220 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/630 2023-03-29T15:22:47.220 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:47.225 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:47.245 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:47.245 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/631) 2023-03-29T15:22:47.633 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:47.644 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:47.645 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:47.920 INFO:teuthology.orchestra.run.smithi111.stdout:generic/631 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:47.928 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/631 2023-03-29T15:22:47.929 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/631 2023-03-29T15:22:47.929 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:47.934 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:47.952 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:47.953 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/632) 2023-03-29T15:22:48.337 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:48.349 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:48.349 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:54.480 INFO:teuthology.orchestra.run.smithi111.stdout:generic/632 6s 2023-03-29T15:22:54.491 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/632 2023-03-29T15:22:54.492 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:54.497 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:54.516 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:54.516 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/633) 2023-03-29T15:22:54.898 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:54.909 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:54.909 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:55.578 INFO:teuthology.orchestra.run.smithi111.stdout:generic/633 [failed, exit status 1]- output mismatch (see /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/633.out.bad) 2023-03-29T15:22:55.623 INFO:teuthology.orchestra.run.smithi111.stdout: --- tests/generic/633.out 2023-03-29 10:59:21.826745871 +0000 2023-03-29T15:22:55.623 INFO:teuthology.orchestra.run.smithi111.stdout: +++ /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/633.out.bad 2023-03-29 15:22:55.565038951 +0000 2023-03-29T15:22:55.623 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -1,2 +1,3 @@ 2023-03-29T15:22:55.623 INFO:teuthology.orchestra.run.smithi111.stdout: QA output created by 633 2023-03-29T15:22:55.623 INFO:teuthology.orchestra.run.smithi111.stdout: Silence is golden 2023-03-29T15:22:55.623 INFO:teuthology.orchestra.run.smithi111.stdout: +vfstest.c: 2418: run_test - Success - failure: setid binaries on regular mounts 2023-03-29T15:22:55.650 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/633 2023-03-29T15:22:55.650 INFO:teuthology.orchestra.run.smithi111.stdout:Failures: generic/633 2023-03-29T15:22:55.650 INFO:teuthology.orchestra.run.smithi111.stdout:Failed 1 of 1 tests 2023-03-29T15:22:55.667 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:55.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T15:22:55.719 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-03-29T15:22:55.719 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-03-29T15:22:55.719 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/634) 2023-03-29T15:22:56.258 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:56.269 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:56.269 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:56.562 INFO:teuthology.orchestra.run.smithi111.stdout:generic/634 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:56.570 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/634 2023-03-29T15:22:56.570 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/634 2023-03-29T15:22:56.570 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:56.575 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:56.595 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:56.595 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/635) 2023-03-29T15:22:56.977 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:56.988 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:56.988 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:57.343 INFO:teuthology.orchestra.run.smithi111.stdout:generic/635 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:57.352 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/635 2023-03-29T15:22:57.352 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/635 2023-03-29T15:22:57.352 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:57.357 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:57.376 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:57.376 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/636) 2023-03-29T15:22:57.757 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:57.768 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:57.768 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:58.037 INFO:teuthology.orchestra.run.smithi111.stdout:generic/636 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:22:58.045 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/636 2023-03-29T15:22:58.045 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/636 2023-03-29T15:22:58.045 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:22:58.051 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:22:58.069 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:22:58.070 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/637) 2023-03-29T15:22:58.449 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:22:58.460 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:22:58.460 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:00.069 INFO:teuthology.orchestra.run.smithi111.stdout:generic/637 2s 2023-03-29T15:23:00.081 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/637 2023-03-29T15:23:00.081 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:23:00.086 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:00.110 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:23:00.110 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/638) 2023-03-29T15:23:00.820 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:23:00.831 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:23:00.832 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:01.198 INFO:teuthology.orchestra.run.smithi111.stdout:generic/638 1s 2023-03-29T15:23:01.210 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/638 2023-03-29T15:23:01.210 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:23:01.215 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:01.234 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:23:01.234 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/639) 2023-03-29T15:23:01.621 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:23:01.632 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:23:01.632 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:02.353 INFO:teuthology.orchestra.run.smithi111.stdout:generic/639 - output mismatch (see /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/639.out.bad) 2023-03-29T15:23:02.358 INFO:teuthology.orchestra.run.smithi111.stdout: --- tests/generic/639.out 2023-03-29 10:59:21.827745855 +0000 2023-03-29T15:23:02.358 INFO:teuthology.orchestra.run.smithi111.stdout: +++ /tmp/tmp.CzuMqE4LuJxfstests-dev/results//generic/639.out.bad 2023-03-29 15:23:02.350926737 +0000 2023-03-29T15:23:02.358 INFO:teuthology.orchestra.run.smithi111.stdout: @@ -1,4 +1,5 @@ 2023-03-29T15:23:02.358 INFO:teuthology.orchestra.run.smithi111.stdout: QA output created by 639 2023-03-29T15:23:02.358 INFO:teuthology.orchestra.run.smithi111.stdout: +umount: /tmp/tmp.6dtVtxiLHFtest: target is busy. 2023-03-29T15:23:02.358 INFO:teuthology.orchestra.run.smithi111.stdout: The result should be 64 bytes filled with 0xcd: 2023-03-29T15:23:02.358 INFO:teuthology.orchestra.run.smithi111.stdout: 000000 cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd cd >................< 2023-03-29T15:23:02.358 INFO:teuthology.orchestra.run.smithi111.stdout: * 2023-03-29T15:23:02.363 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/639 2023-03-29T15:23:02.363 INFO:teuthology.orchestra.run.smithi111.stdout:Failures: generic/639 2023-03-29T15:23:02.363 INFO:teuthology.orchestra.run.smithi111.stdout:Failed 1 of 1 tests 2023-03-29T15:23:02.369 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:02.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T15:23:02.387 INFO:tasks.cephfs.xfstests_dev:Command failed 2023-03-29T15:23:02.387 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2023-03-29T15:23:02.387 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/640) 2023-03-29T15:23:02.770 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:23:02.781 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:23:02.782 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:03.160 INFO:teuthology.orchestra.run.smithi111.stdout:generic/640 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:23:03.168 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/640 2023-03-29T15:23:03.168 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/640 2023-03-29T15:23:03.168 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:23:03.174 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:03.193 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:23:03.194 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/641) 2023-03-29T15:23:03.571 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:23:03.582 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:23:03.582 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:03.854 INFO:teuthology.orchestra.run.smithi111.stdout:generic/641 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:23:03.862 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/641 2023-03-29T15:23:03.862 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/641 2023-03-29T15:23:03.862 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:23:03.868 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:03.886 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:23:03.887 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/642) 2023-03-29T15:23:04.268 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:23:04.278 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:23:04.279 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:04.590 INFO:teuthology.orchestra.run.smithi111.stdout:generic/642 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:23:04.598 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/642 2023-03-29T15:23:04.599 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/642 2023-03-29T15:23:04.599 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:23:04.604 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:04.623 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:23:04.623 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/643) 2023-03-29T15:23:05.016 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:23:05.027 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:23:05.027 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:05.296 INFO:teuthology.orchestra.run.smithi111.stdout:generic/643 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:23:05.304 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/643 2023-03-29T15:23:05.305 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/643 2023-03-29T15:23:05.305 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:23:05.310 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:05.328 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:23:05.328 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/644) 2023-03-29T15:23:05.715 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:23:05.726 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:23:05.726 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:06.037 INFO:teuthology.orchestra.run.smithi111.stdout:generic/644 [not run] vfstest not support by ceph-fuse 2023-03-29T15:23:06.045 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/644 2023-03-29T15:23:06.045 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/644 2023-03-29T15:23:06.045 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:23:06.051 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:06.069 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:23:06.070 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/645) 2023-03-29T15:23:06.453 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:23:06.464 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:23:06.464 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:06.746 INFO:teuthology.orchestra.run.smithi111.stdout:generic/645 [not run] vfstest not support by ceph-fuse 2023-03-29T15:23:06.754 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/645 2023-03-29T15:23:06.754 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/645 2023-03-29T15:23:06.755 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:23:06.760 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:06.779 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:23:06.779 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/646) 2023-03-29T15:23:07.161 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:23:07.173 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:23:07.173 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:07.441 INFO:teuthology.orchestra.run.smithi111.stdout:generic/646 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:23:07.450 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/646 2023-03-29T15:23:07.450 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/646 2023-03-29T15:23:07.450 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:23:07.455 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:07.474 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:23:07.474 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/647) 2023-03-29T15:23:07.862 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:23:07.873 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:23:07.874 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:08.369 INFO:teuthology.orchestra.run.smithi111.stdout:generic/647 1s 2023-03-29T15:23:08.380 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/647 2023-03-29T15:23:08.380 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:23:08.386 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:08.404 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:23:08.405 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/648) 2023-03-29T15:23:09.099 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:23:09.110 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:23:09.110 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:09.433 INFO:teuthology.orchestra.run.smithi111.stdout:generic/648 [not run] this test requires a valid $SCRATCH_DEV 2023-03-29T15:23:09.441 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/648 2023-03-29T15:23:09.441 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/648 2023-03-29T15:23:09.442 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:23:09.447 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:09.466 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:23:09.466 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/649) 2023-03-29T15:23:09.848 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:23:09.859 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:23:09.859 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:10.225 INFO:teuthology.orchestra.run.smithi111.stdout:generic/649 [not run] Reflink not supported by test filesystem type: ceph-fuse 2023-03-29T15:23:10.233 INFO:teuthology.orchestra.run.smithi111.stdout:Ran: generic/649 2023-03-29T15:23:10.233 INFO:teuthology.orchestra.run.smithi111.stdout:Not run: generic/649 2023-03-29T15:23:10.234 INFO:teuthology.orchestra.run.smithi111.stdout:Passed all 1 tests 2023-03-29T15:23:10.239 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:23:10.258 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2023-03-29T15:23:10.259 DEBUG:teuthology.orchestra.run.smithi111:> (cd /tmp/tmp.CzuMqE4LuJxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/650) 2023-03-29T15:23:10.952 INFO:teuthology.orchestra.run.smithi111.stdout:FSTYP -- ceph-fuse 2023-03-29T15:23:10.963 INFO:teuthology.orchestra.run.smithi111.stdout:PLATFORM -- Linux/x86_64 smithi111 6.3.0-rc3-g1776e8f494e9 #1 SMP PREEMPT_DYNAMIC Mon Mar 27 03:45:47 UTC 2023 2023-03-29T15:23:10.964 INFO:teuthology.orchestra.run.smithi111.stdout: 2023-03-29T15:24:10.806 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:24:10.811 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:24:11.096 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-03-29T15:24:11.095+0000 7f93ce7f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 610628) UID: 0 2023-03-29T15:24:11.097 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-03-29T15:24:11.096+0000 7f22b21f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 610628) UID: 0 2023-03-29T15:24:11.097 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T15:24:11.097+0000 7f7a699f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 610628) UID: 0 2023-03-29T15:24:11.099 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-03-29T15:24:11.099+0000 7f046aff9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 610628) UID: 0 2023-03-29T15:27:59.310 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-03-29T15:27:59.308+0000 7f0459feb700 -1 bdev(0x560ff6a1c000 /var/lib/ceph/osd/ceph-1/block) _sync_write pwritev error: (5) Input/output error 2023-03-29T15:27:59.310 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-03-29T15:27:59.308+0000 7f04607f8700 -1 bdev(0x560ff6a1c000 /var/lib/ceph/osd/ceph-1/block) _aio_thread got r=-5 ((5) Input/output error) 2023-03-29T15:27:59.353 INFO:tasks.ceph.osd.1.smithi111.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-3066-gf03d69a8/rpm/el8/BUILD/ceph-18.0.0-3066-gf03d69a8/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f04607f8700 time 2023-03-29T15:27:59.338545+0000 2023-03-29T15:27:59.353 INFO:tasks.ceph.osd.1.smithi111.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-3066-gf03d69a8/rpm/el8/BUILD/ceph-18.0.0-3066-gf03d69a8/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2023-03-29T15:27:59.363 INFO:tasks.ceph.osd.1.smithi111.stderr: ceph version 18.0.0-3066-gf03d69a8 (f03d69a8741c605ae6711afe2c9991ac46095f7d) reef (dev) 2023-03-29T15:27:59.363 INFO:tasks.ceph.osd.1.smithi111.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x560ff206accb] 2023-03-29T15:27:59.363 INFO:tasks.ceph.osd.1.smithi111.stderr: 2: (KernelDevice::_aio_thread()+0x1798) [0x560ff2c7a908] 2023-03-29T15:27:59.363 INFO:tasks.ceph.osd.1.smithi111.stderr: 3: (KernelDevice::AioCompletionThread::entry()+0x11) [0x560ff2c7cfa1] 2023-03-29T15:27:59.363 INFO:tasks.ceph.osd.1.smithi111.stderr: 4: /lib64/libpthread.so.0(+0x81ca) [0x7f04774081ca] 2023-03-29T15:27:59.363 INFO:tasks.ceph.osd.1.smithi111.stderr: 5: clone() 2023-03-29T15:27:59.364 INFO:tasks.ceph.osd.1.smithi111.stderr:*** Caught signal (Aborted) ** 2023-03-29T15:27:59.364 INFO:tasks.ceph.osd.1.smithi111.stderr: in thread 7f04607f8700 thread_name:bstore_aio 2023-03-29T15:27:59.364 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-03-29T15:27:59.364+0000 7f04607f8700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-3066-gf03d69a8/rpm/el8/BUILD/ceph-18.0.0-3066-gf03d69a8/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f04607f8700 time 2023-03-29T15:27:59.338545+0000 2023-03-29T15:27:59.364 INFO:tasks.ceph.osd.1.smithi111.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-3066-gf03d69a8/rpm/el8/BUILD/ceph-18.0.0-3066-gf03d69a8/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2023-03-29T15:27:59.364 INFO:tasks.ceph.osd.1.smithi111.stderr: 2023-03-29T15:27:59.364 INFO:tasks.ceph.osd.1.smithi111.stderr: ceph version 18.0.0-3066-gf03d69a8 (f03d69a8741c605ae6711afe2c9991ac46095f7d) reef (dev) 2023-03-29T15:27:59.364 INFO:tasks.ceph.osd.1.smithi111.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x560ff206accb] 2023-03-29T15:27:59.364 INFO:tasks.ceph.osd.1.smithi111.stderr: 2: (KernelDevice::_aio_thread()+0x1798) [0x560ff2c7a908] 2023-03-29T15:27:59.364 INFO:tasks.ceph.osd.1.smithi111.stderr: 3: (KernelDevice::AioCompletionThread::entry()+0x11) [0x560ff2c7cfa1] 2023-03-29T15:27:59.365 INFO:tasks.ceph.osd.1.smithi111.stderr: 4: /lib64/libpthread.so.0(+0x81ca) [0x7f04774081ca] 2023-03-29T15:27:59.365 INFO:tasks.ceph.osd.1.smithi111.stderr: 5: clone() 2023-03-29T15:27:59.365 INFO:tasks.ceph.osd.1.smithi111.stderr: 2023-03-29T15:27:59.382 INFO:tasks.ceph.osd.1.smithi111.stderr: ceph version 18.0.0-3066-gf03d69a8 (f03d69a8741c605ae6711afe2c9991ac46095f7d) reef (dev) 2023-03-29T15:27:59.382 INFO:tasks.ceph.osd.1.smithi111.stderr: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f0477412cf0] 2023-03-29T15:27:59.382 INFO:tasks.ceph.osd.1.smithi111.stderr: 2: gsignal() 2023-03-29T15:27:59.383 INFO:tasks.ceph.osd.1.smithi111.stderr: 3: abort() 2023-03-29T15:27:59.383 INFO:tasks.ceph.osd.1.smithi111.stderr: 4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0x197) [0x560ff206ad8b] 2023-03-29T15:27:59.383 INFO:tasks.ceph.osd.1.smithi111.stderr: 5: (KernelDevice::_aio_thread()+0x1798) [0x560ff2c7a908] 2023-03-29T15:27:59.383 INFO:tasks.ceph.osd.1.smithi111.stderr: 6: (KernelDevice::AioCompletionThread::entry()+0x11) [0x560ff2c7cfa1] 2023-03-29T15:27:59.383 INFO:tasks.ceph.osd.1.smithi111.stderr: 7: /lib64/libpthread.so.0(+0x81ca) [0x7f04774081ca] 2023-03-29T15:27:59.383 INFO:tasks.ceph.osd.1.smithi111.stderr: 8: clone() 2023-03-29T15:27:59.387 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-03-29T15:27:59.388+0000 7f0460ff9700 -1 bdev(0x560ff688dc00 /var/lib/ceph/osd/ceph-1/block) _aio_thread got r=-5 ((5) Input/output error) 2023-03-29T15:27:59.395 INFO:tasks.ceph.osd.1.smithi111.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-3066-gf03d69a8/rpm/el8/BUILD/ceph-18.0.0-3066-gf03d69a8/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f0460ff9700 time 2023-03-29T15:27:59.389063+0000 2023-03-29T15:27:59.395 INFO:tasks.ceph.osd.1.smithi111.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-3066-gf03d69a8/rpm/el8/BUILD/ceph-18.0.0-3066-gf03d69a8/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2023-03-29T15:27:59.396 INFO:tasks.ceph.osd.1.smithi111.stderr: ceph version 18.0.0-3066-gf03d69a8 (f03d69a8741c605ae6711afe2c9991ac46095f7d) reef (dev) 2023-03-29T15:27:59.396 INFO:tasks.ceph.osd.1.smithi111.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x560ff206accb] 2023-03-29T15:27:59.396 INFO:tasks.ceph.osd.1.smithi111.stderr: 2: (KernelDevice::_aio_thread()+0x1798) [0x560ff2c7a908] 2023-03-29T15:27:59.396 INFO:tasks.ceph.osd.1.smithi111.stderr: 3: (KernelDevice::AioCompletionThread::entry()+0x11) [0x560ff2c7cfa1] 2023-03-29T15:27:59.396 INFO:tasks.ceph.osd.1.smithi111.stderr: 4: /lib64/libpthread.so.0(+0x81ca) [0x7f04774081ca] 2023-03-29T15:27:59.396 INFO:tasks.ceph.osd.1.smithi111.stderr: 5: clone() 2023-03-29T15:27:59.397 INFO:tasks.ceph.osd.1.smithi111.stderr:2023-03-29T15:27:59.397+0000 7f0460ff9700 -1 /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-3066-gf03d69a8/rpm/el8/BUILD/ceph-18.0.0-3066-gf03d69a8/src/blk/kernel/KernelDevice.cc: In function 'void KernelDevice::_aio_thread()' thread 7f0460ff9700 time 2023-03-29T15:27:59.389063+0000 2023-03-29T15:27:59.397 INFO:tasks.ceph.osd.1.smithi111.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/18.0.0-3066-gf03d69a8/rpm/el8/BUILD/ceph-18.0.0-3066-gf03d69a8/src/blk/kernel/KernelDevice.cc: 633: ceph_abort_msg("Unexpected IO error. This may suggest a hardware issue. Please check your kernel log!") 2023-03-29T15:27:59.397 INFO:tasks.ceph.osd.1.smithi111.stderr: 2023-03-29T15:27:59.397 INFO:tasks.ceph.osd.1.smithi111.stderr: ceph version 18.0.0-3066-gf03d69a8 (f03d69a8741c605ae6711afe2c9991ac46095f7d) reef (dev) 2023-03-29T15:27:59.397 INFO:tasks.ceph.osd.1.smithi111.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string, std::allocator > const&)+0xd7) [0x560ff206accb] 2023-03-29T15:27:59.397 INFO:tasks.ceph.osd.1.smithi111.stderr: 2: (KernelDevice::_aio_thread()+0x1798) [0x560ff2c7a908] 2023-03-29T15:27:59.397 INFO:tasks.ceph.osd.1.smithi111.stderr: 3: (KernelDevice::AioCompletionThread::entry()+0x11) [0x560ff2c7cfa1] 2023-03-29T15:27:59.397 INFO:tasks.ceph.osd.1.smithi111.stderr: 4: /lib64/libpthread.so.0(+0x81ca) [0x7f04774081ca] 2023-03-29T15:27:59.397 INFO:tasks.ceph.osd.1.smithi111.stderr: 5: clone() 2023-03-29T15:27:59.397 INFO:tasks.ceph.osd.1.smithi111.stderr: 2023-03-29T15:28:12.866 INFO:tasks.ceph.osd.1.smithi111.stderr:daemon-helper: command crashed with signal 6 2023-03-29T15:28:18.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-03-29T15:28:25.141 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~7s 2023-03-29T15:28:31.752 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~13s 2023-03-29T15:28:38.363 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~20s 2023-03-29T15:28:44.974 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~26s 2023-03-29T15:28:51.587 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~33s 2023-03-29T15:28:58.199 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~40s 2023-03-29T15:29:04.811 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~46s 2023-03-29T15:29:11.422 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~53s 2023-03-29T15:29:18.034 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~60s 2023-03-29T15:29:22.378 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:29:22.384 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:29:22.915 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-03-29T15:29:22.914+0000 7f22b21f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 612399) UID: 0 2023-03-29T15:29:22.915 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T15:29:22.913+0000 7f7a699f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 612399) UID: 0 2023-03-29T15:29:22.917 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-03-29T15:29:22.914+0000 7f93ce7f9700 -1 received signal: Hangup from killall ceph-osd -1 (PID: 612399) UID: 0 2023-03-29T15:29:24.646 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~66s 2023-03-29T15:29:31.257 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~73s 2023-03-29T15:29:37.868 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~79s 2023-03-29T15:29:44.479 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~86s 2023-03-29T15:29:51.090 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~93s 2023-03-29T15:29:57.700 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~99s 2023-03-29T15:30:04.311 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~106s 2023-03-29T15:30:10.923 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~112s 2023-03-29T15:30:17.533 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~119s 2023-03-29T15:30:24.145 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~126s 2023-03-29T15:30:30.761 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~132s 2023-03-29T15:30:37.372 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~139s 2023-03-29T15:30:43.983 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~145s 2023-03-29T15:30:50.594 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~152s 2023-03-29T15:30:57.206 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~159s 2023-03-29T15:31:03.818 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~165s 2023-03-29T15:31:10.429 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~172s 2023-03-29T15:31:17.039 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~179s 2023-03-29T15:31:23.650 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~185s 2023-03-29T15:31:30.262 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~192s 2023-03-29T15:31:36.873 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~198s 2023-03-29T15:31:43.483 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~205s 2023-03-29T15:31:50.095 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~212s 2023-03-29T15:31:56.707 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~218s 2023-03-29T15:32:03.319 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~225s 2023-03-29T15:32:09.931 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~231s 2023-03-29T15:32:16.542 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~238s 2023-03-29T15:32:23.153 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~245s 2023-03-29T15:32:29.764 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~251s 2023-03-29T15:32:36.375 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~258s 2023-03-29T15:32:42.988 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~264s 2023-03-29T15:32:49.598 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~271s 2023-03-29T15:32:56.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~278s 2023-03-29T15:33:02.821 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~284s 2023-03-29T15:33:09.433 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~291s 2023-03-29T15:33:16.044 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~298s 2023-03-29T15:33:22.655 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~304s 2023-03-29T15:33:22.655 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2023-03-29T15:33:22.656 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T15:33:22.656 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-29T15:33:22.918 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345108k,nr_inodes=4086277,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275004k,mode=700,uid=1000,gid=1268 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 ceph-fuse /tmp/tmp.6dtVtxiLHFtest fuse.ceph-fuse rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0 2023-03-29T15:33:22.919 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi111.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2023-03-29T15:33:22.919 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-03-29T15:33:22.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T15:33:22.919 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-03-29T15:33:23.163 DEBUG:teuthology.orchestra.run.smithi111:> set -ex 2023-03-29T15:33:23.163 DEBUG:teuthology.orchestra.run.smithi111:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-29T15:33:23.285 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345108k,nr_inodes=4086277,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=39,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3275004k,mode=700,uid=1000,gid=1268 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 ceph-fuse /tmp/tmp.6dtVtxiLHFtest fuse.ceph-fuse rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other 0 0 2023-03-29T15:33:23.285 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.2 is not mounted at ubuntu@smithi111.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.2 2023-03-29T15:33:23.285 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi111.front.sepia.ceph.com 2023-03-29T15:33:23.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T15:33:23.286 DEBUG:teuthology.orchestra.run.smithi111:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-03-29T15:33:23.417 INFO:teuthology.orchestra.run.smithi111.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-03-29T15:33:23.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T15:33:23.419 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T15:33:23.419 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-29T15:33:23.480 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345064k,nr_inodes=4086266,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274996k,mode=700,uid=1000,gid=1268 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-03-29T15:33:23.480 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.1 is not mounted at ubuntu@smithi119.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1 2023-03-29T15:33:23.480 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-03-29T15:33:23.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T15:33:23.480 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-03-29T15:33:23.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T15:33:23.571 INFO:teuthology.orchestra.run.smithi119.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-03-29T15:33:23.571 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-03-29T15:33:23.571 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-03-29T15:33:23.615 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16345064k,nr_inodes=4086266,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=43,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3274996k,mode=700,uid=1000,gid=1268 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-03-29T15:33:23.615 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.3 is not mounted at ubuntu@smithi119.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.3 2023-03-29T15:33:23.615 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-03-29T15:33:23.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-03-29T15:33:23.616 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-03-29T15:33:23.698 INFO:teuthology.orchestra.run.smithi119.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-03-29T15:33:23.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-03-29T15:33:25.309 INFO:tasks.ceph.osd.0:Sent signal 15 2023-03-29T15:33:25.310 INFO:tasks.ceph.osd.2:Sent signal 15 2023-03-29T15:33:25.310 INFO:tasks.ceph.osd.3:Sent signal 15 2023-03-29T15:33:25.310 INFO:tasks.ceph.osd.4:Sent signal 15 2023-03-29T15:33:25.311 INFO:tasks.ceph.osd.5:Sent signal 15 2023-03-29T15:33:25.311 INFO:tasks.ceph.osd.6:Sent signal 15 2023-03-29T15:33:25.311 INFO:tasks.ceph.osd.7:Sent signal 15 2023-03-29T15:33:25.311 INFO:tasks.ceph.mds.a:Sent signal 15 2023-03-29T15:33:25.311 INFO:tasks.ceph.mds.c:Sent signal 15 2023-03-29T15:33:25.312 INFO:tasks.ceph.mds.b:Sent signal 15 2023-03-29T15:33:25.312 INFO:tasks.ceph.mds.d:Sent signal 15 2023-03-29T15:33:25.312 INFO:tasks.ceph.mon.a:Sent signal 15 2023-03-29T15:33:25.312 INFO:tasks.ceph.mon.b:Sent signal 15 2023-03-29T15:33:25.312 INFO:tasks.ceph.mon.c:Sent signal 15 2023-03-29T15:33:25.312 INFO:tasks.ceph.mgr.y:Sent signal 15 2023-03-29T15:33:25.313 INFO:tasks.ceph.mgr.x:Sent signal 15 2023-03-29T15:33:25.313 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-03-29T15:33:25.312+0000 7fd2277f9700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 110634) UID: 0 2023-03-29T15:33:25.313 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-03-29T15:33:25.312+0000 7fd2277f9700 -1 osd.4 47 *** Got signal Terminated *** 2023-03-29T15:33:25.313 INFO:tasks.ceph.osd.4.smithi119.stderr:2023-03-29T15:33:25.312+0000 7fd2277f9700 -1 osd.4 47 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-03-29T15:33:25.314 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-03-29T15:33:25.312+0000 7f0c30df9700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 110635) UID: 0 2023-03-29T15:33:25.314 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-03-29T15:33:25.312+0000 7f0c30df9700 -1 osd.5 47 *** Got signal Terminated *** 2023-03-29T15:33:25.314 INFO:tasks.ceph.osd.5.smithi119.stderr:2023-03-29T15:33:25.312+0000 7f0c30df9700 -1 osd.5 47 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-03-29T15:33:25.314 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-03-29T15:33:25.312+0000 7fabb15f9700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 110640) UID: 0 2023-03-29T15:33:25.314 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-03-29T15:33:25.312+0000 7fabb15f9700 -1 osd.6 47 *** Got signal Terminated *** 2023-03-29T15:33:25.314 INFO:tasks.ceph.osd.6.smithi119.stderr:2023-03-29T15:33:25.312+0000 7fabb15f9700 -1 osd.6 47 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-03-29T15:33:25.314 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-03-29T15:33:25.312+0000 7f9655df9700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 110641) UID: 0 2023-03-29T15:33:25.314 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-03-29T15:33:25.312+0000 7f9655df9700 -1 osd.7 47 *** Got signal Terminated *** 2023-03-29T15:33:25.315 INFO:tasks.ceph.osd.7.smithi119.stderr:2023-03-29T15:33:25.312+0000 7f9655df9700 -1 osd.7 47 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-03-29T15:33:25.315 INFO:tasks.ceph.mds.b.smithi119.stderr:2023-03-29T15:33:25.313+0000 7fa6931f9700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i b (PID: 118610) UID: 0 2023-03-29T15:33:25.315 INFO:tasks.ceph.mds.b.smithi119.stderr:2023-03-29T15:33:25.313+0000 7fa6931f9700 -1 mds.b *** got signal Terminated *** 2023-03-29T15:33:25.316 INFO:tasks.ceph.mds.d.smithi119.stderr:2023-03-29T15:33:25.313+0000 7f7f2ebf9700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i d (PID: 118611) UID: 0 2023-03-29T15:33:25.316 INFO:tasks.ceph.mds.d.smithi119.stderr:2023-03-29T15:33:25.313+0000 7f7f2ebf9700 -1 mds.d *** got signal Terminated *** 2023-03-29T15:33:25.316 INFO:tasks.ceph.mon.c.smithi119.stderr:2023-03-29T15:33:25.314+0000 7f4a8b1ff700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i c (PID: 109932) UID: 0 2023-03-29T15:33:25.317 INFO:tasks.ceph.mon.c.smithi119.stderr:2023-03-29T15:33:25.314+0000 7f4a8b1ff700 -1 mon.c@2(peon) e1 *** Got Signal Terminated *** 2023-03-29T15:33:25.317 INFO:tasks.ceph.mon.b.smithi119.stderr:2023-03-29T15:33:25.314+0000 7fdb001ff700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i b (PID: 109929) UID: 0 2023-03-29T15:33:25.317 INFO:tasks.ceph.mon.b.smithi119.stderr:2023-03-29T15:33:25.314+0000 7fdb001ff700 -1 mon.b@1(peon) e1 *** Got Signal Terminated *** 2023-03-29T15:33:25.319 INFO:tasks.ceph.mds.a.smithi111.stderr:2023-03-29T15:33:25.318+0000 7f3a763f9700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i a (PID: 128389) UID: 0 2023-03-29T15:33:25.319 INFO:tasks.ceph.mds.a.smithi111.stderr:2023-03-29T15:33:25.318+0000 7f3a763f9700 -1 mds.a *** got signal Terminated *** 2023-03-29T15:33:25.327 INFO:tasks.ceph.mon.a.smithi111.stderr:2023-03-29T15:33:25.324+0000 7f79c5fff700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 110133) UID: 0 2023-03-29T15:33:25.327 INFO:tasks.ceph.mon.a.smithi111.stderr:2023-03-29T15:33:25.324+0000 7f79c5fff700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-03-29T15:33:25.332 INFO:tasks.ceph.mds.c.smithi111.stderr:2023-03-29T15:33:25.326+0000 7f155ddf9700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mds -f --cluster ceph -i c (PID: 128392) UID: 0 2023-03-29T15:33:25.332 INFO:tasks.ceph.mds.c.smithi111.stderr:2023-03-29T15:33:25.326+0000 7f155ddf9700 -1 mds.c *** got signal Terminated *** 2023-03-29T15:33:25.391 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-03-29T15:33:25.331+0000 7f22b21f9700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 110475) UID: 0 2023-03-29T15:33:25.391 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-03-29T15:33:25.331+0000 7f22b21f9700 -1 osd.3 47 *** Got signal Terminated *** 2023-03-29T15:33:25.391 INFO:tasks.ceph.osd.3.smithi111.stderr:2023-03-29T15:33:25.331+0000 7f22b21f9700 -1 osd.3 47 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-03-29T15:33:25.405 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-03-29T15:33:25.311+0000 7f93ce7f9700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 110471) UID: 0 2023-03-29T15:33:25.405 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-03-29T15:33:25.311+0000 7f93ce7f9700 -1 osd.2 47 *** Got signal Terminated *** 2023-03-29T15:33:25.406 INFO:tasks.ceph.osd.2.smithi111.stderr:2023-03-29T15:33:25.311+0000 7f93ce7f9700 -1 osd.2 47 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-03-29T15:33:25.514 INFO:tasks.ceph.mgr.x.smithi119.stderr:daemon-helper: command crashed with signal 15 2023-03-29T15:33:25.622 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T15:33:25.311+0000 7f7a699f9700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 110467) UID: 0 2023-03-29T15:33:25.622 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T15:33:25.311+0000 7f7a699f9700 -1 osd.0 47 *** Got signal Terminated *** 2023-03-29T15:33:25.622 INFO:tasks.ceph.osd.0.smithi111.stderr:2023-03-29T15:33:25.311+0000 7f7a699f9700 -1 osd.0 47 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-03-29T15:34:18.764 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:34:18.778 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:34:49.041 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:34:49.049 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:35:19.232 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:35:19.239 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:35:49.446 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:35:49.452 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:36:19.632 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:36:19.637 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:36:49.837 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:36:49.842 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:37:20.182 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:37:20.194 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:37:50.341 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:37:50.346 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:38:20.448 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:38:20.457 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:38:50.719 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:38:50.726 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:39:20.985 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:39:20.993 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:39:51.175 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:39:51.185 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:40:21.464 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:40:21.473 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:40:51.679 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:40:51.688 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:41:21.892 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:41:21.898 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:41:52.203 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:41:52.208 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:42:22.403 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:42:22.409 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:42:52.676 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:42:52.682 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:43:23.082 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:43:23.087 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:43:53.219 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:43:53.224 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:44:23.475 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:44:23.479 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:44:53.645 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:44:53.655 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:45:23.938 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:45:23.947 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:45:54.107 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:45:54.115 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:46:24.379 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:46:24.387 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:46:54.671 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:46:54.680 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:47:24.886 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:47:24.895 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:47:55.063 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:47:55.072 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:48:25.289 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:48:25.294 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:48:55.602 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:48:55.610 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:49:25.825 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:49:25.834 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:49:56.049 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:49:56.058 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:50:26.336 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:50:26.343 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:50:56.593 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:50:56.598 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:51:26.798 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:51:26.803 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:51:57.004 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:51:57.009 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:52:27.323 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:52:27.328 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:52:57.514 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:52:57.520 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:53:27.701 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:53:27.709 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:53:57.920 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:53:57.924 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:54:28.098 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:54:28.105 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:54:58.313 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:54:58.318 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:55:28.524 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:55:28.529 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:55:58.823 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:55:58.832 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:56:29.095 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:56:29.101 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:56:59.310 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:56:59.316 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:57:29.516 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:57:29.521 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:57:59.744 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:57:59.752 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:58:29.994 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:58:30.003 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:59:00.233 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:59:00.242 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:59:30.482 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T15:59:30.490 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:00:00.703 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:00:00.711 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:00:30.956 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:00:30.964 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:01:01.239 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:01:01.245 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:01:31.423 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:01:31.428 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:02:01.611 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:02:01.620 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:02:32.024 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:02:32.028 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:03:02.146 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:03:02.150 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:03:32.352 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:03:32.360 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:04:02.550 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:04:02.558 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:04:32.781 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:04:32.789 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:05:02.986 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:05:02.995 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:05:33.184 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:05:33.191 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:06:03.320 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:06:03.329 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:06:33.586 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:06:33.594 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:07:03.893 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:07:03.900 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:07:34.098 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:07:34.107 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:08:04.326 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:08:04.335 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:08:34.585 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:08:34.589 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:09:04.797 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:09:04.806 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:09:35.062 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:09:35.067 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:10:05.172 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:10:05.177 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:10:35.380 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:10:35.390 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:11:05.544 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:11:05.551 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:11:35.684 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:11:35.690 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:12:05.994 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:12:06.002 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:12:36.296 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:12:36.302 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:13:06.501 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:13:06.509 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:13:36.807 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:13:36.815 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:14:07.020 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:14:07.029 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:14:37.244 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:14:37.250 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:15:07.457 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:15:07.466 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:15:37.937 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:15:37.943 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:16:08.192 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:16:08.197 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:16:38.412 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:16:38.419 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:17:08.571 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:17:08.577 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:17:38.760 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:17:38.768 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:18:08.990 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:18:08.998 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:18:39.251 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:18:39.256 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:19:09.426 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:19:09.430 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:19:39.630 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:19:39.635 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:20:09.960 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:20:09.967 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:20:40.149 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:20:40.154 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:21:10.349 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:21:10.355 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:21:40.596 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:21:40.604 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:22:10.835 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:22:10.844 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:22:41.083 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:22:41.092 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:23:11.321 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:23:11.326 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:23:41.493 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:23:41.499 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:24:11.783 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:24:11.788 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:24:41.976 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:24:41.981 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:25:12.176 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:25:12.185 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:25:42.374 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:25:42.382 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:26:12.576 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:26:12.585 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:26:42.877 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:26:42.885 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:27:13.049 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:27:13.053 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:27:43.300 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:27:43.305 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:28:13.636 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:28:13.644 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:28:43.772 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:28:43.780 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:29:14.047 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:29:14.055 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:29:44.273 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:29:44.278 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:30:14.454 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:30:14.460 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:30:44.896 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:30:44.901 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:31:15.097 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:31:15.102 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:31:45.331 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:31:45.339 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:32:15.622 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:32:15.631 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:32:45.957 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:32:45.962 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:33:16.163 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:33:16.172 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:33:46.411 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:33:46.420 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:34:16.620 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:34:16.627 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:34:46.833 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:34:46.840 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:35:17.019 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:35:17.026 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:35:47.175 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:35:47.183 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:36:17.408 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:36:17.416 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:36:47.580 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:36:47.588 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:37:17.775 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:37:17.784 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:37:48.001 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:37:48.009 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:38:18.280 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:38:18.287 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:38:48.493 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:38:48.501 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:39:18.649 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:39:18.657 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:39:48.855 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:39:48.863 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:40:19.110 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:40:19.115 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:40:49.324 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:40:49.333 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:41:19.627 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:41:19.635 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:41:49.835 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:41:49.844 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:42:20.099 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:42:20.108 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:42:50.405 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:42:50.414 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:43:20.703 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:43:20.711 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:43:50.959 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:43:50.968 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:44:21.181 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:44:21.190 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:44:51.523 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:44:51.531 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:45:21.700 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:45:21.705 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:45:51.945 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:45:51.954 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:46:22.229 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:46:22.237 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:46:52.492 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:46:52.498 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:47:22.673 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:47:22.678 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:47:52.893 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:47:52.901 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:48:23.122 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:48:23.131 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:48:53.423 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:48:53.432 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:49:23.645 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:49:23.653 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:49:53.865 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:49:53.870 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:50:24.020 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:50:24.024 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:50:54.258 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:50:54.267 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:51:24.443 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:51:24.450 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:51:54.696 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:51:54.702 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:52:24.921 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:52:24.929 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:52:55.202 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:52:55.232 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:53:25.649 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:53:25.657 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:53:55.947 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:53:55.955 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:54:26.209 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:54:26.218 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:54:56.406 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:54:56.426 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:55:26.638 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:55:26.646 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:55:56.839 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:55:56.847 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:56:27.049 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:56:27.055 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:56:57.288 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:56:57.300 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:57:27.510 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:57:27.518 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:57:57.724 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:57:57.732 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:58:27.954 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:58:27.959 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:58:58.284 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:58:58.290 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:59:28.479 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:59:28.486 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:59:58.815 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T16:59:58.823 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:00:29.006 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:00:29.015 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:00:59.249 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:00:59.255 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:01:29.529 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:01:29.537 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:01:59.734 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:01:59.741 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:02:30.016 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:02:30.025 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:03:00.230 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:03:00.238 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:03:30.486 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:03:30.494 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:04:00.697 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:04:00.704 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:04:30.902 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:04:30.910 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:05:01.169 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:05:01.175 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:05:31.573 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:05:31.582 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:06:01.780 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:06:01.789 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:06:32.019 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:06:32.027 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:07:02.244 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:07:02.251 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:07:32.471 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:07:32.480 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:08:02.696 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:08:02.704 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:08:32.877 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:08:32.885 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:09:03.073 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:09:03.081 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:09:33.236 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:09:33.244 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:10:03.468 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:10:03.477 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:10:33.706 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:10:33.714 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:11:03.955 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:11:03.960 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:11:34.169 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:11:34.174 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:12:04.341 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:12:04.349 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:12:34.631 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:12:34.640 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:13:04.921 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:13:04.929 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:13:35.134 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:13:35.142 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:14:05.338 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:14:05.346 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:14:35.577 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:14:35.583 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:15:05.739 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:15:05.744 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:15:35.935 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:15:35.944 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:16:06.220 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:16:06.226 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:16:36.397 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:16:36.405 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:17:06.611 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:17:06.619 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:17:36.808 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:17:36.817 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:18:06.985 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:18:06.991 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:18:37.239 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:18:37.245 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:19:07.503 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:19:07.508 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:19:37.642 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:19:37.647 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:20:07.856 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:20:07.862 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:20:38.052 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:20:38.061 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:21:08.248 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:21:08.255 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:21:38.479 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:21:38.484 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:22:08.901 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:22:08.910 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:22:39.118 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:22:39.123 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:23:09.356 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:23:09.365 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:23:39.571 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:23:39.580 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:24:09.835 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:24:09.841 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:24:39.965 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:24:39.969 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:25:10.143 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:25:10.150 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:25:40.267 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:25:40.272 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:26:10.396 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:26:10.401 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:26:40.545 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:26:40.552 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:27:10.677 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:27:10.686 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:27:41.035 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:27:41.046 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:28:11.297 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:28:11.306 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:28:41.570 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:28:41.577 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:29:11.699 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:29:11.707 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:29:41.902 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:29:41.908 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:30:12.247 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:30:12.254 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:30:42.476 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:30:42.484 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:31:12.721 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:31:12.729 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:31:42.969 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:31:42.978 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:32:13.225 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:32:13.231 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:32:43.535 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:32:43.542 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:33:13.713 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:33:13.721 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:33:43.934 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:33:43.938 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:34:14.051 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:34:14.056 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:34:44.418 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:34:44.424 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:35:14.613 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:35:14.619 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:35:44.835 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:35:44.841 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:36:15.095 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:36:15.100 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:36:45.376 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:36:45.384 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:37:15.581 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:37:15.606 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:37:45.951 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:37:45.960 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:38:16.234 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:38:16.239 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:38:46.579 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:38:46.585 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:39:16.862 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:39:16.871 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:39:47.352 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:39:47.360 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:40:17.604 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:40:17.613 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:40:47.879 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:40:47.887 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:41:18.021 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:41:18.029 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:41:48.296 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:41:48.304 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:42:18.483 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:42:18.491 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:42:48.658 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:42:48.665 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:43:18.846 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:43:18.854 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:43:49.112 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:43:49.120 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:44:19.314 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:44:19.323 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:44:49.531 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:44:49.539 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:45:19.751 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:45:19.760 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:45:49.951 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:45:49.958 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:46:20.195 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:46:20.202 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:46:50.393 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:46:50.402 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:47:20.909 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:47:20.917 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:47:51.133 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:47:51.139 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:48:21.353 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:48:21.361 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:48:51.533 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:48:51.539 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:49:21.787 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:49:21.793 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:49:51.967 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:49:51.976 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:50:22.185 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:50:22.194 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:50:52.473 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:50:52.481 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:51:22.655 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:51:22.663 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:51:52.841 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:51:52.851 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:52:23.193 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:52:23.202 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:52:53.370 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:52:53.378 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:53:23.559 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:53:23.567 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:53:53.833 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:53:53.839 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:54:24.047 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:54:24.052 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:54:54.316 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:54:54.320 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:55:24.437 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:55:24.443 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:55:54.660 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:55:54.669 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:56:25.022 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:56:25.030 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:56:55.158 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:56:55.167 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:57:25.456 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:57:25.463 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:57:55.686 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:57:55.691 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:58:25.829 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:58:25.838 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:58:56.089 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:58:56.093 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:59:26.317 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:59:26.323 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:59:56.516 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T17:59:56.524 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:00:26.697 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:00:26.705 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:00:56.880 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:00:56.888 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:01:27.102 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:01:27.111 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:01:57.391 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:01:57.397 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:02:27.622 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:02:27.627 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:02:57.850 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:02:57.858 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:03:28.106 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:03:28.112 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:03:58.312 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:03:58.321 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:04:28.494 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:04:28.502 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:04:58.717 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:04:58.724 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:05:28.904 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:05:28.912 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:05:59.083 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:05:59.090 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:06:29.390 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:06:29.395 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:06:59.636 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:06:59.644 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:07:29.826 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:07:29.837 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:07:59.980 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:07:59.989 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:08:30.197 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:08:30.205 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:09:00.357 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:09:00.363 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:09:30.571 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:09:30.579 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:10:00.867 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:10:00.874 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:10:31.058 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:10:31.064 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:11:01.325 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:11:01.334 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:11:31.516 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:11:31.524 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:12:01.758 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:12:01.766 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:12:32.032 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:12:32.041 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:13:02.250 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:13:02.257 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:13:32.478 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:13:32.487 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:14:02.752 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:14:02.760 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:14:32.916 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:14:32.924 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:15:03.119 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:15:03.127 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:15:33.286 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:15:33.290 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:16:03.480 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:16:03.489 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:16:33.749 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:16:33.755 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:17:03.963 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:17:03.971 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:17:34.191 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:17:34.199 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:18:04.449 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:18:04.454 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:18:34.787 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:18:34.795 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:19:04.958 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:19:04.962 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:19:35.296 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:19:35.304 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:20:05.477 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:20:05.485 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:20:35.787 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:20:35.796 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:21:06.042 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:21:06.050 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:21:36.320 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:21:36.328 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:22:06.595 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:22:06.603 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:22:36.741 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:22:36.749 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:23:07.014 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:23:07.023 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:23:37.229 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:23:37.238 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:24:07.450 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:24:07.459 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:24:37.723 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:24:37.731 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:25:08.079 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:25:08.085 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:25:38.284 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:25:38.293 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:26:08.557 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:26:08.562 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:26:38.753 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:26:38.761 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:27:09.018 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:27:09.024 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:27:39.299 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:27:39.305 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:28:09.497 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:28:09.505 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:28:39.719 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:28:39.725 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:29:10.000 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:29:10.005 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:29:40.428 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:29:40.437 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:30:10.687 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:30:10.696 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:30:40.948 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:30:40.953 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:31:11.129 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:31:11.134 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:31:41.311 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:31:41.319 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:32:11.535 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:32:11.543 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:32:41.772 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:32:41.780 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:33:11.996 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:33:12.005 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:33:42.235 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:33:42.241 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:34:12.539 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:34:12.547 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:34:42.825 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:34:42.834 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:35:13.136 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:35:13.145 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:35:43.363 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:35:43.372 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:36:13.655 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:36:13.663 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:36:43.849 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:36:43.855 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:37:14.066 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:37:14.075 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:37:44.328 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:37:44.336 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:38:14.553 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:38:14.561 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:38:44.758 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:38:44.764 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:39:14.926 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:39:14.930 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:39:45.080 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:39:45.085 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:40:15.367 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:40:15.375 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:40:45.565 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:40:45.570 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:41:15.775 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:41:15.783 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:41:46.084 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:41:46.092 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:42:16.306 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:42:16.314 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:42:46.484 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:42:46.492 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:43:16.674 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:43:16.682 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:43:46.857 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:43:46.865 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:44:17.051 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:44:17.057 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:44:47.327 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:44:47.335 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:45:17.563 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:45:17.571 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:45:47.827 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:45:47.832 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:46:18.119 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:46:18.125 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:46:48.524 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:46:48.532 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:47:18.748 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:47:18.753 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:47:48.936 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:47:48.942 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:48:19.189 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:48:19.195 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:48:49.440 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:48:49.445 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:49:19.604 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:49:19.613 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:49:49.823 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:49:49.828 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:50:19.995 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:50:20.000 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:50:50.191 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:50:50.196 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:51:20.476 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:51:20.513 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:51:50.852 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:51:50.861 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:52:21.113 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:52:21.119 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:52:51.544 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:52:51.550 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:53:21.734 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:53:21.743 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:53:51.997 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:53:52.006 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:54:22.203 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:54:22.212 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:54:52.476 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:54:52.484 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:55:22.722 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:55:22.728 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:55:52.904 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:55:52.917 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:56:23.209 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:56:23.215 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:56:53.396 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:56:53.400 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:57:23.575 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:57:23.582 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:57:53.759 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:57:53.768 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:58:23.982 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:58:23.989 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:58:54.229 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:58:54.237 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:59:24.443 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:59:24.451 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:59:54.691 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T18:59:54.700 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:00:24.913 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:00:24.920 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:00:55.144 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:00:55.149 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:01:25.409 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:01:25.413 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:01:55.561 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:01:55.674 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:02:25.962 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:02:25.971 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:02:56.250 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:02:56.254 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:03:26.448 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:03:26.453 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:03:56.712 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:03:56.721 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:04:26.934 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:04:26.941 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:04:57.190 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:04:57.198 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:05:27.429 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:05:27.436 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:05:57.673 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:05:57.680 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:06:27.848 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:06:27.856 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:06:58.036 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:06:58.044 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:07:28.310 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:07:28.319 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:07:58.791 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:07:58.799 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:08:28.993 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:08:29.001 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:08:59.236 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:08:59.244 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:09:29.500 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:09:29.509 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:09:59.782 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:09:59.791 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:10:30.002 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:10:30.137 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:11:00.281 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:11:00.289 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:11:30.481 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:11:30.487 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:12:00.811 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:12:00.816 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:12:31.039 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:12:31.044 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:13:01.370 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:13:01.379 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:13:31.653 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:13:31.662 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:14:01.881 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:14:01.885 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:14:32.044 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:14:32.050 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:15:02.217 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:15:02.368 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:15:32.525 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:15:32.563 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:16:02.770 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:16:02.778 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:16:33.021 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:16:33.025 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:17:03.289 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:17:03.295 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:17:33.509 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:17:33.516 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:18:03.714 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:18:03.798 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:18:34.046 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:18:34.052 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:19:04.341 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:19:04.348 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:19:34.504 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:19:34.513 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:20:04.780 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:20:04.788 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:20:34.999 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:20:35.004 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:21:05.170 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:21:05.177 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:21:35.537 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:21:35.544 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:22:05.743 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:22:05.751 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:22:35.940 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:22:35.949 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:23:06.154 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:23:06.163 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:23:36.451 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:23:36.457 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:24:06.740 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:24:06.746 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:24:36.993 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:24:36.999 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:25:07.199 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:25:07.207 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:25:37.423 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:25:37.431 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:26:07.617 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:26:07.626 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:26:37.810 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:26:37.818 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:27:07.991 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:27:07.998 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:27:38.166 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:27:38.171 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:28:08.390 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:28:08.494 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:28:38.766 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:28:38.775 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:29:08.969 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:29:08.977 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:29:39.251 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:29:39.259 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:30:09.509 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:30:09.571 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:30:39.805 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:30:39.811 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:31:10.085 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:31:10.091 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:31:40.282 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:31:40.291 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:32:10.505 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:32:10.513 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:32:40.752 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:32:40.837 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:33:11.050 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:33:11.058 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:33:41.272 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:33:41.345 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:34:11.620 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:34:11.629 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:34:41.891 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:34:41.896 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:35:12.044 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:35:12.052 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:35:42.253 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:35:42.261 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:36:12.517 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:36:12.637 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:36:42.831 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:36:42.839 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:37:13.016 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:37:13.023 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:37:43.216 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:37:43.224 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:38:13.505 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:38:13.512 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:38:43.633 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:38:43.639 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:39:13.815 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:39:13.823 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:39:44.030 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:39:44.036 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:40:14.277 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:40:14.285 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:40:44.470 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:40:44.474 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:41:14.665 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:41:14.673 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:41:44.936 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:41:44.944 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:42:15.160 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:42:15.168 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:42:45.448 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:42:45.456 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:43:15.667 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:43:15.676 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:43:45.926 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:43:45.935 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:44:16.190 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:44:16.198 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:44:46.429 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:44:46.436 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:45:16.709 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:45:16.718 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:45:46.900 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:45:46.908 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:46:17.137 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:46:17.145 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:46:47.432 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:46:47.439 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:47:17.604 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:47:17.608 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:47:47.777 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:47:47.891 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:48:18.116 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:48:18.124 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:48:48.357 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:48:48.366 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:49:18.554 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:49:18.562 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:49:48.810 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:49:48.819 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:50:19.032 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:50:19.038 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:50:49.312 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:50:49.318 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:51:19.557 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:51:19.565 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:51:49.748 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:51:49.757 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:52:19.978 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:52:19.983 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:52:50.172 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:52:50.179 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:53:20.423 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:53:20.431 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:53:50.642 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:53:50.651 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:54:20.869 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:54:20.876 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:54:51.073 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:54:51.079 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:55:21.285 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:55:21.320 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:55:51.598 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:55:51.606 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:56:21.775 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:56:21.782 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:56:51.938 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:56:51.946 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:57:22.177 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:57:22.186 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:57:52.467 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:57:52.475 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:58:22.655 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:58:22.665 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:58:52.878 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:58:52.884 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:59:23.084 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:59:23.089 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:59:53.365 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T19:59:53.373 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:00:23.556 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:00:23.565 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:00:53.717 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:00:53.726 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:01:23.931 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:01:23.940 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:01:54.184 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:01:54.192 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:02:24.423 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:02:24.431 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:02:54.678 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:02:54.686 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:03:24.929 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:03:24.937 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:03:55.184 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:03:55.189 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:04:25.364 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:04:25.369 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:04:55.546 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:04:55.552 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:05:25.776 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:05:25.783 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:05:55.970 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:05:55.978 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:06:26.171 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:06:26.180 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:06:56.443 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:06:56.452 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:07:26.749 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:07:26.754 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:07:56.933 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:07:56.941 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:08:27.234 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:08:27.240 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:08:57.433 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:08:57.439 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:09:27.706 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:09:27.711 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:09:57.874 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:09:57.882 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:10:28.173 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:10:28.182 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:10:58.384 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:10:58.392 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:11:28.634 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:11:28.642 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:11:58.839 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:11:58.849 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:12:29.120 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:12:29.126 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:12:59.385 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:12:59.390 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:13:29.587 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:13:29.596 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:13:59.796 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:13:59.805 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:14:30.002 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:14:30.011 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:15:00.223 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:15:00.228 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:15:30.451 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:15:30.460 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:16:00.727 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:16:00.736 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:16:30.941 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:16:30.950 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:17:01.229 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:17:01.238 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:17:31.510 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:17:31.518 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:18:01.723 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:18:01.730 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:18:31.926 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:18:31.931 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:19:02.151 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:19:02.155 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:19:32.361 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:19:32.370 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:20:02.570 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:20:02.579 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:20:32.759 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:20:32.768 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:21:03.064 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:21:03.072 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:21:33.264 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:21:33.273 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:22:03.448 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:22:03.457 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:22:33.666 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:22:33.674 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:23:03.947 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:23:03.955 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:23:34.159 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:23:34.167 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:24:04.362 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:24:04.371 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:24:34.637 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:24:34.646 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:25:04.834 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:25:04.843 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:25:35.067 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:25:35.073 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:26:05.310 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:26:05.316 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:26:35.660 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:26:35.844 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:27:06.080 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:27:06.088 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:27:36.311 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:27:36.319 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:28:06.548 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:28:06.555 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:28:36.779 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:28:36.786 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:29:07.027 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:29:07.035 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:29:37.311 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:29:37.320 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:30:07.591 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:30:07.600 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:30:37.864 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:30:37.872 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:31:08.069 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:31:08.075 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:31:38.266 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:31:38.275 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:32:08.475 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:32:08.481 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:32:38.685 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:32:38.689 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:33:08.863 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:33:08.868 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:33:39.085 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:33:39.093 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:34:09.275 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:34:09.280 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:34:39.441 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:34:39.449 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:35:09.612 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:35:09.621 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:35:39.826 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:35:39.834 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:36:10.037 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:36:10.045 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:36:40.216 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:36:40.224 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:37:10.403 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:37:10.408 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:37:40.565 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:37:40.573 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:38:10.753 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:38:10.758 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:38:40.959 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:38:40.968 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:39:11.144 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:39:11.153 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:39:41.353 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:39:41.359 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:40:11.561 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:40:11.569 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:40:41.765 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:40:41.773 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:41:11.903 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:41:11.915 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:41:42.084 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:41:42.092 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:42:12.299 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:42:12.306 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:42:42.527 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:42:42.532 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:43:12.722 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:43:12.727 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:43:42.897 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:43:42.902 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:44:13.178 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:44:13.184 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:44:43.372 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:44:43.378 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:45:13.647 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:45:13.656 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:45:43.877 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:45:43.885 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:46:14.105 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:46:14.112 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:46:44.278 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:46:44.287 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:47:14.561 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:47:14.569 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:47:44.816 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:47:44.824 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:48:15.072 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:48:15.080 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:48:45.316 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:48:45.321 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:49:15.546 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:49:15.943 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:49:46.222 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:49:46.231 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:50:16.419 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:50:16.428 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:50:46.675 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:50:46.683 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:51:16.964 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:51:16.969 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:51:47.201 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:51:47.208 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:52:17.324 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:52:17.328 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:52:47.537 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:52:47.543 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:53:17.752 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:53:17.760 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:53:48.033 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:53:48.041 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:54:18.277 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:54:18.282 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:54:48.473 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:54:48.479 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:55:18.666 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:55:18.674 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:55:48.888 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:55:48.895 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:56:19.168 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:56:19.176 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:56:49.370 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:56:49.378 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:57:19.569 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:57:19.578 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:57:49.801 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:57:49.809 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:58:20.075 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:58:20.082 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:58:50.187 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:58:50.193 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:59:20.359 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:59:20.364 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:59:50.620 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T20:59:50.629 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:00:20.769 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:00:20.774 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:00:51.009 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:00:51.017 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:01:21.315 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:01:21.323 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:01:51.546 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:01:51.553 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:02:21.693 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:02:21.701 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:02:51.921 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:02:51.930 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:03:22.188 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:03:22.196 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:03:52.477 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:03:52.485 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:04:22.673 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:04:22.681 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:04:52.894 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:04:52.903 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:05:23.182 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:05:23.189 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:05:53.405 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:05:53.411 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:06:23.597 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:06:23.605 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:06:53.885 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:06:53.893 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:07:24.031 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:07:24.040 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:07:54.195 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:07:54.204 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:08:24.374 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:08:24.382 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:08:54.591 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:08:54.599 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:09:24.779 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:09:24.786 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:09:54.984 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:09:54.994 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:10:25.186 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:10:25.206 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:10:55.412 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:10:55.421 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:11:25.618 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:11:25.626 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:11:55.840 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:11:55.849 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:12:26.054 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:12:26.062 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:12:56.327 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:12:56.335 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:13:26.531 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:13:26.538 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:13:56.761 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:13:56.766 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:14:26.934 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:14:26.940 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:14:57.191 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:14:57.198 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:15:27.420 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:15:27.428 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:15:57.718 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:15:57.724 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:16:27.904 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:16:27.924 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:16:58.129 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:16:58.136 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:17:28.349 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:17:28.356 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:17:58.529 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:17:58.537 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:18:28.737 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:18:28.742 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:18:58.940 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:18:58.946 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:19:29.233 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:19:29.241 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:19:59.353 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:19:59.361 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:20:29.552 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:20:29.558 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:20:59.757 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:20:59.764 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:21:29.976 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:21:29.982 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:22:00.209 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:22:00.215 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:22:30.482 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:22:30.487 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:23:00.701 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:23:00.708 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:23:30.886 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:23:30.891 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:24:01.112 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:24:01.117 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:24:31.344 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:24:31.353 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:25:01.557 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:25:01.564 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:25:31.845 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:25:31.849 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:26:02.083 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:26:02.090 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:26:32.367 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:26:32.372 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:27:02.604 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:27:02.610 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:27:32.818 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:27:32.826 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:28:02.955 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:28:02.963 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:28:33.214 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:28:33.221 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:29:03.461 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:29:03.469 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:29:33.749 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:29:33.755 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:30:04.060 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:30:04.065 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:30:34.276 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:30:34.284 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:31:04.423 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:31:04.428 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:31:34.591 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:31:34.601 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:32:04.802 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:32:04.811 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:32:35.050 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:32:35.059 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:33:05.323 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:33:05.332 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:33:35.628 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:33:35.641 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:34:05.933 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:34:05.941 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:34:36.172 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:34:36.179 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:35:06.324 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:35:06.333 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:35:36.533 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:35:36.541 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:36:06.768 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:36:06.774 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:36:36.982 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:36:36.991 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:37:07.207 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:37:07.214 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:37:37.421 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:37:37.429 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:38:07.657 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:38:07.666 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:38:37.949 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:38:37.958 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:39:08.110 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:39:08.118 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:39:38.358 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:39:38.366 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:40:08.557 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:40:08.566 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:40:38.780 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:40:38.786 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:41:08.959 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:41:08.973 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:41:39.189 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:41:39.197 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:42:09.362 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:42:09.369 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:42:39.532 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:42:39.541 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:43:09.842 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:43:09.857 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:43:40.070 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:43:40.076 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:44:10.348 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:44:10.357 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:44:40.615 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:44:40.623 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:45:10.872 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:45:10.880 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:45:41.170 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:45:41.178 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:46:11.451 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:46:11.458 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:46:41.619 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:46:41.628 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:47:11.903 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:47:11.912 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:47:42.116 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:47:42.128 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:48:12.445 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:48:12.454 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:48:42.653 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:48:42.660 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:49:12.763 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:49:12.770 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:49:42.869 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:49:42.877 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:50:13.078 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:50:13.086 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:50:43.231 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:50:43.237 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:51:13.430 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:51:13.438 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:51:43.582 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:51:43.591 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:52:13.767 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:52:13.773 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:52:44.023 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:52:44.030 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:53:14.203 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:53:14.209 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:53:44.400 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:53:44.408 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:54:14.571 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:54:14.579 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:54:44.735 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:54:44.744 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:55:15.009 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:55:15.016 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:55:45.211 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:55:45.217 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:56:15.409 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:56:15.417 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:56:45.691 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:56:45.700 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:57:16.005 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:57:16.087 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:57:46.327 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:57:46.337 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:58:16.564 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:58:16.572 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:58:46.813 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:58:46.817 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:59:17.035 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:59:17.044 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:59:47.333 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T21:59:47.342 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:00:17.614 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:00:17.621 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:00:47.775 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:00:47.783 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:01:18.055 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:01:18.063 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:01:48.202 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:01:48.211 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:02:18.469 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:02:18.475 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:02:48.725 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:02:48.730 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:03:18.902 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:03:18.909 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:03:49.113 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:03:49.120 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:04:19.383 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:04:19.388 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:04:49.585 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:04:49.593 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:05:19.804 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:05:19.809 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:05:50.000 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:05:50.015 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:06:20.197 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:06:20.203 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:06:50.395 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:06:50.400 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:07:20.578 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:07:20.586 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:07:50.749 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:07:50.762 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:08:20.969 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:08:20.977 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:08:51.250 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:08:51.258 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:09:21.539 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:09:21.547 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:09:51.750 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:09:51.759 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:10:22.025 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:10:22.032 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:10:52.338 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:10:52.347 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:11:22.562 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:11:22.570 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:11:52.817 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:11:52.822 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:12:22.985 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:12:22.991 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:12:53.201 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:12:53.210 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:13:23.499 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:13:23.508 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:13:53.690 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:13:53.697 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:14:23.952 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:14:23.957 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:14:54.201 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:14:54.207 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:15:24.382 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:15:24.391 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:15:54.643 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:15:54.651 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:16:24.840 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:16:24.849 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:16:55.138 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:16:55.144 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:17:25.360 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:17:25.369 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:17:55.548 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:17:55.556 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:18:25.762 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:18:25.768 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:18:56.006 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:18:56.015 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:19:26.299 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:19:26.304 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:19:56.459 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:19:56.470 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:20:26.734 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:20:26.740 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:20:56.936 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:20:56.945 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:21:27.136 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:21:27.145 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:21:57.351 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:21:57.357 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:22:27.545 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:22:27.551 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:22:57.745 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:22:57.750 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:23:27.942 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:23:27.947 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:23:58.239 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:23:58.244 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:24:28.353 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:24:28.358 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:24:58.455 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:24:58.464 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:25:28.639 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:25:28.648 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:25:58.762 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:25:58.772 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:26:28.975 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:26:28.983 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:26:59.114 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:26:59.123 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:27:29.404 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:27:29.414 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:27:59.601 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:27:59.610 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:28:29.814 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:28:29.821 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:28:59.951 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:28:59.960 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:29:30.150 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:29:30.158 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:30:00.422 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:30:00.430 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:30:30.712 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:30:30.718 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:31:00.917 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:31:00.927 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:31:31.032 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:31:31.038 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:32:01.286 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:32:01.295 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:32:31.492 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:32:31.501 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:33:01.789 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:33:01.799 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:33:32.090 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:33:32.096 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:34:02.306 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:34:02.314 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:34:32.471 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:34:32.480 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:35:02.602 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:35:02.608 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:35:32.766 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:35:32.772 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:36:03.037 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:36:03.043 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:36:33.208 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:36:33.218 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:37:03.499 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:37:03.511 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:37:33.663 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:37:33.669 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:38:03.841 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:38:03.846 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:38:34.058 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:38:34.062 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:39:04.235 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:39:04.239 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:39:34.424 DEBUG:teuthology.orchestra.run.smithi111:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:39:34.429 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-03-29T22:40:03.902 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2023-03-29T22:40:03.903 DEBUG:teuthology.task.console_log:Killing console logger for smithi111 2023-03-29T22:40:03.904 DEBUG:teuthology.task.console_log:Killing console logger for smithi119 2023-03-29T22:40:03.904 DEBUG:teuthology.exit:Finished running handlers